Skip to content

Commit 98b287b

Browse files
update backends tutorials intro (#266)
* add draft chart to markdown file * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add mermaid sphinx extension * revise backend content * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add some color styling * add sphinx mermaid package to env * save conflict changes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update conda lock files * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add myst parser to use mermaid lexer with sphinx * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix dash to underscore * try changing order of extensions * add source_suffix sphinx config * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add curly braces * remove diagram (moved to docs) and update text * revert environment changes to main * add h5netcdf to list of backends --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
1 parent 516bcec commit 98b287b

File tree

1 file changed

+29
-24
lines changed

1 file changed

+29
-24
lines changed

advanced/backends/backends.md

+29-24
Original file line numberDiff line numberDiff line change
@@ -1,44 +1,49 @@
1-
# Creating new backends
1+
# Reading data using backends
22

33
## Introduction
44

5-
You can [read different type of files](https://docs.xarray.dev/en/stable/user-guide/io.html)) in `xr.open_dataset` by specifying the engine to be used:
5+
You can [read different types of files](https://docs.xarray.dev/en/stable/user-guide/io.html) in `xr.open_dataset` by specifying the engine to be used:
66

77
```python
88
import xarray as xr
99
xr.open_dataset("my_file.grib" , engine="cfgrib")
1010
```
1111

12-
For each available engine there is an underlying backend, that reads the data and pack them in a dataset.
12+
Navigating Xarray backends can be confusing,
13+
so we recommend checking out [this flow chart](https://docs.xarray.dev/en/stable/user-guide/io.html)
14+
to help you figure out which engine you need and how to use it.
1315

14-
Xarray bundles several backends internally for the following formats:
16+
You can see what backends are currently available in your working environment
17+
with `xarray.backends.list_engines()`.
1518

16-
- netcdf4 - netCDF4
17-
- scipy - netCDF3
18-
- zarr - Zarr
19-
- pydap - DAP
20-
- ...
19+
## Why use the Xarray backend API to write your own backend?
2120

22-
External Backends that use the new backend API (xarray >= v0.18.0) that allows to add support for backend without any change to Xarray
21+
- Your users don't need to learn a new interface; they can use `xr.open_dataset` with the `engine` kwarg.
22+
- With little extra effort you can have lazy loading with Dask. Simply implement a function for reading blocks and Xarray will manage lazy loading with Dask for you.
23+
- It's easy to implement: using the backend API (introduced in v0.18.0), you don't need to integrate any code in Xarray.
2324

24-
- [cfgrib](https://github.com/ecmwf/cfgrib) - GRIB
25-
- [tiledb](https://github.com/TileDB-Inc/TileDB-CF-Py) - TileDB
26-
- [rioxarray](https://corteva.github.io/rioxarray/stable/) - GeoTIFF, JPEG-2000, ESRI-hdr, etc (via GDAL)
27-
- [xarray-sentinel](https://github.com/bopen/xarray-sentinel) - Sentinel-1 SAFE
28-
- ...
25+
## More Information
2926

30-
## Why using the Xarray backend API
27+
See the [documentation](https://docs.xarray.dev/en/stable/internals/how-to-add-new-backend.html) for more details on adding and registering a new backend.
3128

32-
- Your users don't need to learn a new interface that is they can use `xr.open_dataset` with `engine` kwarg.
33-
- With little extra effort you can have lazy loading with Dask. you have to implement a function for reading blocks and Xarray will manage lazy loading with Dask for you
34-
- It's easy to implement: you don't need to integrate any code in Xarray
29+
Follow the tutorials on creating a new backend for binary files.
30+
31+
```{tableofcontents}
3532
36-
## Next
33+
```
3734

38-
See the [documentation](https://docs.xarray.dev/en/stable/internals/how-to-add-new-backend.html) for more.
35+
### Links to internal backends
3936

40-
Follow the tutorial on creating a new backend for binary files.
37+
- [netcdf4](https://pypi.org/project/netCDF4/) - netCDF4
38+
- [scipy](https://scipy.org/) - netCDF3
39+
- [zarr](https://pypi.org/project/zarr/) - Zarr
40+
- [pydap](https://pydap.github.io/pydap/) - Data Access Protocol (DAP/DODS/OPeNDAP)
41+
- [h5netcdf](https://h5netcdf.org/) - hdf5
4142

42-
```{tableofcontents}
43+
### Links to external backends (not comprehensive)
4344

44-
```
45+
- [cfgrib](https://github.com/ecmwf/cfgrib) - GRIB
46+
- [tiledb](https://github.com/TileDB-Inc/TileDB-CF-Py) - TileDB
47+
- [rioxarray](https://corteva.github.io/rioxarray/stable/) - GeoTIFF, JPEG-2000, ESRI-hdr, etc (via GDAL)
48+
- [xarray-sentinel](https://github.com/bopen/xarray-sentinel) - Sentinel-1 SAFE
49+
- ...

0 commit comments

Comments
 (0)