This repository was archived by the owner on Oct 24, 2024. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 42
Added docs page on io #158
Merged
Merged
Changes from 2 commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
ab998f1
added page on io
TomNicholas 3188c78
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 69bb8e3
whatsnew
TomNicholas 373e085
Merge branch 'io_docs' of https://github.com/TomNicholas/datatree int…
TomNicholas f5c4ea8
Merge branch 'main' into io_docs
TomNicholas 878f9f4
fix typo which inverted meaning of key sentence
TomNicholas 0eb5c39
fix headings and link
TomNicholas a1d712a
add missing links
TomNicholas 293bd69
fix final link
TomNicholas File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
.. _data structures: | ||
|
||
Reading and Writing Files | ||
======================== | ||
|
||
.. note:: | ||
|
||
This page builds on the information given in xarray's main page on | ||
`reading and writing files <https://docs.xarray.dev/en/stable/user-guide/io.html>`_, | ||
so it is suggested that you are familiar with those first. | ||
|
||
|
||
netCDF | ||
====== | ||
|
||
Groups | ||
------ | ||
|
||
Whilst netCDF groups can only be loaded individually as Dataset objects, a whole file of many nested groups can be loaded | ||
as a single ``:py:class::DataTree`` object. | ||
To open a whole netCDF file as a tree of groups use the ``:py:func::open_datatree()`` function. | ||
To save a DataTree object as a netCDF file containing many groups, use the ``:py:meth::DataTree.to_netcdf()`` method. | ||
|
||
|
||
.. _netcdf datatree group warning | ||
|
||
.. warning:: | ||
``DataTree`` objects do not follow the exact same data model as netCDF files, which means that perfect round-tripping | ||
is not always possible. | ||
|
||
In particular in the netCDF data model dimensions are entities that can exist regardless of whether any variable possesses them. | ||
This is in contrast to `xarray's data model`_ (and hence `datatree's data model`_) in which the dimensions of a (Dataset/Tree) | ||
object are simply the set of dimensions present across all variables in that dataset. | ||
This means that if a netCDF file contains dimensions but variables which possess those dimensions, | ||
these dimensions will not be present when that file is opened as a DataTree object. | ||
Saving this DataTree object to file will therefore not preserve these "unused" dimensions. | ||
|
||
Zarr | ||
==== | ||
|
||
Groups | ||
------ | ||
|
||
Nested groups in zarr stores can be represented by loading the store as a ``:py:class::DataTree`` object, similarly to netCDF. | ||
To open a whole zarr store as a tree of groups use the ``:py:func::open_datatree()`` function. | ||
To save a DataTree object as a zarr store containing many groups, use the ``:py:meth::DataTree.to_zarr()`` method. | ||
|
||
.. note:: | ||
Note that perfect round-tripping should always be possible with a zarr store (:ref:`unlike for netCDF files<netcdf datatree group warning>`), | ||
as zarr does not support "unused" dimensions. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @jhamman can you confirm this statement about zarr is true? I couldn't really tell from looking at the v2 spec quickly. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is correct. |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.