Merge branch 'main' into to_dataset-silently-drops

Author: max-sixty

.binder/environment.yml

@@ -2,7 +2,7 @@ name: xarray-examples
 channels:
   - conda-forge
 dependencies:
-  - python=3.9
+  - python=3.10
   - boto3
   - bottleneck
   - cartopy
@@ -25,7 +25,7 @@ dependencies:
   - numpy
   - packaging
   - pandas
-  - pint
+  - pint>=0.22
   - pip
   - pooch
   - pydap

ci/requirements/all-but-dask.yml

@@ -26,7 +26,7 @@ dependencies:
   - numpy
   - packaging
   - pandas
-  - pint<0.21
+  - pint>=0.22
   - pip
   - pseudonetcdf
   - pydap

ci/requirements/environment-py311.yml

@@ -28,7 +28,7 @@ dependencies:
   - numpy
   - packaging
   - pandas
-  - pint<0.21
+  - pint>=0.22
   - pip
   - pooch
   - pre-commit

ci/requirements/environment-windows-py311.yml

@@ -25,7 +25,7 @@ dependencies:
   - numpy
   - packaging
   - pandas
-  - pint<0.21
+  - pint>=0.22
   - pip
   - pre-commit
   - pseudonetcdf

ci/requirements/environment-windows.yml

@@ -25,7 +25,7 @@ dependencies:
   - numpy
   - packaging
   - pandas
-  - pint<0.21
+  - pint>=0.22
   - pip
   - pre-commit
   - pseudonetcdf

ci/requirements/environment.yml

@@ -29,7 +29,7 @@ dependencies:
   - opt_einsum
   - packaging
   - pandas
-  - pint<0.21
+  - pint>=0.22
   - pip
   - pooch
   - pre-commit

ci/requirements/min-all-deps.yml

@@ -35,7 +35,7 @@ dependencies:
   - numpy=1.22
   - packaging=21.3
   - pandas=1.4
-  - pint=0.19
+  - pint=0.22
   - pip
   - pseudonetcdf=3.2
   - pydap=3.3

doc/api.rst

@@ -557,6 +557,7 @@ Datetimelike properties
    DataArray.dt.seconds
    DataArray.dt.microseconds
    DataArray.dt.nanoseconds
+   DataArray.dt.total_seconds
 
 **Timedelta methods**:
 
@@ -602,7 +603,7 @@ Dataset methods
    Dataset.as_numpy
    Dataset.from_dataframe
    Dataset.from_dict
-   Dataset.to_array
+   Dataset.to_dataarray
    Dataset.to_dataframe
    Dataset.to_dask_dataframe
    Dataset.to_dict

doc/howdoi.rst

@@ -36,7 +36,7 @@ How do I ...
    * - rename a variable, dimension or coordinate
      - :py:meth:`Dataset.rename`, :py:meth:`DataArray.rename`, :py:meth:`Dataset.rename_vars`, :py:meth:`Dataset.rename_dims`,
    * - convert a DataArray to Dataset or vice versa
-     - :py:meth:`DataArray.to_dataset`, :py:meth:`Dataset.to_array`, :py:meth:`Dataset.to_stacked_array`, :py:meth:`DataArray.to_unstacked_dataset`
+     - :py:meth:`DataArray.to_dataset`, :py:meth:`Dataset.to_dataarray`, :py:meth:`Dataset.to_stacked_array`, :py:meth:`DataArray.to_unstacked_dataset`
    * - extract variables that have certain attributes
      - :py:meth:`Dataset.filter_by_attrs`
    * - extract the underlying array (e.g. NumPy or Dask arrays)

doc/user-guide/reshaping.rst

@@ -59,11 +59,11 @@ use :py:meth:`~xarray.DataArray.squeeze`
 Converting between datasets and arrays
 --------------------------------------
 
-To convert from a Dataset to a DataArray, use :py:meth:`~xarray.Dataset.to_array`:
+To convert from a Dataset to a DataArray, use :py:meth:`~xarray.Dataset.to_dataarray`:
 
 .. ipython:: python
 
-    arr = ds.to_array()
+    arr = ds.to_dataarray()
     arr
 
 This method broadcasts all data variables in the dataset against each other,
@@ -77,7 +77,7 @@ To convert back from a DataArray to a Dataset, use
 
     arr.to_dataset(dim="variable")
 
-The broadcasting behavior of ``to_array`` means that the resulting array
+The broadcasting behavior of ``to_dataarray`` means that the resulting array
 includes the union of data variable dimensions:
 
 .. ipython:: python
@@ -88,7 +88,7 @@ includes the union of data variable dimensions:
     ds2
 
     # the resulting array has 6 elements
-    ds2.to_array()
+    ds2.to_dataarray()
 
 Otherwise, the result could not be represented as an orthogonal array.
 
@@ -161,8 +161,8 @@ arrays as inputs. For datasets with only one variable, we only need ``stack``
 and ``unstack``, but combining multiple variables in a
 :py:class:`xarray.Dataset` is more complicated. If the variables in the dataset
 have matching numbers of dimensions, we can call
-:py:meth:`~xarray.Dataset.to_array` and then stack along the the new coordinate.
-But :py:meth:`~xarray.Dataset.to_array` will broadcast the dataarrays together,
+:py:meth:`~xarray.Dataset.to_dataarray` and then stack along the the new coordinate.
+But :py:meth:`~xarray.Dataset.to_dataarray` will broadcast the dataarrays together,
 which will effectively tile the lower dimensional variable along the missing
 dimensions. The method :py:meth:`xarray.Dataset.to_stacked_array` allows
 combining variables of differing dimensions without this wasteful copying while

doc/whats-new.rst

@@ -24,10 +24,13 @@ New Features
 
 - Use `opt_einsum <https://optimized-einsum.readthedocs.io/en/stable/>`_ for :py:func:`xarray.dot` by default if installed.
   By `Deepak Cherian <https://github.com/dcherian>`_. (:issue:`7764`, :pull:`8373`).
+- Add ``DataArray.dt.total_seconds()`` method to match the Pandas API. (:pull:`8435`).
+  By `Ben Mares <https://github.com/maresb>`_.
 
 Breaking changes
 ~~~~~~~~~~~~~~~~
 
+- Bump minimum tested pint version to ``>=0.22``. By `Deepak Cherian <https://github.com/dcherian>`_.
 
 Deprecations
 ~~~~~~~~~~~~
@@ -39,6 +42,15 @@ Deprecations
   this was one place in the API where dimension positions were used.
   (:pull:`8341`)
   By `Maximilian Roos <https://github.com/max-sixty>`_.
+- Rename :py:meth:`Dataset.to_array` to  :py:meth:`Dataset.to_dataarray` for
+  consistency with :py:meth:`DataArray.to_dataset` &
+  :py:func:`open_dataarray` functions. This is a "soft" deprecation — the
+  existing methods work and don't raise any warnings, given the relatively small
+  benefits of the change.
+  By `Maximilian Roos <https://github.com/max-sixty>`_.
+- Finally remove ``keep_attrs`` kwarg from :py:meth:`DataArray.resample` and
+  :py:meth:`Dataset.resample`. These were deprecated a long time ago.
+  By `Deepak Cherian <https://github.com/dcherian>`_.
 
 Bug fixes
 ~~~~~~~~~
@@ -6710,7 +6722,7 @@ Backwards incompatible changes
 Enhancements
 ~~~~~~~~~~~~
 
-- New ``xray.Dataset.to_array`` and enhanced
+- New ``xray.Dataset.to_dataarray`` and enhanced
   ``xray.DataArray.to_dataset`` methods make it easy to switch back
   and forth between arrays and datasets:
 
@@ -6721,8 +6733,8 @@ Enhancements
           coords={"c": 42},
           attrs={"Conventions": "None"},
       )
-      ds.to_array()
-      ds.to_array().to_dataset(dim="variable")
+      ds.to_dataarray()
+      ds.to_dataarray().to_dataset(dim="variable")
 
 - New ``xray.Dataset.fillna`` method to fill missing values, modeled
   off the pandas method of the same name:

xarray/core/_typed_ops.py

@@ -83,6 +83,10 @@ def __eq__(self, other: DsCompatible) -> Self:  # type:ignore[override]
     def __ne__(self, other: DsCompatible) -> Self:  # type:ignore[override]
         return self._binary_op(other, nputils.array_ne)
 
+    # When __eq__ is defined but __hash__ is not, then an object is unhashable,
+    # and it should be declared as follows:
+    __hash__: None  # type:ignore[assignment]
+
     def __radd__(self, other: DsCompatible) -> Self:
         return self._binary_op(other, operator.add, reflexive=True)
 
@@ -291,6 +295,10 @@ def __eq__(self, other: DaCompatible) -> Self:  # type:ignore[override]
     def __ne__(self, other: DaCompatible) -> Self:  # type:ignore[override]
         return self._binary_op(other, nputils.array_ne)
 
+    # When __eq__ is defined but __hash__ is not, then an object is unhashable,
+    # and it should be declared as follows:
+    __hash__: None  # type:ignore[assignment]
+
     def __radd__(self, other: DaCompatible) -> Self:
         return self._binary_op(other, operator.add, reflexive=True)
 
@@ -643,6 +651,10 @@ def __ne__(self, other: VarCompatible) -> Self:
     def __ne__(self, other: VarCompatible) -> Self | T_DataArray:
         return self._binary_op(other, nputils.array_ne)
 
+    # When __eq__ is defined but __hash__ is not, then an object is unhashable,
+    # and it should be declared as follows:
+    __hash__: None  # type:ignore[assignment]
+
     def __radd__(self, other: VarCompatible) -> Self:
         return self._binary_op(other, operator.add, reflexive=True)
 
@@ -851,6 +863,10 @@ def __eq__(self, other: GroupByCompatible) -> Dataset:  # type:ignore[override]
     def __ne__(self, other: GroupByCompatible) -> Dataset:  # type:ignore[override]
         return self._binary_op(other, nputils.array_ne)
 
+    # When __eq__ is defined but __hash__ is not, then an object is unhashable,
+    # and it should be declared as follows:
+    __hash__: None  # type:ignore[assignment]
+
     def __radd__(self, other: GroupByCompatible) -> Dataset:
         return self._binary_op(other, operator.add, reflexive=True)
 
@@ -973,6 +989,10 @@ def __eq__(self, other: T_Xarray) -> T_Xarray:  # type:ignore[override]
     def __ne__(self, other: T_Xarray) -> T_Xarray:  # type:ignore[override]
         return self._binary_op(other, nputils.array_ne)
 
+    # When __eq__ is defined but __hash__ is not, then an object is unhashable,
+    # and it should be declared as follows:
+    __hash__: None  # type:ignore[assignment]
+
     def __radd__(self, other: T_Xarray) -> T_Xarray:
         return self._binary_op(other, operator.add, reflexive=True)
 

xarray/core/accessor_dt.py

@@ -74,6 +74,8 @@ def _access_through_series(values, name):
     if name == "season":
         months = values_as_series.dt.month.values
         field_values = _season_from_months(months)
+    elif name == "total_seconds":
+        field_values = values_as_series.dt.total_seconds().values
     elif name == "isocalendar":
         # special NaT-handling can be removed when
         # https://github.com/pandas-dev/pandas/issues/54657 is resolved
@@ -574,6 +576,13 @@ class TimedeltaAccessor(TimeAccessor[T_DataArray]):
            43200, 64800])
     Coordinates:
       * time     (time) timedelta64[ns] 1 days 00:00:00 ... 5 days 18:00:00
+    >>> ts.dt.total_seconds()
+    <xarray.DataArray 'total_seconds' (time: 20)>
+    array([ 86400., 108000., 129600., 151200., 172800., 194400., 216000.,
+           237600., 259200., 280800., 302400., 324000., 345600., 367200.,
+           388800., 410400., 432000., 453600., 475200., 496800.])
+    Coordinates:
+      * time     (time) timedelta64[ns] 1 days 00:00:00 ... 5 days 18:00:00
     """
 
     @property
@@ -596,6 +605,11 @@ def nanoseconds(self) -> T_DataArray:
         """Number of nanoseconds (>= 0 and less than 1 microsecond) for each element"""
         return self._date_field("nanoseconds", np.int64)
 
+    # Not defined as a property in order to match the Pandas API
+    def total_seconds(self) -> T_DataArray:
+        """Total duration of each element expressed in seconds."""
+        return self._date_field("total_seconds", np.float64)
+
 
 class CombinedDatetimelikeAccessor(
     DatetimeAccessor[T_DataArray], TimedeltaAccessor[T_DataArray]

xarray/core/common.py

@@ -860,7 +860,6 @@ def _resample(
         base: int | None,
         offset: pd.Timedelta | datetime.timedelta | str | None,
         origin: str | DatetimeLike,
-        keep_attrs: bool | None,
         loffset: datetime.timedelta | str | None,
         restore_coord_dims: bool | None,
         **indexer_kwargs: str,
@@ -989,13 +988,6 @@ def _resample(
         from xarray.core.pdcompat import _convert_base_to_offset
         from xarray.core.resample import RESAMPLE_DIM
 
-        if keep_attrs is not None:
-            warnings.warn(
-                "Passing ``keep_attrs`` to ``resample`` has no effect and will raise an"
-                " error in xarray 0.20. Pass ``keep_attrs`` directly to the applied"
-                " function, e.g. ``resample(...).mean(keep_attrs=True)``."
-            )
-
         # note: the second argument (now 'skipna') use to be 'dim'
         if (
             (skipna is not None and not isinstance(skipna, bool))
@@ -1173,7 +1165,7 @@ def _dataset_indexer(dim: Hashable) -> DataArray:
                     var for var in cond if dim not in cond[var].dims
                 )
                 keepany = cond_wdim.any(dim=(d for d in cond.dims.keys() if d != dim))
-                return keepany.to_array().any("variable")
+                return keepany.to_dataarray().any("variable")
 
             _get_indexer = (
                 _dataarray_indexer if isinstance(cond, DataArray) else _dataset_indexer

xarray/core/computation.py

@@ -1603,7 +1603,9 @@ def cross(
     >>> ds_a = xr.Dataset(dict(x=("dim_0", [1]), y=("dim_0", [2]), z=("dim_0", [3])))
     >>> ds_b = xr.Dataset(dict(x=("dim_0", [4]), y=("dim_0", [5]), z=("dim_0", [6])))
     >>> c = xr.cross(
-    ...     ds_a.to_array("cartesian"), ds_b.to_array("cartesian"), dim="cartesian"
+    ...     ds_a.to_dataarray("cartesian"),
+    ...     ds_b.to_dataarray("cartesian"),
+    ...     dim="cartesian",
     ... )
     >>> c.to_dataset(dim="cartesian")
     <xarray.Dataset>

xarray/core/dataarray.py

@@ -7034,7 +7034,6 @@ def resample(
         base: int | None = None,
         offset: pd.Timedelta | datetime.timedelta | str | None = None,
         origin: str | DatetimeLike = "start_day",
-        keep_attrs: bool | None = None,
         loffset: datetime.timedelta | str | None = None,
         restore_coord_dims: bool | None = None,
         **indexer_kwargs: str,
@@ -7156,7 +7155,6 @@ def resample(
             base=base,
             offset=offset,
             origin=origin,
-            keep_attrs=keep_attrs,
             loffset=loffset,
             restore_coord_dims=restore_coord_dims,
             **indexer_kwargs,