Skip to content

Add dtype support for reduce methods. #1841

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jan 20, 2018
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion doc/whats-new.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,9 @@ Documentation

Enhancements
~~~~~~~~~~~~

- reduce methods such as :py:func:`DataArray.sum()` now accepts ``dtype``
arguments. (:issue:`1838`)
By `Keisuke Fujii <https://github.com/fujiisoup>`_.
- :py:func:`~plot.contourf()` learned to contour 2D variables that have both a
1D co-ordinate (e.g. time) and a 2D co-ordinate (e.g. depth as a function of
time) (:issue:`1737`).
Expand Down
12 changes: 7 additions & 5 deletions xarray/core/duck_array_ops.py
Original file line number Diff line number Diff line change
Expand Up @@ -175,11 +175,11 @@ def _create_nan_agg_method(name, numeric_only=False, np_compat=False,
no_bottleneck=False, coerce_strings=False,
keep_dims=False):
def f(values, axis=None, skipna=None, **kwargs):
# ignore keyword args inserted by np.mean and other numpy aggregators
# automatically:
kwargs.pop('dtype', None)
kwargs.pop('out', None)
if kwargs.pop('out', None) is not None:
raise ValueError('`out` is not valid for {}'.format(name))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably be TypeError for invalid argument names.


# If dtype is supplied, we use numpy's method.
dtype = kwargs.get('dtype', None)
values = asarray(values)

if coerce_strings and values.dtype.kind in 'SU':
Expand All @@ -192,14 +192,16 @@ def f(values, axis=None, skipna=None, **kwargs):
% (name, values.dtype))
nanname = 'nan' + name
if (isinstance(axis, tuple) or not values.dtype.isnative or
no_bottleneck):
no_bottleneck or
(dtype is not None and np.dtype(dtype) != values.dtype)):
# bottleneck can't handle multiple axis arguments or non-native
# endianness
if np_compat:
eager_module = npcompat
else:
eager_module = np
else:
kwargs.pop('dtype', None)
eager_module = bn
func = _dask_or_eager_func(nanname, eager_module)
using_numpy_nan_func = (eager_module is np or
Expand Down
18 changes: 18 additions & 0 deletions xarray/tests/test_dataarray.py
Original file line number Diff line number Diff line change
Expand Up @@ -1784,6 +1784,24 @@ def test_reduce(self):
dims=['x', 'y']).mean('x')
assert_equal(actual, expected)

def test_reduce_dtype(self):
coords = {'x': [-1, -2], 'y': ['ab', 'cd', 'ef'],
'lat': (['x', 'y'], [[1, 2, 3], [-1, -2, -3]]),
'c': -999}
orig = DataArray([[-1, 0, 1], [-3, 0, 3]], coords, dims=['x', 'y'])

for dtype in [np.float32, np.float64, np.float128]:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like you'll need to remove np.float128. It appears that it's not available on Windows, which is triggering this test failure.

assert orig.astype(float).mean(dtype=dtype).dtype == dtype

def test_reduce_out(self):
coords = {'x': [-1, -2], 'y': ['ab', 'cd', 'ef'],
'lat': (['x', 'y'], [[1, 2, 3], [-1, -2, -3]]),
'c': -999}
orig = DataArray([[-1, 0, 1], [-3, 0, 3]], coords, dims=['x', 'y'])

with pytest.raises(ValueError):
orig.mean(out=np.ones(orig.shape))

# skip due to bug in older versions of numpy.nanpercentile
def test_quantile(self):
for q in [0.25, [0.50], [0.25, 0.75]]:
Expand Down