Skip to content

ENH: standardize fill_value behavior across the API #15587

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from
Closed
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 31 additions & 1 deletion pandas/core/missing.py
Original file line number Diff line number Diff line change
@@ -12,7 +12,11 @@
is_float_dtype, is_datetime64_dtype,
is_datetime64tz_dtype, is_integer_dtype,
_ensure_float64, is_scalar,
needs_i8_conversion, is_integer)
needs_i8_conversion, is_integer,
is_list_like, is_dict_like,
is_numeric_dtype,
is_datetime64_any_dtype, is_float,
is_complex, is_timedelta64_dtype)
from pandas.types.missing import isnull


@@ -621,3 +625,29 @@ def fill_zeros(result, x, y, name, fill):
result = result.reshape(shape)

return result


def validate_fill_value(value, dtype):
if is_list_like(value) or is_dict_like(value) or callable(value):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add a doc-string :>

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why dont' you check not is_scalar? (which allows strings, datetimes, and all pandas scalars).

raise TypeError('"fill_value" parameter must be '
'a scalar, but you passed a '
'"{0}"'.format(type(value).__name__))
elif not isnull(value):
from datetime import datetime, timedelta
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

put imports at the top of the file


if is_numeric_dtype(dtype):
if not (is_float(value) or is_integer(value) or is_complex(value)):
raise TypeError('"fill_value" parameter must be '
'numeric, but you passed a '
'"{0}"'.format(type(value).__name__))
elif is_datetime64_any_dtype(dtype):
if not isinstance(value, (np.datetime64, datetime)):
raise TypeError('"fill_value" parameter must be a '
'datetime, but you passed a '
'"{0}"'.format(type(value).__name__))
elif is_timedelta64_dtype(dtype):
if not isinstance(value, (np.timedelta64, timedelta)):
raise TypeError('"value" parameter must be '
'a timedelta, but you passed a '
'"{0}"'.format(type(value).__name__))
# if object dtype, do nothing.
4 changes: 4 additions & 0 deletions pandas/core/reshape.py
Original file line number Diff line number Diff line change
@@ -405,6 +405,10 @@ def _slow_pivot(index, columns, values):


def unstack(obj, level, fill_value=None):
if fill_value:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually I would always pass this (we will make None an acceptable fill_value below)

from pandas.core.missing import validate_fill_value
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

import at the top

validate_fill_value(fill_value, obj.values.dtype)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just pass obj.dtype, we never explictly call .values

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

obj may be a DataFrame AFAIK. I call values to get the numpy array here to consolidate the dtype (which means that e.g. a DataFrame with columns of mixed type will accept fill_value according to object rules). Is there a way to get this without accessing the underlying array directly?


if isinstance(level, (tuple, list)):
return _unstack_multiple(obj, level)