Skip to content

DataFrame.apply() does not retain category dtype if applied along axis 1 #23875

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
JackCaster opened this issue Nov 23, 2018 · 1 comment
Closed

Comments

@JackCaster
Copy link

Code Sample, a copy-pastable example if possible

import pandas as pd

def transform(s):
    return s

df = pd.DataFrame({'a':range(10), 'b':range(10)})

df['a'] = df['a'].astype('category')
print("(Before apply) df['a'] dtype is {}".format(df['a'].dtype)) # -> return category

df = df.apply(transform, axis=1)
print("(After apply) df['a'] dtype is {}".format(df['a'].dtype)) # -> return int64

Problem description

apply() does not retain the categorical dtype if applied on columns, but it does retain the dtype if applied on rows (set axis=0 in the snippet above).

Expected Output

The method should return a categorical dtype.

Output of pd.show_versions()

[paste the output of pd.show_versions() here below this line]

INSTALLED VERSIONS

commit: None
python: 3.6.7.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 69 Stepping 1, GenuineIntel
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: None.None

pandas: 0.23.4
pytest: None
pip: 18.1
setuptools: 40.5.0
Cython: None
numpy: 1.15.4
scipy: 1.1.0
pyarrow: None
xarray: None
IPython: 7.1.1
sphinx: None
patsy: 0.5.1
dateutil: 2.7.5
pytz: 2018.7
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 3.0.1
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

@TomAugspurger
Copy link
Contributor

When you apply a function to the rows, pandas passes each row as a Series.

So transform gets series like

In [27]: df.iloc[0]
Out[27]:
a    0
b    0
Name: 0, dtype: object

As you can se, that's object dtype, so the type information isn't available any more, regardless of what transform does.

@TomAugspurger TomAugspurger added this to the No action milestone Nov 24, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants