Skip to content

Commit 1619cab

Browse files
BUG: Preserve extension dtypes in MultiIndex levels during concat (pandas-dev#58421)
1 parent e8eabea commit 1619cab

File tree

3 files changed

+14
-10
lines changed

3 files changed

+14
-10
lines changed

doc/source/whatsnew/v3.0.0.rst

+1
Original file line numberDiff line numberDiff line change
@@ -712,6 +712,7 @@ MultiIndex
712712
- :func:`MultiIndex.get_level_values` accessing a :class:`DatetimeIndex` does not carry the frequency attribute along (:issue:`58327`, :issue:`57949`)
713713
- Bug in :class:`DataFrame` arithmetic operations in case of unaligned MultiIndex columns (:issue:`60498`)
714714
- Bug in :class:`DataFrame` arithmetic operations with :class:`Series` in case of unaligned MultiIndex (:issue:`61009`)
715+
- Fixed a bug where extension dtypes like ``timestamp[pyarrow]`` were not preserved when building ``MultiIndex`` levels during ``pd.concat`` operations. (:issue:`58421`)
715716
-
716717

717718
I/O

pandas/core/reshape/concat.py

+2-4
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@
2222

2323
from pandas.core.dtypes.common import (
2424
is_bool,
25+
is_extension_array_dtype,
2526
is_scalar,
2627
)
2728
from pandas.core.dtypes.concat import concat_compat
@@ -36,6 +37,7 @@
3637
factorize_from_iterables,
3738
)
3839
import pandas.core.common as com
40+
from pandas.core.construction import array
3941
from pandas.core.indexes.api import (
4042
Index,
4143
MultiIndex,
@@ -47,10 +49,6 @@
4749
)
4850
from pandas.core.internals import concatenate_managers
4951

50-
from pandas.core.dtypes.common import is_extension_array_dtype
51-
52-
from pandas.core.construction import array
53-
5452
if TYPE_CHECKING:
5553
from collections.abc import (
5654
Callable,

pandas/tests/frame/methods/test_concat_arrow_index.py

+11-6
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,14 @@
1-
import pandas as pd
21
import pytest
32

3+
import pandas as pd
44

55
schema = {
66
"id": "int64[pyarrow]",
77
"time": "timestamp[s][pyarrow]",
88
"value": "float[pyarrow]",
99
}
1010

11+
1112
@pytest.mark.parametrize("dtype", ["timestamp[s][pyarrow]"])
1213
def test_concat_preserves_pyarrow_timestamp(dtype):
1314
dfA = (
@@ -38,9 +39,13 @@ def test_concat_preserves_pyarrow_timestamp(dtype):
3839
)
3940

4041
df = pd.concat([dfA, dfB], keys=[0, 1], names=["run"])
41-
42-
# chech whether df.index is multiIndex
43-
assert isinstance(df.index, pd.MultiIndex), f"Expected MultiIndex, but received {type(df.index)}"
44-
42+
43+
# check whether df.index is multiIndex
44+
assert isinstance(df.index, pd.MultiIndex), (
45+
f"Expected MultiIndex, but received {type(df.index)}"
46+
)
47+
4548
# Verifying special dtype timestamp[s][pyarrow] stays intact after concat
46-
assert df.index.levels[2].dtype == dtype, f"Expected {dtype}, but received {df.index.levels[2].dtype}"
49+
assert df.index.levels[2].dtype == dtype, (
50+
f"Expected {dtype}, but received {df.index.levels[2].dtype}"
51+
)

0 commit comments

Comments
 (0)