Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Fix Series comparison fails when index dtypes differ (object vs string) (#61099) #61199

Closed
wants to merge 15 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
.tags
.cache/
.vscode/
.venv/

# Compiled source #
###################
Expand Down Expand Up @@ -133,6 +134,7 @@ doc/build/html/index.html
doc/tmp.sv
env/
doc/source/savefig/
venv/

# Interactive terminal generated files #
########################################
Expand Down

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
/*!
* Bootstrap v5.3.3 (https://getbootstrap.com/)
* Copyright 2011-2024 The Bootstrap Authors (https://github.com/twbs/bootstrap/graphs/contributors)
* Licensed under MIT (https://github.com/twbs/bootstrap/blob/main/LICENSE)
*/

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
/*!
* Font Awesome Free 6.5.2 by @fontawesome - https://fontawesome.com
* License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License)
* Copyright 2024 Fonticons, Inc.
*/

Large diffs are not rendered by default.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
<!--
AUTO-GENERATED from webpack.config.js, do **NOT** edit by hand.
These are re-used in layout.html
-->

{% macro head_pre_assets() %}
<!-- Loaded before other Sphinx assets -->
<link href="{{ pathto('_static/styles/theme.css', 1) }}?digest=26a4bc78f4c0ddb94549" rel="stylesheet" />
<link href="{{ pathto('_static/styles/pydata-sphinx-theme.css', 1) }}?digest=26a4bc78f4c0ddb94549" rel="stylesheet" />
{% endmacro %}

{% macro head_js_preload() %}
<!-- So that users can add custom icons -->
<script src="{{ pathto('_static/scripts/fontawesome.js', 1) }}?digest=26a4bc78f4c0ddb94549"></script>
<!-- Pre-loaded scripts that we'll load fully later -->
<link rel="preload" as="script" href="{{ pathto('_static/scripts/bootstrap.js', 1) }}?digest=26a4bc78f4c0ddb94549" />
<link rel="preload" as="script" href="{{ pathto('_static/scripts/pydata-sphinx-theme.js', 1) }}?digest=26a4bc78f4c0ddb94549" />
{% endmacro %}

{% macro body_post() %}
<!-- Scripts loaded after <body> so the DOM is not blocked -->
<script defer src="{{ pathto('_static/scripts/bootstrap.js', 1) }}?digest=26a4bc78f4c0ddb94549"></script>
<script defer src="{{ pathto('_static/scripts/pydata-sphinx-theme.js', 1) }}?digest=26a4bc78f4c0ddb94549"></script>
{% endmacro %}
1 change: 1 addition & 0 deletions doc/source/whatsnew/v3.0.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -706,6 +706,7 @@ Indexing
- Bug in :meth:`MultiIndex.insert` when a new value inserted to a datetime-like level gets cast to ``NaT`` and fails indexing (:issue:`60388`)
- Bug in printing :attr:`Index.names` and :attr:`MultiIndex.levels` would not escape single quotes (:issue:`60190`)
- Bug in reindexing of :class:`DataFrame` with :class:`PeriodDtype` columns in case of consolidated block (:issue:`60980`, :issue:`60273`)
- Bug in :meth:`Series.equals` and :meth:`Series.__eq__` where equality comparison would fail if the index values were equal but had differing dtypes (e.g. ``object`` vs ``string``). (:issue:`61099`)

Missing
^^^^^^^
Expand Down
5 changes: 5 additions & 0 deletions pandas/core/indexes/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -5471,6 +5471,11 @@ def equals(self, other: Any) -> bool:
# quickly return if the lengths are different
return False

if (isinstance(self.dtype, StringDtype) or is_object_dtype(self.dtype)) and (
isinstance(other.dtype, StringDtype) or is_object_dtype(other.dtype)
):
return array_equivalent(self._values, other._values)

if (
isinstance(self.dtype, StringDtype)
and self.dtype.na_value is np.nan
Expand Down
36 changes: 36 additions & 0 deletions pandas/tests/indexes/test_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
ensure_index,
ensure_index_from_sequences,
)
from pandas.testing import assert_series_equal


class TestIndex:
Expand Down Expand Up @@ -1717,3 +1718,38 @@ def test_is_monotonic_pyarrow_list_type():
idx = Index([[1], [2, 3]], dtype=pd.ArrowDtype(pa.list_(pa.int64())))
assert not idx.is_monotonic_increasing
assert not idx.is_monotonic_decreasing


def test_index_equals_string_vs_object():
# GH 61099
idx1 = Index(["a", "b", "c"])
idx2 = Index(["a", "b", "c"], dtype="string")
assert idx1.equals(idx2)
assert idx2.equals(idx1)


def test_compare_string_vs_object_index_equality():
# GH 61099
s1 = Series([1, 2, 3], index=["a", "b", "c"]) # dtype=object
s2 = Series([0, 1, 2], index=Index(["a", "b", "c"], dtype="string")) # dtype=string
result = s1 > s2
expected = Series([True, True, True], index=["a", "b", "c"])
assert_series_equal(result, expected)


def test_align_string_vs_object_index():
# GH 61099
s1 = Series([1, 2, 3], index=["a", "b", "c"]) # object
s2 = Series([1, 2, 3], index=Index(["a", "b", "c"], dtype="string")) # string
s1_aligned, s2_aligned = s1.align(s2)
assert list(s1_aligned.index) == list(s2_aligned.index)


def test_comparison_without_manual_casting():
# GH 61099
s1 = Series([1, 2, 3], index=["a", "b", "c"]) # object index
s2 = Series([1, 2, 3], index=Index(["a", "b", "c"], dtype="string"))
# Should not raise
result = s1 == s2
expected = Series([True, True, True], index=["a", "b", "c"])
assert_series_equal(result, expected)
Loading