BUG: Fix pandas-dev#57608 : queries on categorical string columns in

HDFStore.select() return unexpected results.
In function __init__() of class Selection (pandas/core/io/pytables.py),
the method self.terms.evaluate() was not returning the correct value
for the where condition. The issue stemmed from the function
convert_value() of class BinOp (pandas/core/computation/pytables.py),
where the function searchedsorted() did not return the correct index
when matching the where condition in the metadata (categories table).
Replacing searchsorted() with np.where() resolves this issue.

Author: SofiaSM45

doc/source/whatsnew/v3.0.0.rst

@@ -742,6 +742,7 @@ I/O
 - Bug in :meth:`read_stata` where the missing code for double was not recognised for format versions 105 and prior (:issue:`58149`)
 - Bug in :meth:`set_option` where setting the pandas option ``display.html.use_mathjax`` to ``False`` has no effect (:issue:`59884`)
 - Bug in :meth:`to_excel` where :class:`MultiIndex` columns would be merged to a single row when ``merge_cells=False`` is passed (:issue:`60274`)
+- Bug in :meth:`HDFStore.select` causing queries on categorical string columns to return unexpected results (:issue:`57608`)
 
 Period
 ^^^^^^

pandas/core/computation/pytables.py

@@ -239,7 +239,8 @@ def stringify(value):
             if conv_val not in metadata:
                 result = -1
             else:
-                result = metadata.searchsorted(conv_val, side="left")
+                # Find the index of the first match of conv_val in metadata
+                result = np.where(metadata == conv_val)[0][0]
             return TermValue(result, result, "integer")
         elif kind == "integer":
             try:

pandas/tests/io/pytables/test_store.py

@@ -1106,3 +1106,22 @@ def test_store_bool_index(tmp_path, setup_path):
     df.to_hdf(path, key="a")
     result = read_hdf(path, "a")
     tm.assert_frame_equal(expected, result)
+
+
+@pytest.mark.parametrize("model", ['name', 'longname', 'verylongname'])
+def test_select_categorical_string_columns(tmp_path, model):
+    # Corresponding to BUG: 57608
+
+    path = tmp_path / "test.h5"
+
+    models = pd.api.types.CategoricalDtype(categories=['name', 'longname', 'verylongname'])
+    df = pd.DataFrame({'modelId': ['name', 'longname', 'longname'],
+                       'value': [1, 2, 3]}).astype({'modelId': models, 'value': int})
+
+    with pd.HDFStore(path, 'w') as store:
+        store.append('df', df, data_columns=['modelId'])
+
+    with pd.HDFStore(path, 'r') as store:
+        result = store.select('df', 'modelId == model')
+        expected = df[df['modelId'] == model]
+        tm.assert_frame_equal(result, expected)