Skip to content

Commit 3dcf7f0

Browse files
author
Adrien Di Mascio
committed
BUG: close hdf store on any error
In `read_hdf`, if the `store.select()` call throws either a `ValueError`, a `TypeError` or a `KeyError` then the store is closed. However, if any other exception is thrown (e.g. an `AssertionError` if there are gaps in blocks ref_loc) , the store is not closed and some client code could end up trying to reopen the store and hit an error like: `the file XXX is already opened. Please close it before reopening in write mode`. Furthermore, the exception is re-raised in all cases. This commit just closes store in all cases. Closes #28430
1 parent 6b189d7 commit 3dcf7f0

File tree

3 files changed

+26
-9
lines changed

3 files changed

+26
-9
lines changed

doc/source/whatsnew/v1.0.0.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -727,6 +727,7 @@ I/O
727727
- Bug in :meth:`DataFrame.to_clipboard` which did not work reliably in ipython (:issue:`22707`)
728728
- Bug in :func:`read_json` where default encoding was not set to ``utf-8`` (:issue:`29565`)
729729
- Bug in :class:`PythonParser` where str and bytes were being mixed when dealing with the decimal field (:issue:`29650`)
730+
- :meth:`read_hdf` now closes the store on any error thrown (:issue:`28430`)
730731
-
731732

732733
Plotting

pandas/io/pytables.py

Lines changed: 6 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -426,15 +426,12 @@ def read_hdf(
426426
chunksize=chunksize,
427427
auto_close=auto_close,
428428
)
429-
except (ValueError, TypeError, KeyError):
430-
if not isinstance(path_or_buf, HDFStore):
431-
# if there is an error, close the store if we opened it.
432-
try:
433-
store.close()
434-
except AttributeError:
435-
pass
436-
437-
raise
429+
finally:
430+
# if there is an error, close the store if we opened it.
431+
try:
432+
store.close()
433+
except AttributeError:
434+
pass
438435

439436

440437
def _is_metadata_of(group: "Node", parent_group: "Node") -> bool:

pandas/tests/io/pytables/test_store.py

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -447,6 +447,25 @@ def check_default_mode():
447447
check("w")
448448
check_default_mode()
449449

450+
def test_store_closed_on_error(self, mocker):
451+
"""check hdf store is closed whichever error occurs"""
452+
mocker.patch("pandas.io.pytables.HDFStore.select").side_effect = AssertionError(
453+
"Gaps in blk ref_locs"
454+
)
455+
df = tm.makeDataFrame()
456+
with ensure_clean_path(self.path) as path:
457+
df.to_hdf(path, "df")
458+
try:
459+
pd.read_hdf(path, "df")
460+
except AssertionError: # this is expected, because of our mock
461+
pass
462+
try:
463+
HDFStore(path, mode="w")
464+
except ValueError as exc:
465+
pytest.fail(
466+
"store not closed properly, got error {exc}".format(exc=exc)
467+
)
468+
450469
def test_reopen_handle(self, setup_path):
451470

452471
with ensure_clean_path(setup_path) as path:

0 commit comments

Comments
 (0)