Skip to content

zarr 2.10.0 fails to load pre-existing nested files #840

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
abred opened this issue Sep 28, 2021 · 5 comments · Fixed by #850
Closed

zarr 2.10.0 fails to load pre-existing nested files #840

abred opened this issue Sep 28, 2021 · 5 comments · Fixed by #850

Comments

@abred
Copy link

abred commented Sep 28, 2021

Minimal, reproducible code sample, a copy-pastable example if possible

import numpy as np
import zarr
a = np.ones((2,2)) 
z = zarr.open(zarr.NestedDirectoryStore("test.zarr"), 'w')
z['test'] = a

z = zarr.open(zarr.NestedDirectoryStore("test.zarr"), 'r')
print(np.array(z['test']))

arr = zarr.open(zarr.NestedDirectoryStore("path/to/existing/zarr/file.zarr"), 'r')
print(np.max(arr['volumes/raw']))

output with zarr 2.10.0:
array([[1., 1.],
[1., 1.]])
0

output with zarr 2.3.0:
array([[1., 1.],
[1., 1.]])
6733

Problem description

With the current version of zarr (2.10.0) pre-existing zarr files using NestedDirectoryStore are all zero when read.
With an older version (tested with 2.3.0) the correct values are loaded.
Creating a new zarr file using NestedDirectoryStore on 2.10.0 and reading it works correctly.

Version and installation information

Please provide the following:

  • Value of zarr.__version__: '2.10.0'
  • Value of numcodecs.__version__: '0.9.1'
  • Version of Python interpreter: 3.8.11
  • Operating system (Linux/Windows/Mac): Linux
  • How Zarr was installed (e.g., "using pip into virtual environment", or "using conda"): pip install
@joshmoore
Copy link
Member

Hi @abred. My apologies. I look at this almost every morning and think, "I need to write a test for this" but keep failing to do so. Is there any chance of your uploading something that will reproduce the error (e.g. by adding a very small dataset to the fixtures/ directory). I don't think the error will be hard to debug once we can see it happening.

joshmoore added a commit to joshmoore/zarr-python that referenced this issue Oct 18, 2021
joshmoore added a commit to joshmoore/zarr-python that referenced this issue Oct 18, 2021
Each fixture (flat & nested) should have a version
which does not include any new metadata.
@joshmoore
Copy link
Member

I'm managed to reproduce. Please see #850

joshmoore added a commit that referenced this issue Oct 19, 2021
* Fix #840

* Add legacy tests (See #840)

Each fixture (flat & nested) should have a version
which does not include any new metadata.

* Fix PEP8 issues

* Try dropping editable installs

see: pypa/pip#10573

* Handle case of store being a dict

* Exclude unexpected failure from code coverage

* Add 2.10.2 release notes
@joshmoore
Copy link
Member

@abred : please let us know if the issues persist.

@abred
Copy link
Author

abred commented Oct 21, 2021

Thank you for fixing this! And sorry for not getting back to you in time, I was on vacation. It's working now.

@joshmoore
Copy link
Member

👍 Thanks for letting us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants