Skip to content

Increase metadata parsing to OME-Zarr & update to v0.4 (or v0.5) ome-ngff standard #44

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jluethi opened this issue May 18, 2022 · 2 comments

Comments

@jluethi
Copy link
Collaborator

jluethi commented May 18, 2022

We should look into adding more metadata to the OME-Zarr file, like the scale of channels (often not equal in all axes, e.g. larger in Z axis than x & y) and omero metadata like staining info, names, default z planes to show etc.

Also, the v0.4 ome-ngff standard (and the v0.5 draft) describe additional metadata for the plate in the .zattrs file: The wells would have rowIndex & columnIndex. As such, we would need all rows from A to last row (even if no wells in A exist) and all columns from 1 to max (even if no columns in col 1 exist) in the plate metadata. See examples discussed here: #34

Unclear whether the metadata would help with the errors generated in napari, see here: #35

@jluethi
Copy link
Collaborator Author

jluethi commented Jun 1, 2022

I dug a bit into the omero metadata today and figured out where we need to save the omero metadata to have some things like channel names (see also #61 regarding keeping channel numbers consistent), rescaling parameters (to set reasonable contrast limits), colormaps and more :)

From the OME-NGFF specification:

.                             # Root folder, potentially in S3,
│
└── 5966.zarr                 # One plate (id=5966) converted to Zarr
    ├── .zgroup
    ├── .zattrs               # Implements "plate" specification
    ├── A                     # First row of the plate
    │   ├── .zgroup
    │   │
    │   ├── 1                 # First column of row A
    │   │   ├── .zgroup
    │   │   ├── .zattrs       # Implements "well" specification
    │   │   │
    │   │   ├── 0             # First field of view of well A1
    │   │   │   │
    │   │   │   ├── .zgroup
    │   │   │   ├── .zattrs   # Implements "multiscales", "omero"    <= HERE they go
    │   │   │   ├── 0
    │   │   │   │   ...       # Resolution levels
    │   │   │   ├── n
    │   │   │   └── labels    # Labels (optional)
    │   │   ├── ...           # Fields of view
    │   │   └── m
    │   ├── ...               # Columns
    │   └── 12
    ├── ...                   # Rows
    └── H

These .zattrs files for use currently just contain the multiscales data, e.g.:

{
    "multiscales": [
        {
            "axes": [
                {
                    "name": "c",
                    "type": "channel"
                },
                {
                    "name": "z",
                    "type": "space",
                    "unit": "micrometer"
                },
                {
                    "name": "y",
                    "type": "space"
                },
                {
                    "name": "x",
                    "type": "space"
                }
            ],
            "datasets": [
                {
                    "path": "0"
                },
                {
                    "path": "1"
                },
                {
                    "path": "2"
                },
                {
                    "path": "3"
                },
                {
                    "path": "4"
                }
            ],
            "version": "0.4"
        }
    ]
}

We can add omero metadata after that as:

  ],                         <= add a comma after brackets from last entry
    "omero":
      {
        "id": 1,
        "name": "example.tif",
        "version": "0.3",
        "channels": [
            {
                "active": true,
                "coefficient": 1,
                "color": "00FFFF",
                "family": "linear",
                "inverted": false,
                "label": "DAPI",
                "window": {
                    "end": 600,
                    "max": 65535,
                    "min": 0,
                    "start": 110
                }
            },
            {
                "active": true,
                "coefficient": 1,
                "color": "FF00FF",
                "family": "linear",
                "inverted": false,
                "label": "nanog",
                "window": {
                    "end": 200,
                    "max": 65535,
                    "min": 0,
                    "start": 115
                }
            },
            {
                "active": true,
                "coefficient": 1,
                "color": "FFFF00",
                "family": "linear",
                "inverted": false,
                "label": "Lamin B1",
                "window": {
                    "end": 1000,
                    "max": 65535,
                    "min": 0,
                    "start": 115
                }
            }
        ]
      }

Omero metadata is a dictionary (in {}), not a list (in []). We need to have one channels entry per channel in the OME-Zarr and the order is the same as those channels, e.g. channel 0 is the first entry (here named DAPI), channel 1 the second etc.

Important metadata for visualization are (per channel):
"color": A reasonable choice for a colormap, primary concern is that they are distinct colormaps (=> we can come up with a good priority order for that)
"label": The name the channel displays as a napari layer name
"window": Information about the dynamic range. Max & min are based on the image type (we have 16bit images => 0, 65535). start & end are the rescaling parameters. For Yokogawa images, start of 110 or 115 is a good default, end may vary. We could also think about computing those based on percentiles of intensities in the image.

Important for the parsing at the moment: Only the omero parameters for the first well are currently parsed when loading an OME-Zarr plate into napari. We should fill in all, so that users could also load a single well later, but when we want to debug it (or when values would vary between wells), only the values for the top-left well actually matter. In the 23 well test case, those are the .zattrs for well B03, site 1.
I don't know yet what it would do in the multi-site case, as this has not been implemented for the ome-zarr reader yet.

@jluethi
Copy link
Collaborator Author

jluethi commented Jun 21, 2022

The channel handling also solved the adding omero metadata issue, see here: #61

We can track parsing scale & translation information in this issue here: https://github.com/fractal-analytics-platform/mwe_fractal/issues/46
(but decisions on what we want to do with it depend on the single- vs. multi-fov per well issue: https://github.com/fractal-analytics-platform/mwe_fractal/issues/74)

@jluethi jluethi closed this as completed Jun 21, 2022
Repository owner moved this from TODO to Done in Fractal Project Management Jun 21, 2022
@jluethi jluethi moved this from Done to Done Archive in Fractal Project Management Oct 5, 2022
jacopo-exact added a commit that referenced this issue Nov 16, 2022
…imultaneous-executions-of-fractal-server

select port when launching fractal-server
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant