Cannot create empty zarr with chunks

[nmoreaud]

Hello

I would like to create a zarr file, and add chunked data incrementally.
In this example, I add items by chunks of size 5.
However, at zarr creation, if item_id coordinate is an empty list, chunks configuration is not correctly stored in test.zarr/item_data/.zarray, and the script finally creates 10 files in test.zarr/item_data/ instead of 2.

import json
from dask import array
import numpy as np
import xarray as xr

zarr_path = 'test.zarr'
block_size = 5
raster_size = 10
item_id_chunk_list = [
    [1, 2, 3, 5, 8],
    [13, 21, 34, 55, 89]
]


def fix_chunks_workaround():
    with open(f'{zarr_path}/item_data/.zarray', mode='r+') as f:
        meta = json.load(f)
        meta['chunks'][0] = block_size
        f.seek(0)
        f.truncate()
        json.dump(meta, f)


def create_ds(item_id_list) -> xr.Dataset:
    ds = xr.Dataset(
        data_vars={
            'item_data': (
                ['item_id', 'x', 'y'],
                array.zeros((len(item_id_list), raster_size, raster_size), dtype=np.float32)
            )
        },
        coords={
            'item_id': ('item_id', array.asarray(item_id_list, dtype=np.int64)),
            'x': ('x', list(range(raster_size))),
            'y': ('y', list(range(raster_size)))
        }
    )
    ds = ds.chunk({'item_id': block_size, 'x': -1, 'y': -1})
    return ds


# ds = create_ds(item_id_chunk_list[0])  # chunks are applied correctly, but item_id coordinate contains fake data.

ds = create_ds([])  # chunks are chosen by xarray. Why?
ds.to_zarr(zarr_path, mode='w', compute=False)
fix_chunks_workaround()

for item_id_chunk in item_id_chunk_list:
    ds = create_ds(item_id_chunk)
    for item_id in item_id_chunk:
        item_data = xr.DataArray(
            array.full((raster_size, raster_size), item_id, dtype=np.float32),
            dims=('x', 'y'))
        ds.loc[{'item_id': item_id}] = item_data
    ds.to_zarr(zarr_path, append_dim='item_id')

Is it the right way to do it?
I wonder if it comes from #8251

[nmoreaud]

As a workaround, I can manually modify chunks in test.zarr/item_data/.zarray after creation, and it seems to work (see fix_chunks_workaround).

After the zarr has been initialized, I wonder if I can fill it with distributed writes.
I see that the the shape metadata, and item_id array are updated with every call to append_dim.
Is there a way synchronize these writes with a lock? (note: I use Celery)