Skip to content

zarr 3#108

Merged
thewtex merged 23 commits into
spatial-image:mainfrom
thewtex:zarr-3
Aug 6, 2025
Merged

zarr 3#108
thewtex merged 23 commits into
spatial-image:mainfrom
thewtex:zarr-3

Conversation

@thewtex

@thewtex thewtex commented Feb 14, 2025

Copy link
Copy Markdown
Contributor
  • COMP: Zarr Python 3 StoreLike support
  • COMP: Use LocalStore for Zarr Python 3
  • COMP: Remove dimension_deparator arg to MemoryStore
  • Remove home-page from pyproject.toml

zarr.storage.BaseStore is not available in Zarr Python 3.
Replaces DirectoryStore.
@thewtex thewtex mentioned this pull request Feb 14, 2025
xarray-dataclasses version needed to be explicitly specified because
pixi brings in 0.1.0 by default, which does not work.

ipfsspec causes the solver to hang, but it is not needed, so it is
removed.

itk-filtering and dask-image duplicates were removed.
@thewtex thewtex mentioned this pull request Feb 25, 2025
@thewtex thewtex force-pushed the zarr-3 branch 2 times, most recently from 8cea275 to de233f2 Compare February 25, 2025 02:03
For Zarr-Python 3 compatibility.
@thewtex

thewtex commented Feb 25, 2025

Copy link
Copy Markdown
Contributor Author

@melonora

melonora commented Mar 6, 2025

Copy link
Copy Markdown
Collaborator

Hi @thewtex, sorry for the radio silence. Is there part here you would like help with or we can share the workload a bit?

@thewtex

thewtex commented Mar 6, 2025

Copy link
Copy Markdown
Contributor Author

@melonora thanks for the offer to help 🙏

We are currently blocked by astropenguin/xarray-dataclasses#240 , which prevents resolution of a valid dependency tree.

A forked version could be published and used until it is resolved.

@melonora

melonora commented Mar 6, 2025

Copy link
Copy Markdown
Collaborator

I can have a look early next week

@doronbehar

Copy link
Copy Markdown

Hey, just wanted to nudge a little bit - here on NixOS we are still waiting for this PR :).

@melonora

melonora commented Jul 19, 2025

Copy link
Copy Markdown
Collaborator

Apologies, I got overthrown later with lab work. I had a look at the repo and the problem is now that the person owning the repository seems to have deleted all the code on main right now. I will pull the last working release and work from there.
Alternatively, check if we can get away with not using it.

@melonora

Copy link
Copy Markdown
Collaborator

@doronbehar The fork on my branch is I think almost ready after I fix the conflicts. I would like to have one look with @thewtex before releasing from my fork, but then after this we can adjust this PR and get it merged.

As said before, from the original repository all the code on main got deleted. It seems like the maintainer was in the process of moving to the use of dataspecs but didn't have time to complete it. The release of the fork would now only adjust the xarray constraints. I will check whether after that I can finish the maintainers work on the migration to dataspecs but that would take a while as I am also finishing my PhD.

@melonora

Copy link
Copy Markdown
Collaborator

@doronbehar I discussed with @thewtex and we think it is best to move the fork to the xarray org, where it could be better maintained. I have opened an issue on the original repository explaining the situation and asking whether we could get the permissions on PyPi so we can release under the same name. (astropenguin/xarray-dataclasses#244)

@thewtex

thewtex commented Jul 22, 2025

Copy link
Copy Markdown
Contributor Author

CC @ianhi

@melonora

Copy link
Copy Markdown
Collaborator

@doronbehar Just to update, I am trying to get the repository in the xarray-contrib organization right now, see xarray-contrib/xarray-contrib#16

@melonora

Copy link
Copy Markdown
Collaborator

@doronbehar @thewtex xarray-dataclass has been released. I pushed the change here already, but looks like we have some more work to do.

@thewtex

thewtex commented Jul 30, 2025

Copy link
Copy Markdown
Contributor Author

@melonora woohoo! 🎇

It looks like zarr-python 3 issues?

@thewtex

thewtex commented Jul 30, 2025

Copy link
Copy Markdown
Contributor Author

@melonora

Copy link
Copy Markdown
Collaborator

hmm can't see that conversation, but I will check later

@thewtex

thewtex commented Jul 30, 2025

Copy link
Copy Markdown
Contributor Author

Todo:

  • migrate spatial-image to xarray-dataclass
  • test updates for zarr v3

And corresponding spatial-image version bump to avoid conflicts with
xarray-dataclasses.
thewtex and others added 5 commits July 31, 2025 15:37
- Add asyncio import for v3 async store API
- Update check_valid_ngff() to handle both v2 and v3 store APIs
- Method 1: Try accessing via zarr group attrs (works for both)
- Method 2: Try v2 dict-like store access for consolidated metadata
- Method 3: Try v3 async store API with prototype parameter
- Handle different buffer/bytes formats between versions
- Gracefully fallback between methods to support both versions
@thewtex thewtex merged commit dad40c1 into spatial-image:main Aug 6, 2025
13 checks passed
@thewtex thewtex deleted the zarr-3 branch August 6, 2025 16:13
@thewtex thewtex mentioned this pull request Aug 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants