Skip to content

Defer chunk/shard choice to iohub#555

Merged
talonchandler merged 4 commits intomainfrom
chunks-cleanup
Apr 24, 2026
Merged

Defer chunk/shard choice to iohub#555
talonchandler merged 4 commits intomainfrom
chunks-cleanup

Conversation

@ieivanov
Copy link
Copy Markdown
Contributor

@ieivanov ieivanov commented Apr 23, 2026

Summary

Stack built on top of #554. Removes waveorder's hand-rolled chunk shapes and lets iohub pick them. Also pins iohub to the feat/sharding_in_utils branch so the new sharding defaults are exercised end-to-end.

  • apply_inverse_transfer_function.get_reconstruction_output_metadata: drop the hardcoded "chunks": (1, 1, 1, Y, X) from the returned dict. create_empty_plate (from Replace create_empty_hcs_zarr with iohub's create_empty_plate #554) now picks chunks via iohub's version-specific defaults (and shards for v0.5).
  • compute_transfer_function: drop every explicit chunks= arg passed to create_image. iohub's default is shape[-min(3, len(shape)):] padded with leading 1s — docstring: "ZYX stack size will be used if not specified." For a standard TCZYX array that's (1, 1, Z, Y, X) — one full ZYX volume per chunk — which is what we want for both 2D reconstructions (array Z-axis is 1, so the chunk collapses to an XY plane) and 3D reconstructions (full-ZYX chunk).
  • _write_phase_tf / _write_fluorescence_tf / _write_vector_birefringence_tf lose their zyx_shape parameter since chunks no longer depend on the input's shape.
  • Tests updated to match the new helper signatures.

Dependency / Python

  • Pin iohub to git+https://github.com/czbiohub-sf/iohub@feat/sharding_in_utils.
  • Bump requires-python to >=3.12 (iohub's branch requires it), drop 3.11 from the classifiers and the CI matrix (["3.12", "3.13"]).

TODO (follow-ups)

  • Revert the iohub pin to a released version (iohub>=...,<0.4) once feat/sharding_in_utils is merged and published.

Test plan

  • uv run pytest — 347 passed, 1 skipped, 2 xfailed.

🤖 Generated with Claude Code

ieivanov and others added 3 commits April 23, 2026 14:49
iohub's `create_empty_plate` does the same plate + channel
create/append work. The only waveorder-specific behavior was
writing a `plate_metadata` dict into the plate's zattrs; that's
now done inline in apply_inverse_transfer_function.py after
create_empty_plate returns.

Tests: 347 passed, 1 skipped, 2 xfailed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- apply_inverse_transfer_function.get_reconstruction_output_metadata:
  no longer pins "chunks": (1, 1, 1, Y, X). create_empty_plate picks
  chunks (and shards for v0.5) from its own defaults.
- compute_transfer_function: drop the explicit chunks= passed to every
  create_image call. iohub's default is shape[-3:] padded with leading
  1s (docstring: "ZYX stack size will be used if not specified"), which
  matches what we want for both 2D (Z=1 array axis, XY chunk) and 3D
  (full-ZYX chunk) reconstructions. _write_phase_tf and
  _write_fluorescence_tf no longer need zyx_shape.
- Tests updated to match the new helper signatures.

Tests: 347 passed, 1 skipped, 2 xfailed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
iohub's feat/sharding_in_utils branch requires Python >=3.12, so
drop 3.11 from the supported range, classifiers, and CI matrix.

TODO left in pyproject.toml: revert to a released iohub once that
branch is merged and published.

Tests: 347 passed, 1 skipped, 2 xfailed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Base automatically changed from switch-to-iohub-create-empty-plate to main April 24, 2026 15:23
@talonchandler talonchandler merged commit 77e6be7 into main Apr 24, 2026
7 checks passed
@talonchandler talonchandler deleted the chunks-cleanup branch April 24, 2026 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] waveorder writes ome-zarr data as XY images rather than XYZ chunks

2 participants