Draft
Conversation
* refactor: restructure viscy into uv workspace monorepo with viscy-transforms subpackage BREAKING CHANGE: - Import path changed: from viscy.transforms import X → from viscy_transforms import X - Removed legacy modules: viscy.utils, viscy.cli, viscy.unet, viscy.evaluation - Removed applications/, examples/, and docs/ directories Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org> * docs: updated readme with citations / examples from main Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org> * docs: added symlinking uv cache on hpc systems Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org> * add the jupyter and ipykernel to a optional visual group * re organize the transforms * add jupyternotebook back * build: updated some dep groups Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org> * build: added matplotlib back in Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org> * build: add ruff and prek to the dev dep group * docs: correct the docstring for the transform that doesn't exist lol Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org> --------- Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org> Co-authored-by: Sricharan Reddy Varra <sricharan.varra@biohub.org> Co-authored-by: Eduardo Hirata-Miyasaki <edhiratam@gmail.com> Co-authored-by: Alexandr Kalinin <alxndrkalinin@users.noreply.github.com>
* add planning roadmap * docs: start milestone v1.1 Models * docs: define milestone v1.1 requirements * docs: create milestone v1.1 roadmap (5 phases) * docs(package-scaffold-shared-components): research phase domain * docs(06): create phase plan - package scaffold and shared components * feat(06-01): create viscy-models package scaffold - Add pyproject.toml with hatchling build, torch/timm/monai/numpy deps - Create src layout with _components, unet, contrastive, vae subpackages - Add PEP 561 py.typed marker - Add test scaffolding with device fixture Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(06-01): register viscy-models in workspace - Add viscy-models to root dependencies and uv sources - Update lockfile with timm and viscy-models dependencies - Verified: uv sync, import, and pytest collection all succeed Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(06-01): complete package scaffold plan - Add 06-01-SUMMARY.md with execution results - Update STATE.md with position, metrics, and decisions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(06-02): extract shared components into _components/ module - stems.py: UNeXt2Stem, StemDepthtoChannels from v0.3.3 unext2.py - heads.py: PixelToVoxelHead, UnsqueezeHead, PixelToVoxelShuffleHead - blocks.py: icnr_init, _get_convnext_stage, UNeXt2UpStage, UNeXt2Decoder - __init__.py: re-exports all 8 public components - Zero imports from unet/, vae/, or contrastive/ - All attribute names preserved for state dict compatibility * feat(06-03): migrate ConvBlock2D and ConvBlock3D to unet/_layers/ - Copy ConvBlock2D from v0.3.3 source to snake_case file - Copy ConvBlock3D from v0.3.3 source to snake_case file - Preserve register_modules/add_module pattern for state dict key compatibility - Update _layers/__init__.py with public re-exports - Fix docstring formatting for ruff D-series compliance * test(06-03): add tests for ConvBlock2D and ConvBlock3D - 6 tests for ConvBlock2D: forward pass, state dict keys, residual, filter steps, instance norm - 4 tests for ConvBlock3D: forward pass, state dict keys, dropout registration, layer order - All 10 tests verify shape, naming patterns, and module registration * test(06-02): add forward-pass tests for all _components - test_stems.py: UNeXt2Stem shape, StemDepthtoChannels shape + mismatch error - test_heads.py: PixelToVoxelHead, UnsqueezeHead, PixelToVoxelShuffleHead shapes - test_blocks.py: icnr_init, _get_convnext_stage, UNeXt2UpStage, UNeXt2Decoder - 10 tests total, all passing on CPU * docs(06-03): complete UNet ConvBlock layers plan - SUMMARY.md with migration details and self-check - STATE.md updated: phase 6 plan 3/3, decisions recorded * docs(06-02): complete shared components extraction plan - SUMMARY.md documents 8 extracted components with 10 tests - STATE.md updated with decisions from 06-02 execution * docs(phase-6): complete phase execution and verification Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(07-core-unet-models): research phase domain * docs(07): create phase plan for core UNet models * feat(07-01): migrate UNeXt2 model class to viscy-models - Copy UNeXt2 class (~70 lines) from monolithic unext2.py - Update imports to use viscy_models._components (stems, heads, blocks) - Preserve all attribute names for state dict compatibility - Export UNeXt2 from viscy_models.unet public API Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(07-01): add 6 UNeXt2 forward-pass tests and fix deconv tuple bug - Add tests: default, small backbone, multichannel, diff stack depths, deconv, stem validation - Fix deconv decoder tuple bug in UNeXt2UpStage (trailing comma created tuple not module) - Mark deconv test xfail: original code has channel mismatch in deconv forward path - All 26 tests pass (25 passed, 1 xfailed) with no regressions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(07-01): complete UNeXt2 migration plan - Add 07-01-SUMMARY.md with execution results and deviation documentation - Update STATE.md: phase 7, plan 1/2 complete, new decisions logged Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-02): migrate FullyConvolutionalMAE to viscy-models - Copy FCMAE and all helper classes/functions to unet/fcmae.py - Replace old viscy imports with viscy_models._components imports - Remove duplicated PixelToVoxelShuffleHead (import from _components.heads) - Fix mutable list defaults to tuples (encoder_blocks, dims) - Export both UNeXt2 and FullyConvolutionalMAE from unet/__init__.py * test(07-02): migrate 11 FCMAE tests to viscy-models - Copy all 11 test functions with zero logic changes - Update imports from viscy.unet.networks.fcmae to viscy_models.unet.fcmae - Import PixelToVoxelShuffleHead from viscy_models._components.heads - All 37 tests pass across full suite (no regressions) * docs(07-02): complete FCMAE migration plan (Phase 7 complete) - Add 07-02-SUMMARY.md with execution results - Update STATE.md: Phase 7 complete, 12 plans total, decisions logged * docs(phase-7): complete phase execution and verification Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(phase-8): research representation models migration * docs(08): create phase plan for representation models * feat(08-02): migrate BetaVae25D and BetaVaeMonai to viscy-models - Add BetaVae25D with VaeUpStage, VaeEncoder, VaeDecoder helpers - Add BetaVaeMonai wrapping MONAI VarAutoEncoder - Fix VaeDecoder mutable list defaults to tuples (COMPAT-02) - Change VaeEncoder pretrained default to False - Preserve all attribute names for state dict compatibility Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(08-01): migrate ContrastiveEncoder and ResNet3dEncoder to viscy-models - Add ContrastiveEncoder with convnext/resnet50 backbone support via timm - Add ResNet3dEncoder with MONAI ResNetFeatures backend - Fix ResNet50 bug: use encoder.num_features instead of encoder.head.fc.in_features - Add pretrained parameter (default False) for pure nn.Module semantics - Preserve state dict attribute names (stem, encoder, projection) - Share projection_mlp utility between both encoder classes * test(08-01): add 5 forward-pass tests for contrastive models - 3 tests for ContrastiveEncoder: convnext_tiny, resnet50, custom stem - 2 tests for ResNet3dEncoder: resnet18, resnet10 - Verify embedding and projection output shapes - ResNet50 test uses in_stack_depth=10 for valid stem channel alignment * test(08-02): add forward-pass tests for BetaVae25D and BetaVaeMonai - 2 BetaVae25D tests: resnet50 and convnext_tiny backbones - 2 BetaVaeMonai tests: 2D and 3D spatial configurations - Verify SimpleNamespace output with recon_x, mean, logvar, z - Fix ResNet50 expected spatial dims (64x64 not 128x128) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(08-01): complete contrastive model migration plan - Add 08-01-SUMMARY.md with execution results - Update STATE.md to Phase 8, plan 1/2 * docs(08-02): complete VAE migration plan (Phase 8 complete) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(phase-8): complete phase execution and verification Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(09): research legacy UNet models migration * docs(09): create phase plan for legacy UNet models * feat(09-01): migrate Unet2d and Unet25d to viscy-models - Copy Unet2d from v0.3.3 with import path update to viscy_models.unet._layers - Copy Unet25d from v0.3.3 with import path update to viscy_models.unet._layers - Fix mutable default num_filters=[] to num_filters=() in both models - Add module docstrings and __all__ exports - Update unet/__init__.py to export all 4 models (UNeXt2, FCMAE, Unet2d, Unet25d) - Preserve register_modules/add_module pattern for state dict compatibility - All 45 existing tests still pass Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(09-01): add pytest tests for Unet2d and Unet25d - 12 tests for Unet2d: default forward, variable depth, multichannel, residual, task mode, dropout, state dict keys, custom num_filters - 11 tests for Unet25d: default Z-compression, preserved depth, variable depth, multichannel, residual, task mode, state dict keys with skip_conv_layer, custom filters - Fix list(num_filters) conversion in both models for tuple default compatibility - Total test suite: 68 passed, 1 xfailed, 0 failures Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(09-01): complete legacy UNet migration plan - SUMMARY.md with task commits, deviations, and self-check - STATE.md updated to Phase 9 complete (15 plans total) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(phase-9): complete phase execution and verification Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(10): create phase plan for public API and CI integration * feat(10-01): add top-level re-exports for all 8 model classes - Import UNeXt2, FullyConvolutionalMAE, Unet2d, Unet25d from unet subpackage - Import ContrastiveEncoder, ResNet3dEncoder from contrastive subpackage - Import BetaVae25D, BetaVaeMonai from vae subpackage - Update __all__ with all 8 classes in alphabetical order Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(10-01): add state dict key compatibility regression tests - 24 tests covering all 8 migrated model architectures - Each model tested for parameter count, top-level prefixes, and sentinel keys - Guards COMPAT-01: state dict keys must match for checkpoint loading - Tests import from top-level viscy_models package (validates public API) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore(10-01): add viscy-models to CI test matrix - Add package dimension to test matrix (viscy-transforms, viscy-models) - Use cross-platform --cov=src/ instead of named package coverage - Matrix now produces 18 jobs (3 OS x 3 Python x 2 packages) - check job automatically aggregates all test results via alls-green Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(10-01): complete public API & CI integration plan (v1.1 milestone complete) - Add 10-01-SUMMARY.md with execution results - Update STATE.md: phase 10 complete, v1.1 milestone done, 100% progress Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(phase-10): complete phase execution and verification (v1.1 milestone complete) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: consolidate ConvBlock2D/3D into _components Move conv_block_2d.py and conv_block_3d.py from unet/_layers/ to _components/ alongside all other shared building blocks. All reusable layers now live in one place. unet/_layers/ retained as backward- compatible re-export shim. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * remove the _layers * update the readme * update the main readme * fix description in toml * changing ruff formatting , dosctrings and imports * renaming folder to components * updatet planning docs * update to components * numpy docstring * add claude.md and contributing.md --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
edyoshikun
changed the title
* add planning roadmap * docs: start milestone v1.1 Extract viscy-data * docs: complete viscy-data project research * docs: define milestone v1.1 requirements * docs: create milestone v1.1 roadmap (4 phases) * docs(06-package-scaffolding-and-foundation): create phase plan * feat(06-01): create viscy-data package directory structure with pyproject.toml - Add pyproject.toml with hatchling build, uv-dynamic-versioning, all base deps - Declare optional dependency groups: triplet, livecell, mmap, all - Add PEP 561 py.typed marker and tests/__init__.py - Configure pattern-prefix for independent versioning Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(06-01): add type definitions and package init with re-exports - Copy all type definitions from viscy/data/typing.py into _typing.py - Add INDEX_COLUMNS from viscy/data/triplet.py for shared access - Update typing_extensions.NotRequired to typing.NotRequired (Python >=3.11) - Create __init__.py with full re-export of all public types - Add README.md required by hatchling build Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(06-01): integrate viscy-data as workspace dependency in root pyproject.toml - Add viscy-data to root dependencies list - Register viscy-data as workspace source in [tool.uv.sources] - Verified editable install and full import chain works Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(06-01): complete package scaffolding plan with summary and state update - Add 06-01-SUMMARY.md documenting viscy-data package creation - Update STATE.md with plan position, metrics, and decisions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(06-02): extract shared utility functions into _utils.py - Extract _ensure_channel_list, _search_int_in_str, _collate_samples, _read_norm_meta from hcs.py - Extract _scatter_channels, _gather_channels, _transform_channel_wise from triplet.py - Update imports to use viscy_data._typing instead of viscy.data.typing - Add __all__ listing all 7 utility functions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(06-02): complete utility module extraction plan - Add 06-02-SUMMARY.md documenting utility extraction - Update STATE.md: Phase 6 complete, progress 80% Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(phase-6): complete phase execution * docs(07-code-migration): create phase plan * feat(07-01): migrate select.py, distributed.py, segmentation.py to viscy-data - Copy select.py with well/FOV filtering utilities (no internal viscy imports) - Copy distributed.py with ShardedDistributedSampler (no internal viscy imports) - Copy segmentation.py with viscy.data.typing -> viscy_data._typing import update - Add missing docstrings to satisfy ruff D rules Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-01): migrate hcs.py to viscy-data with utility import rewiring - Copy HCSDataModule, SlidingWindowDataset, MaskTestDataset from main - Replace viscy.data.typing imports with viscy_data._typing - Remove 4 utility function definitions (now in _utils.py) - Add import from viscy_data._utils for shared utilities - Remove unused re and collate_meta_tensor imports - Add missing docstrings for ruff D compliance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-01): migrate gpu_aug.py to viscy-data with dependency rewiring - Copy GPUTransformDataModule, CachedOmeZarrDataset, CachedOmeZarrDataModule - Rewire viscy.data.distributed -> viscy_data.distributed - Rewire viscy.data.hcs utility imports -> viscy_data._utils - Rewire viscy.data.select -> viscy_data.select - Rewire viscy.data.typing -> viscy_data._typing - Add missing docstrings for ruff D compliance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(07-01): complete core data module migration plan - Add 07-01-SUMMARY.md documenting migration of 5 core modules - Update STATE.md: phase 7 plan 1 of 4, decisions, metrics Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-03): migrate mmap_cache.py and ctmc_v1.py to viscy-data - Rewire all imports from viscy.data to viscy_data prefix - Add lazy import for tensordict with clear error message - Add docstrings for ruff D compliance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-03): migrate livecell.py with lazy optional dependency imports - Rewire imports from viscy.data to viscy_data prefix - Add lazy imports for pycocotools, tifffile, torchvision - Add import guards in LiveCellDataset and LiveCellTestDataset __init__ - Add docstrings for ruff D compliance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-03): migrate combined.py as-is with import rewiring - Rewire viscy.data.distributed to viscy_data.distributed - Rewire viscy.data.hcs._collate_samples to viscy_data._utils._collate_samples - Preserve all 6 public classes without structural changes - Add docstrings for ruff D compliance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-02): migrate cell_classification.py and cell_division_triplet.py - Rewire imports from viscy.data to viscy_data prefix - Add lazy import for pandas in cell_classification.py with clear error message - Import _transform_channel_wise from viscy_data._utils (not triplet.py) - Import INDEX_COLUMNS and AnnotationColumns from viscy_data._typing - Add docstrings for ruff D compliance * docs(07-03): complete optional dependency module migration plan Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(07-02): complete specialized module migration plan - Add 07-02-SUMMARY.md documenting triplet, classification, and cell division module migration - Update STATE.md with position, decisions, and metrics Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-04): add complete public API exports to viscy_data __init__.py - Export all 45 public names (17 types, 2 utilities, 26 DataModules/Datasets/enums) - Eager imports from all 13 modules (lazy guards handled internally by each module) - Comprehensive __all__ list for IDE autocompletion and star-import support - Ruff-sorted import ordering passes all lint checks Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(07-04): complete public API exports plan - phase 7 fully done - 07-04-SUMMARY.md documenting 45 public exports and full package verification - STATE.md updated: phase 7 complete (4/4 plans), 12 total plans done Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(phase-7): complete code migration execution * docs(08-test-migration-and-validation): create phase plan * test(08-01): add conftest.py with HCS OME-Zarr fixtures for viscy-data - Copy all 6 fixtures and _build_hcs helper from main branch conftest - Replace legacy np.random.rand with np.random.default_rng (NPY002) - No viscy import changes needed (only uses third-party libs) - Provides preprocessed_hcs_dataset, small_hcs_dataset, tracks fixtures Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(08-02): complete smoke tests plan - phase 8 test migration done - Created 08-02-SUMMARY.md documenting 52 smoke tests for viscy_data - Updated STATE.md: phase 8 complete, 14 total plans executed - DATA-TST-02 satisfied: import, __all__, optional dep messages, no legacy namespace Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(08-01): migrate test_hcs, test_triplet, test_select to viscy-data package - Update imports from viscy.data.X to viscy_data - Add BatchedCenterSpatialCropd to _utils.py (fixes batch dim handling) - Fix triplet.py to use BatchedCenterSpatialCropd instead of CenterSpatialCropd - Add tensorstore to test dependency group for triplet tests - All 19 tests pass across 3 test files Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(08-01): complete data test migration plan summary - Create 08-01-SUMMARY.md documenting test migration and bug fixes - Update STATE.md with BatchedCenterSpatialCropd decision revision Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(phase-8): complete test migration and validation * docs(09-ci-integration): create phase plan * feat(09-01): add viscy-data CI test jobs to GitHub Actions workflow - Add test-data job with 3x3 matrix (3 OS x 3 Python) for viscy-data - Add test-data-extras job (ubuntu-latest, Python 3.13) for extras validation - Update check job needs to aggregate all test jobs: test, test-data, test-data-extras Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(09-01): complete CI integration plan - Add 09-01-SUMMARY.md documenting viscy-data CI jobs - Update STATE.md: phase 9 complete, v1.0 milestone complete Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(phase-9): complete CI integration - milestone v1.1 done * chore: complete v1.1 milestone — Extract viscy-data Delivered: viscy-data package with 15 modules, 45 public exports, optional dependency groups, 71 tests, and tiered CI. Archives: - milestones/v1.1-ROADMAP.md - milestones/v1.1-REQUIREMENTS.md Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add missing pandas guards, restore conftest fixture, remove no-op CI filter - Add `if pd is None` guard in ClassificationDataModule.setup() and TripletDataModule._align_tracks_tables_with_positions() to raise helpful ImportError instead of AttributeError when pandas is absent - Fix ClassificationDataset error message to suggest `pip install pandas` instead of `pip install 'viscy-data[triplet]'` (classification doesn't need tensorstore) - Restore `num_timepoints` parameter on `_build_hcs()` and add `temporal_hcs_dataset` fixture from upstream commit 44b25b9 - Remove no-op `-m "not slow"` from test-data-extras CI job (no tests use @pytest.mark.slow) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add CLAUDE.md and update CONTRIBUTING.md Add CLAUDE.md with project-specific instructions for Claude Code sessions. Update CONTRIBUTING.md with ruff config centralization warning and numpy docstring convention note. Synced from 71009b5. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update uv.lock after rebase onto modular-viscy-staging Regenerate lockfile to include viscy-data workspace dependencies alongside viscy-models from the updated base branch. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * port changes from tests-zarrv3 branch * ruff * add additional test for the main dataloaders * redundant tets 3. I think test 2 alreaady takes care of this. * rename INDEX_COLUMNS * remove unused LABEL classes * fix(livecell): assign transform result and avoid mutable defaults LiveCellTestDataset.__getitem__ discarded the return value of self.transform(sample), so MONAI transforms had no effect. Also replace mutable default lists in LiveCellDataModule.__init__ with None to prevent cross-instance state sharing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cell_classification): raise ValueError, tighten val_fovs type, fix mutable default - `raise (f"Unknown stage: {stage}")` raised a string instead of an exception — use `ValueError`. - `val_fovs: list[str] | None` was unconditionally indexed in `setup()` — remove the `None` option since it's always required. - `_subset(..., exclude_timepoints=[])` used a mutable default — replace with `None`. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(gpu_aug): remove _filter_fit_fovs override so exclude_fovs is applied CachedOmeZarrDataModule accepted exclude_fovs but its local _filter_fit_fovs override only filtered wells, silently ignoring excluded FOVs. Remove the override so the SelectWell mixin's implementation (which filters both wells and FOVs) is used. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: rename select.py to _select.py (private module) The module contains mostly private helpers (_filter_wells, _filter_fovs) and a mixin dataclass (SelectWell). Renaming to _select.py signals it is internal implementation detail. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: update package descriptions to "AI x Imaging" and CLAUDE.md Update viscy-data description from "virtual staining microscopy" to "AI x Imaging tasks" in README, pyproject.toml, and __init__.py. Add viscy-models test example to CLAUDE.md. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(ctmc_v1): add missing prefetch_factor attribute CTMCv1DataModule.__init__ did not set self.prefetch_factor, causing an AttributeError when train_dataloader() or val_dataloader() was called (inherited from GPUTransformDataModule). Set it to None. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add functional tests for ctmc, livecell, segmentation, classification Add unit tests for the four data modules that previously only had import smoke tests: - test_ctmc_v1.py: setup, val subsample ratio, batch shape - test_livecell.py: dataset/datamodule with mock TIFF + COCO data - test_segmentation.py: paired pred/target datasets, z-slice - test_cell_classification.py: annotation CSV, FOV split, timepoint exclusion Also adds shared fixtures to conftest.py (single_channel_hcs_pair, segmentation_hcs_pair, classification_hcs_dataset). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(livecell): add missing prefetch_factor; add dataloader iteration tests Add prefetch_factor attribute to LiveCellDataModule (same fix as 97455eb for CTMC). Add batch iteration + shape validation to classification and livecell datamodule tests to match coverage patterns in test_hcs/test_triplet. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: delete leftover select.py after rename to _select.py Commit 5b132f9 renamed select.py to _select.py but did not remove the original file. Nothing imports from it. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use explicit positions[0] instead of leaked loop variable CachedOmeZarrDataset and MmappedDataset both built self.channels using the loop variable `position` after iterating, implicitly depending on the last element. Use positions[0] to be explicit and avoid UnboundLocalError on empty input. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(livecell): handle empty annotations and correct return type Guard torch.stack against empty annotation lists in LiveCellTestDataset.__getitem__ when load_labels=True, returning properly shaped empty tensors instead of crashing. Fix _parse_image_names return type annotation: list[Path] -> list[str]. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(segmentation): defer open_ome_zarr from __init__ to setup() Store only paths in __init__ and open OME-Zarr stores in setup("test"), consistent with other DataModules in the package and Lightning conventions for resource lifecycle. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * print to logger * refactor(triplet): remove BatchedCenterSpatialCropd from viscy-data The transform already exists in viscy-transforms and viscy-data should not depend on it. Replace the final crop with a shape validation check in on_after_batch_transfer() and require initial_yx_patch_size to match the desired output size. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(docs): convert Sphinx-style docstrings to numpy style in _utils.py Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(init): lazy-load submodules via PEP 562 __getattr__ Replace eager imports of all DataModule/Dataset submodules with on-demand loading. Modules with optional dependencies (triplet, livecell, mmap_cache) are no longer imported at `import viscy_data` time. Add __init__.pyi stub for type-checker/IDE support. Also split CI test-data job to run without --all-extras so the base package is validated independently from optional dependencies. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: guard None dereferences and replace mutable default arguments - cell_classification.py: guard _read_norm_meta() returning None - hcs.py: guard MaskTestDataset with ground_truth_masks=None, add missing array_key parameter, replace mutable default [] with None - triplet.py, cell_division_triplet.py: replace mutable default [] with None for normalizations/augmentations parameters Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: correct return type annotations in _utils.py _gather_channels and _transform_channel_wise return Tensor, not list[Tensor]. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(combined): check stage before calling dm.setup() Move the unsupported-stage guard to the top of setup() in ConcatDataModule and CachedConcatDataModule so constituent data modules are not set up for stages that will be rejected. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add tests for cell_division_triplet, mmap_cache, and HCS test stage - test_cell_division_triplet.py: 11 smoke tests for dataset and datamodule - test_mmap_cache.py: 5 smoke tests (skipped when tensordict missing) - test_hcs.py: add setup("test") coverage for MaskTestDataset Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Eduardo Hirata-Miyasaki <edhiratam@gmail.com> Co-authored-by: Alexandr Kalinin <alxndrkalinin@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test(10-01): add state dict key compatibility regression tests
- 24 tests covering all 8 migrated model architectures
- Each model tested for parameter count, top-level prefixes, and sentinel keys
- Guards COMPAT-01: state dict keys must match for checkpoint loading
- Tests import from top-level viscy_models package (validates public API)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore(10-01): add viscy-models to CI test matrix
- Add package dimension to test matrix (viscy-transforms, viscy-models)
- Use cross-platform --cov=src/ instead of named package coverage
- Matrix now produces 18 jobs (3 OS x 3 Python x 2 packages)
- check job automatically aggregates all test results via alls-green
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(10-01): complete public API & CI integration plan (v1.1 milestone complete)
- Add 10-01-SUMMARY.md with execution results
- Update STATE.md: phase 10 complete, v1.1 milestone done, 100% progress
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(phase-10): complete phase execution and verification (v1.1 milestone complete)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: consolidate ConvBlock2D/3D into _components
Move conv_block_2d.py and conv_block_3d.py from unet/_layers/ to
_components/ alongside all other shared building blocks. All reusable
layers now live in one place. unet/_layers/ retained as backward-
compatible re-export shim.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* remove the _layers
* update the readme
* docs: start milestone v1.1 Extract viscy-data
* update the main readme
* docs: complete viscy-data project research
* docs: define milestone v1.1 requirements
* docs: create milestone v1.1 roadmap (4 phases)
* docs(06-package-scaffolding-and-foundation): create phase plan
* feat(06-01): create viscy-data package directory structure with pyproject.toml
- Add pyproject.toml with hatchling build, uv-dynamic-versioning, all base deps
- Declare optional dependency groups: triplet, livecell, mmap, all
- Add PEP 561 py.typed marker and tests/__init__.py
- Configure pattern-prefix for independent versioning
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(06-01): add type definitions and package init with re-exports
- Copy all type definitions from viscy/data/typing.py into _typing.py
- Add INDEX_COLUMNS from viscy/data/triplet.py for shared access
- Update typing_extensions.NotRequired to typing.NotRequired (Python >=3.11)
- Create __init__.py with full re-export of all public types
- Add README.md required by hatchling build
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(06-01): integrate viscy-data as workspace dependency in root pyproject.toml
- Add viscy-data to root dependencies list
- Register viscy-data as workspace source in [tool.uv.sources]
- Verified editable install and full import chain works
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(06-01): complete package scaffolding plan with summary and state update
- Add 06-01-SUMMARY.md documenting viscy-data package creation
- Update STATE.md with plan position, metrics, and decisions
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(06-02): extract shared utility functions into _utils.py
- Extract _ensure_channel_list, _search_int_in_str, _collate_samples, _read_norm_meta from hcs.py
- Extract _scatter_channels, _gather_channels, _transform_channel_wise from triplet.py
- Update imports to use viscy_data._typing instead of viscy.data.typing
- Add __all__ listing all 7 utility functions
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(06-02): complete utility module extraction plan
- Add 06-02-SUMMARY.md documenting utility extraction
- Update STATE.md: Phase 6 complete, progress 80%
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(phase-6): complete phase execution
* docs(07-code-migration): create phase plan
* feat(07-01): migrate select.py, distributed.py, segmentation.py to viscy-data
- Copy select.py with well/FOV filtering utilities (no internal viscy imports)
- Copy distributed.py with ShardedDistributedSampler (no internal viscy imports)
- Copy segmentation.py with viscy.data.typing -> viscy_data._typing import update
- Add missing docstrings to satisfy ruff D rules
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(07-01): migrate hcs.py to viscy-data with utility import rewiring
- Copy HCSDataModule, SlidingWindowDataset, MaskTestDataset from main
- Replace viscy.data.typing imports with viscy_data._typing
- Remove 4 utility function definitions (now in _utils.py)
- Add import from viscy_data._utils for shared utilities
- Remove unused re and collate_meta_tensor imports
- Add missing docstrings for ruff D compliance
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(07-01): migrate gpu_aug.py to viscy-data with dependency rewiring
- Copy GPUTransformDataModule, CachedOmeZarrDataset, CachedOmeZarrDataModule
- Rewire viscy.data.distributed -> viscy_data.distributed
- Rewire viscy.data.hcs utility imports -> viscy_data._utils
- Rewire viscy.data.select -> viscy_data.select
- Rewire viscy.data.typing -> viscy_data._typing
- Add missing docstrings for ruff D compliance
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(07-01): complete core data module migration plan
- Add 07-01-SUMMARY.md documenting migration of 5 core modules
- Update STATE.md: phase 7 plan 1 of 4, decisions, metrics
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(07-03): migrate mmap_cache.py and ctmc_v1.py to viscy-data
- Rewire all imports from viscy.data to viscy_data prefix
- Add lazy import for tensordict with clear error message
- Add docstrings for ruff D compliance
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(07-03): migrate livecell.py with lazy optional dependency imports
- Rewire imports from viscy.data to viscy_data prefix
- Add lazy imports for pycocotools, tifffile, torchvision
- Add import guards in LiveCellDataset and LiveCellTestDataset __init__
- Add docstrings for ruff D compliance
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(07-03): migrate combined.py as-is with import rewiring
- Rewire viscy.data.distributed to viscy_data.distributed
- Rewire viscy.data.hcs._collate_samples to viscy_data._utils._collate_samples
- Preserve all 6 public classes without structural changes
- Add docstrings for ruff D compliance
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(07-02): migrate cell_classification.py and cell_division_triplet.py
- Rewire imports from viscy.data to viscy_data prefix
- Add lazy import for pandas in cell_classification.py with clear error message
- Import _transform_channel_wise from viscy_data._utils (not triplet.py)
- Import INDEX_COLUMNS and AnnotationColumns from viscy_data._typing
- Add docstrings for ruff D compliance
* docs(07-03): complete optional dependency module migration plan
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(07-02): complete specialized module migration plan
- Add 07-02-SUMMARY.md documenting triplet, classification, and cell division module migration
- Update STATE.md with position, decisions, and metrics
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(07-04): add complete public API exports to viscy_data __init__.py
- Export all 45 public names (17 types, 2 utilities, 26 DataModules/Datasets/enums)
- Eager imports from all 13 modules (lazy guards handled internally by each module)
- Comprehensive __all__ list for IDE autocompletion and star-import support
- Ruff-sorted import ordering passes all lint checks
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(07-04): complete public API exports plan - phase 7 fully done
- 07-04-SUMMARY.md documenting 45 public exports and full package verification
- STATE.md updated: phase 7 complete (4/4 plans), 12 total plans done
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(phase-7): complete code migration execution
* docs(08-test-migration-and-validation): create phase plan
* test(08-01): add conftest.py with HCS OME-Zarr fixtures for viscy-data
- Copy all 6 fixtures and _build_hcs helper from main branch conftest
- Replace legacy np.random.rand with np.random.default_rng (NPY002)
- No viscy import changes needed (only uses third-party libs)
- Provides preprocessed_hcs_dataset, small_hcs_dataset, tracks fixtures
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(08-02): complete smoke tests plan - phase 8 test migration done
- Created 08-02-SUMMARY.md documenting 52 smoke tests for viscy_data
- Updated STATE.md: phase 8 complete, 14 total plans executed
- DATA-TST-02 satisfied: import, __all__, optional dep messages, no legacy namespace
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(08-01): migrate test_hcs, test_triplet, test_select to viscy-data package
- Update imports from viscy.data.X to viscy_data
- Add BatchedCenterSpatialCropd to _utils.py (fixes batch dim handling)
- Fix triplet.py to use BatchedCenterSpatialCropd instead of CenterSpatialCropd
- Add tensorstore to test dependency group for triplet tests
- All 19 tests pass across 3 test files
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(08-01): complete data test migration plan summary
- Create 08-01-SUMMARY.md documenting test migration and bug fixes
- Update STATE.md with BatchedCenterSpatialCropd decision revision
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(phase-8): complete test migration and validation
* docs(09-ci-integration): create phase plan
* feat(09-01): add viscy-data CI test jobs to GitHub Actions workflow
- Add test-data job with 3x3 matrix (3 OS x 3 Python) for viscy-data
- Add test-data-extras job (ubuntu-latest, Python 3.13) for extras validation
- Update check job needs to aggregate all test jobs: test, test-data, test-data-extras
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(09-01): complete CI integration plan
- Add 09-01-SUMMARY.md documenting viscy-data CI jobs
- Update STATE.md: phase 9 complete, v1.0 milestone complete
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(phase-9): complete CI integration - milestone v1.1 done
* chore: complete v1.1 milestone — Extract viscy-data
Delivered: viscy-data package with 15 modules, 45 public exports,
optional dependency groups, 71 tests, and tiered CI.
Archives:
- milestones/v1.1-ROADMAP.md
- milestones/v1.1-REQUIREMENTS.md
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* harmonize the planning between the modular-data and modular-models
* viscy-utils package
* add applications/dynaclr
* update the monorepo uv
* moving files around
* update planning
* docs: start milestone v2.1 DynaCLR Integration Validation
* docs: define milestone v2.1 requirements
* docs: create milestone v2.1 roadmap (2 phases)
* docs(18-training-validation): create phase plan
* feat(18-01): add training integration tests for ContrastiveModule
- Add fast_dev_run tests for TripletMarginLoss and NTXentLoss code paths
- Add parametrized config class_path resolution tests for fit.yml and predict.yml
- Add tensorboard as test dependency for TensorBoardLogger in integration tests
- Fix workspace exclude to skip non-package application directories
- Use 2D-compatible synthetic data shapes (1,1,4,4) for render_images compatibility
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(18-01): complete training integration tests plan
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(phase-18): complete phase execution
* docs(19-inference-reproducibility): create phase plan
* chore(19-01): add anndata test dependency and HPC conftest fixtures
- Add anndata to dynacrl test dependency group
- Create conftest.py with HPC path constants, skip markers, and fixtures
- Update uv.lock
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(19-01): add inference reproducibility integration tests
- Create test_inference_reproducibility.py with 2 HPC integration tests
- test_checkpoint_loads_into_modular_contrastive_module (INFER-01)
- test_predict_embeddings_and_exact_match (INFER-02 + INFER-03)
- Fix lazy imports in EmbeddingWriter to avoid unconditional umap import
- Fix anndata nullable string compatibility in write_embedding_dataset
- Tests skip gracefully when HPC paths or GPU unavailable
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(19-01): complete inference reproducibility plan
- Add 19-01-SUMMARY.md with execution results and deviation documentation
- Update STATE.md: Phase 19 complete, v2.1 milestone finished
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add seed_everything(42) to all integration tests
Ensures reproducibility by seeding all tests consistently.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(phase-19): complete phase execution
* restructure the examples folder and ruff
* update readme.me hallucination
* update the readmes
* - Add `viscy` console script in viscy-utils pointing to
viscy_utils.cli:main
- Add jsonargparse[signatures] dependency for LightningCLI
- Add 4 CLI smoke tests (help, subcommands, fit --help, predict
--help)
- Replace conda/anaconda with uv in SLURM scripts
- Update SLURM scripts to use `viscy fit/predict` instead of old
monolith
* add the CLI for running training and prediction
* default embedding writer to None
* import within the function
* ruff
* dynaclr typo
* rename folder to dynaclr
* add the classifiers here
* docs: start milestone v2.2 Composable Sampling Framework
* docs: define milestone v2.2 requirements
* docs: create milestone v2.2 roadmap (6 phases)
* docs(20): capture phase context
* docs(20): create phase plan for experiment configuration
* test(20-01): add failing tests for ExperimentConfig and ExperimentRegistry
- 19 test cases covering config creation, defaults, channel maps,
validation errors, YAML loading, tau-range conversion, and lookups
- All tests fail with ModuleNotFoundError (module not yet implemented)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(20-01): implement ExperimentConfig and ExperimentRegistry
- ExperimentConfig dataclass with all fields and defaults
- ExperimentRegistry with fail-fast validation at __post_init__:
empty check, duplicate names, source_channel membership,
channel count consistency, interval_minutes positivity,
condition_wells non-empty, data_path existence, zarr channel match
- channel_maps: per-experiment source position -> zarr index mapping
- from_yaml classmethod for YAML config loading
- tau_range_frames for hours-to-frames conversion with warning
- get_experiment lookup by name with KeyError
- All 19 tests pass
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor(20-01): clean up imports and exclude stale dynacrl workspace member
- Fix ruff I001 (import sorting) and F401 (unused import) in test file
- Exclude applications/dynacrl (typo) from uv workspace to unblock builds
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(20-01): complete ExperimentConfig/ExperimentRegistry plan
- SUMMARY.md with TDD execution results, self-check passed
- STATE.md updated with position, decisions, session continuity
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(20-02): add explicit deps and top-level experiment API exports
- Add iohub>=0.3a2 and pyyaml as explicit dependencies in dynaclr pyproject.toml
- Re-export ExperimentConfig and ExperimentRegistry from dynaclr __init__.py
- Both classes now importable via `from dynaclr import ExperimentConfig, ExperimentRegistry`
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(20-02): add example multi-experiment YAML configuration
- Demonstrate positional channel alignment across 2 experiments
- SEC61 (30min interval, ER) and TOMM20 (15min interval, mito)
- Show condition_wells with infected/uninfected/mock conditions
- Include comments explaining channel alignment and tau_range conversion
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(20-02): complete package wiring and example config plan
- SUMMARY.md with execution results and self-check
- STATE.md updated: Phase 20 complete, 20/25 phases (80%)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(phase-20): complete phase execution
Phase 20 Experiment Configuration verified (11/11 must-haves).
ExperimentConfig + ExperimentRegistry with TDD, package wiring, example YAML.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(21): create phase plan for Cell Index & Lineage
* test(21-01): add failing tests for MultiExperimentIndex
- 17 test cases covering CELL-01 (unified tracks), CELL-02 (lineage), CELL-03 (border clamping)
- All fail with ModuleNotFoundError (dynaclr.index not yet implemented)
- Test fixtures create mini OME-Zarr stores with tracking CSVs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(21-01): implement MultiExperimentIndex with lineage and border clamping
- Unified tracks DataFrame from all experiments with enriched columns
- Lineage reconstruction linking daughters to root ancestor via parent_track_id
- Border clamping: retains border cells with shifted patch origins instead of exclusion
- All 23 tests pass
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor(21-01): fix lint issues and export MultiExperimentIndex
- Remove unused variable (F841) in test_global_track_id_unique_across_experiments
- Use .to_numpy() instead of .values (PD011) in test_exclude_fovs_filter
- Export MultiExperimentIndex from dynaclr __init__.py
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(21-01): complete MultiExperimentIndex plan summary and state update
- 21-01-SUMMARY.md with full execution documentation
- STATE.md updated for 21-01 completion, decisions, session continuity
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test(21-02): add failing tests for valid anchors, properties, and summary
- 8 tests for valid_anchors: basic validity, subset check, end-of-track exclusion,
lineage continuity, different tau ranges, empty tracks, gap handling, self-exclusion
- 9 tests for properties/summary: experiment_groups, condition_groups, summary()
- All 17 new tests fail with TypeError (tau_range_hours not yet accepted)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(21-02): implement valid_anchors, experiment_groups, condition_groups, summary
- Add tau_range_hours parameter to MultiExperimentIndex.__init__
- _compute_valid_anchors: per-experiment tau conversion, lineage-based lookup
- experiment_groups/condition_groups properties returning index arrays
- summary() with experiment counts, observation counts, per-experiment breakdowns
- All 40 tests pass (23 existing + 17 new)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(21-02): complete valid anchors plan
- SUMMARY.md with self-check passed
- STATE.md updated: Phase 21 complete, ready for Phase 22
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(22): research batch sampling phase domain
* docs(22): create phase plan for batch sampling
* test(22-01): add failing tests for FlexibleBatchSampler
- Experiment-aware batching: single-experiment restriction, all experiments appear
- Condition balancing: 2-condition and 3-condition proportional tests
- Leaky mixing: zero leak, 20% leak injection, no-effect when not experiment-aware
- Small group fallback: no crash, warning emission
- Determinism: same seed/epoch reproduces, set_epoch changes sequence
- Sampler protocol: yields list[int], correct __len__
- DDP partitioning: disjoint interleaved batches across ranks
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(22-01): implement FlexibleBatchSampler with experiment-aware, condition-balanced, leaky mixing
- FlexibleBatchSampler(Sampler[list[int]]) with cascade batch construction
- experiment_aware=True restricts each batch to a single experiment
- condition_balanced=True balances condition representation per batch
- leaky > 0.0 injects cross-experiment samples into restricted batches
- Deterministic via np.random.default_rng(seed + epoch)
- DDP support via interleaved batch partitioning across ranks
- Small group fallback to replacement sampling with logged warning
- Pre-computed group indices at __init__ for O(1) lookup
- Fix lint issues in test file (import sorting, .values -> .to_numpy(), nunique -> len(unique))
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor(22-01): export FlexibleBatchSampler from viscy_data package
- Add FlexibleBatchSampler to viscy_data.__init__.py public API
- Place import in alphabetically correct position for ruff isort compliance
- Add to __all__ exports under Utilities section
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(22-01): complete FlexibleBatchSampler core plan
- Create 22-01-SUMMARY.md with TDD execution results
- Update STATE.md: plan 01/02 complete, decisions, session continuity
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test(22-02): add failing tests for temporal enrichment, DDP coverage, validation
- 6 temporal enrichment tests (focal concentration, global_fraction edge cases, validation)
- 5 DDP disjoint coverage tests (interleaving, coverage, epoch reproducibility)
- 3 validation guard tests (missing experiment/condition/hpi columns)
- 2 package import tests (import, __all__)
- All 9 new feature tests fail as expected (RED)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(22-02): implement temporal enrichment, validation guards, DDP coverage
- Add temporal_enrichment, temporal_window_hours, temporal_global_fraction params
- Implement _enrich_temporal: focal/global sampling from experiment pool
- Add column validation guards for experiment/condition/hpi columns
- Conditional precomputation: only groupby columns when feature enabled
- Fix stale smoke test __all__ count (45 -> 46) from Plan 01
- All 35 sampler tests pass, 107 total viscy-data tests pass
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(22-02): complete temporal enrichment + DDP plan
- 22-02-SUMMARY.md with all metrics, decisions, deviations
- STATE.md advanced to Phase 23, progress 22/25 (88%)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(phase-22): complete batch sampling phase execution
Phase 22 verified: FlexibleBatchSampler with all 5 SAMP requirements.
5/5 must-haves passed. 35 tests, 107 full suite pass. No regressions.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(23): create phase plan for Loss & Augmentation
* test(23-01): add failing tests for NTXentHCL
- 12 test cases covering subclass, beta=0 equivalence, hard negatives,
gradients, temperature effect, edge cases, defaults, and CUDA
- All fail with ModuleNotFoundError (dynaclr.loss not yet created)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test(23-02): add failing tests for ChannelDropout and variable tau sampling
- 11 tests for ChannelDropout: zeros, probability bounds, eval mode, per-sample, dtype, input safety, multi-channel, CUDA
- 7 tests for sample_tau: range, exponential decay, uniform, single value, determinism, return type
* feat(23-02): implement ChannelDropout and variable tau sampling
- ChannelDropout nn.Module: per-sample channel zeroing on (B,C,Z,Y,X) tensors
- sample_tau: exponential decay weighted sampling for temporal offsets
* feat(23-01): implement NTXentHCL with hard-negative concentration
- NTXentHCL subclasses NTXentLoss from pytorch_metric_learning
- beta=0.0 delegates to parent for exact numerical equivalence
- beta>0 applies exp(beta*sim) reweighting on negatives in denominator
- Normalized weights preserve loss magnitude across beta values
- All 11 tests pass (1 CUDA test skipped on macOS)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor(23-02): add ChannelDropout and sample_tau to package exports
- Export ChannelDropout from viscy_data top-level
- Export sample_tau from dynaclr top-level
- Include NTXentHCL export added by linter
* docs(23-02): complete ChannelDropout and tau sampling plan
- Summary with TDD metrics, decisions, self-check
- STATE.md updated for Phase 23 completion
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(23-01): complete NTXentHCL loss plan
- Created 23-01-SUMMARY.md with TDD execution results
- Updated STATE.md with HCL implementation decisions
- Self-check passed: all artifacts and commits verified
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(phase-23): complete loss & augmentation phase execution
Phase 23 verified: NTXentHCL (3/3 LOSS reqs), ChannelDropout (AUG-01),
sample_tau (AUG-03). AUG-02 wiring deferred to Phase 24 by design.
30 tests pass across loss, channel_dropout, tau_sampling.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(24): create phase plan
* test(24-01): add failing tests for MultiExperimentTripletDataset
- 7 test cases covering __getitems__ return format, norm_meta, lineage-aware
positive sampling, division event traversal, channel remapping, predict mode,
and dataset length
- All tests fail with ModuleNotFoundError (RED phase)
* feat(24-01): implement MultiExperimentTripletDataset with lineage-aware sampling
- __getitems__ returns batch dicts with anchor/positive Tensors (B,C,Z,Y,X)
- Lineage-aware positive sampling via pre-built (experiment, lineage_id) lookup
- Division events traversed naturally via shared lineage_id
- Per-experiment channel remapping using registry.channel_maps
- Tensorstore I/O with SLURM-aware context and per-FOV caching
- Predict mode returns anchor + TrackingIndex dicts
- Exponential decay tau sampling with fallback to full range scan
* refactor(24-01): add MultiExperimentTripletDataset to package exports
- Export from dynaclr.__init__ for public API access
* docs(24-01): complete MultiExperimentTripletDataset plan
- SUMMARY.md with TDD commits, decisions, self-check
- STATE.md updated: position 24-01, decisions, session continuity
* update uv
* test(24-02): add failing tests for MultiExperimentDataModule
- 6 test cases covering hyperparameter exposure, experiment-level split,
FlexibleBatchSampler wiring, val dataloader, transforms, ChannelDropout
- RED phase: all tests fail with ModuleNotFoundError
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(24-02): implement MultiExperimentDataModule with experiment-level split
- MultiExperimentDataModule composes FlexibleBatchSampler + Dataset +
ChannelDropout + ThreadDataLoader with collate_fn=lambda x: x
- Train/val split by whole experiments via val_experiments parameter
- All sampling, augmentation, and loss hyperparameters exposed as __init__ params
- on_after_batch_transfer applies normalizations + augmentations + final crop
+ ChannelDropout with proper norm_meta handling for all-None case
- 6 TDD tests passing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor(24-02): add MultiExperimentDataModule to dynaclr package exports
- Import MultiExperimentDataModule from dynaclr.datamodule
- Add to __all__ for top-level importability
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(24-02): complete MultiExperimentDataModule plan
- Summary with TDD commits, decisions, and deviation documentation
- STATE.md updated: Phase 24 complete, 96% progress, ready for Phase 25
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(phase-24): complete dataset & datamodule phase execution
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(25): create phase plan
* docs(phase-25): complete integration phase plan
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(25-01): add end-to-end multi-experiment integration tests
- Create test_multi_experiment_fast_dev_run: 2 experiments with different
channel sets (GFP vs RFP), fast_dev_run with NTXentHCL loss
- Create test_multi_experiment_fast_dev_run_with_all_sampling_axes:
experiment_aware + condition_balanced + temporal_enrichment enabled
- Synthetic data helpers for multi-channel HCS OME-Zarr creation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(25-01): add multi-experiment YAML config and class_path validation test
- Create multi_experiment_fit.yml with MultiExperimentDataModule,
NTXentHCL loss, all sampling axes, generic channel names (ch_0/ch_1)
- Add test_multi_experiment_config_class_paths_resolve validating all
class_path entries in the config resolve to importable Python classes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(25-01): complete integration plan - milestone v2.2 complete
- Add 25-01-SUMMARY.md documenting end-to-end integration validation
- Update STATE.md: phase 25/25 complete, progress 100%, milestone v2.2 done
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(phase-25): complete integration phase execution — v2.2 milestone shipped
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* add the smoothness and dynamic range comparison
* add the applications/qc
* bug qc metrics exposing the device
* add batch predict
* adding cli for reduce dimensionality composable
* add example configs for model comparision and smoothness
* add the biological annotations to the zattrs
* adding airtable logic
* harmonize and remove duplication between airtable and qc. moving most things to airtable
* cleanup readme for airtable
* add callback to store embeddings every n epochs and store metadata to the anndata.uns
* fix the apply-linear classifiers to make sure we use the model and version.
* Exclude untracked applications/dynacell from uv workspace
Local debris directory (hydra outputs, pycache) has no pyproject.toml
and breaks uv lock when matched by applications/* glob.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(26): capture phase context
* docs(state): record phase 26 context session
* docs(26): create phase plans
* feat(26-01): extract HCSPredictionWriter to viscy-utils callbacks
- Create prediction_writer.py with HCSPredictionWriter, _pad_shape, _resize_image, _blend_in
- Use TYPE_CHECKING guard for viscy_data imports (HCSDataModule, Sample)
- Add numpy-style docstrings to all functions and class
- Re-export HCSPredictionWriter from callbacks __init__.py
- Fix pre-existing INDEX_COLUMNS -> ULTRACK_INDEX_COLUMNS in embedding_snapshot.py and embedding_writer.py
- Add missing docstrings to embedding_snapshot.py public methods
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(26-01): extract MixedLoss to viscy-utils losses submodule
- Create losses/ submodule with mixed_loss.py containing MixedLoss class
- Uses ms_ssim_25d from viscy_utils.evaluation.metrics internally
- Convert docstrings to numpy-style
- Re-export MixedLoss from losses __init__.py
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(26-01): create translation application scaffold with workspace registration
- Create applications/translation/ with src layout following dynaclr pattern
- Add pyproject.toml with hatchling build, uv-dynamic-versioning, and workspace deps
- Add README.md required by hatchling readme field
- Add __main__.py delegating to viscy_utils.cli.main for LightningCLI entry point
- Add example YAML configs (fit.yml, predict.yml) with HCSPredictionWriter callback
- Create empty tests/__init__.py
- Register viscy-translation in root pyproject.toml workspace sources
- Update uv.lock with new workspace member
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(26-01): complete shared infra extraction + app scaffold plan
- Create 26-01-SUMMARY.md with execution results
- Update STATE.md with plan progress, decisions, session info
- Update ROADMAP.md marking 26-01 as complete
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(26-02): migrate translation engine and evaluation modules
- Copy engine.py with VSUNet, FcmaeUNet, AugmentedPredictionVSUNet, MaskedMSELoss
- Copy evaluation.py with SegmentationMetrics2D
- Update all imports to new package paths (viscy_data, viscy_models, viscy_utils)
- Remove MixedLoss class from engine.py (now imported from viscy_utils.losses)
- Update __init__.py with top-level re-exports
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test(26-02): add translation engine test suite
- Import tests for all public exports (VSUNet, FcmaeUNet, etc.)
- VSUNet init and forward pass smoke tests with synthetic data
- State dict key regression test for checkpoint compatibility
- MixedLoss integration test (from viscy_utils.losses)
- FcmaeUNet init test
- No old import paths grep test
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(26-02): complete engine migration plan
- SUMMARY.md with 2 task commits, 2 auto-fixed deviations
- STATE.md updated: phase 26 complete, 20/25 phases (80%)
- ROADMAP.md updated with plan progress
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test(26): complete UAT - 8 passed, 0 issues
* add the pseudotime evals
* re-structure pseudotime folder
* add the linear classifier evals and restructure folder path
* add evaluations to dynaclr package
* cli and linear classifier init
* fix(translation): address engine and evaluation bugs
- Fix operator precedence bug in evaluation.py boolean condition
- Fix source variable overwritten in AugmentedPredictionVSUNet loop
- Fix unbound return_target in FcmaeUNet.forward_fit_task
- Add weights_only=True to torch.load for security
- Fix mutable default argument model_config: dict = {}
- Simplify redundant Union[nn.Module, MixedLoss] to nn.Module
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(translation): correct example config keys and dependencies
- Fix monitor key loss/val -> loss/validate in fit.yml
- Fix output_path -> output_store param name in predict.yml
- Add lightning>=2.3 as direct dependency
- Remove unused torchvision, pin torchmetrics>=1
- Remove unused SYNTH_OUT_C from conftest
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(viscy-utils): fix prediction_writer indexing and embedding_writer overwrite flag
- Fix numpy indexing in prediction_writer _blend_in broadcast shape
- Fix tuple[int] -> tuple[int, ...] type annotation in _create_image
- Respect overwrite flag in EmbeddingWriter.on_predict_start
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(viscy-utils): correct annotation.py docstring examples
- Fix function name in example: convert_xarray_annotation_to_anndata -> convert
- Fix obsm keys in example: X_PCA/X_UMAP/X_PHATE -> X_pca/X_umap/X_phate
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(viscy-models): prevent Sequence mutation and ZeroDivisionError
- Avoid mutating Sequence input in UNeXt2Decoder.forward
- Add input validation in UNeXt2Stem to prevent ZeroDivisionError
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address PR review blockers (airtable security, tests, qc cleanup)
- Remove api_key/base_id params from AirtableDatasets.__init__; read
credentials exclusively from env vars with clear ValueError on missing
- Add 59 tests for airtable_utils (database + schemas) with full mocking
- Wrap open_ome_zarr in context manager in qc/annotation.py to prevent
file handle leaks on exceptions
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix tests
* add track timing backup, save the aucroc as metric for linear classifier, and fix the overwritting by the linear classifiers.
* fix(viscy-utils): validate every_n_epochs >= 1 in EmbeddingSnapshotCallback
Prevent ZeroDivisionError in _should_collect when every_n_epochs=0.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* consolidate appending column to anndata functionality from reduce dimension and linear classifier
* add DINOv3 to viscy-data
* refactor dynaclr app folder structure
* move losses to viscy-models
* data folder
* porting #360
* rname dino to foundation and support openphenom
* move shells cripts to the configs folder
* fix the import for opephenom
* generalize the qc class
* fix(viscy-data): skip None norm_meta in SlidingWindowDataset collation
Pre-existing bug: when a zarr has no normalization metadata,
sample["norm_meta"] = None was added to the batch dict, causing
default_collate to crash on NoneType. Only add norm_meta when
it is not None.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test(translation): add inference reproducibility test for vscyto3d
Validates refactored FcmaeUNet matches old monolithic code predictions.
Reference generated from main branch on a 512x512 crop of the mehta-lab
VSCyto3D test dataset with fov_statistics normalization.
- test_checkpoint_loads: 0 missing/unexpected state dict keys
- test_predict_and_match_reference: full pipeline (HCSDataModule +
HCSPredictionWriter + VisCyTrainer), Pearson r > 0.999, atol=0.02
HPC-gated: skips when checkpoint/data/reference paths unavailable.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(translation): use self.log_dict instead of self.logger.log_metrics
self.logger.log_metrics() crashes when no logger is attached and
bypasses Lightning's built-in aggregation/sync. Use self.log_dict()
which handles missing loggers and DDP sync correctly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(dynaclr): avoid CPU pos_weight device mismatch in ClassificationModule
pos_weight=torch.tensor(1.0) as a default argument is created on CPU
at class definition time. When the module is moved to GPU,
BCEWithLogitsLoss errors due to device mismatch. Move the default
into __init__ body so the tensor is created at instantiation time.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(dynaclr): fix typo in create_pseudo_tracks docstring
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* add tqdm as default instead of Rich. Rich doesnt show up on the stdout of slurm jobs until it's done.
* de parallelize and default cosine distance for msd and knn for PHATE.
* make a cli for anndata
* add append to obs cli
* adding a cell index that standardizes and spits out parqet
* add example cell index
* update multiexperiment datamodule
* recipes
* add parquet integration test for multi-experiment training
Adds test_multi_experiment_fast_dev_run_with_parquet which verifies the
full Lightning training loop works when loading from a pre-built cell
index parquet via MultiExperimentDataModule.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* demo slurm code
* obs should use fov_name and track_id and id as UUID
* add marker, task and channels to compute the crossval.
* fix prediction remodeling analysis
* test(translation): add training integration tests for VSUNet and FcmaeUNet
Add 6 fast_dev_run tests validating the forward+backward pass:
- VSUNet with MSELoss and MixedLoss (synthetic data)
- FcmaeUNet pretraining (MaskedMSELoss) and fine-tuning (synthetic data)
- VSUNet and FcmaeUNet with real HCSDataModule/CachedOmeZarrDataModule
Also remove architecture-specific language from package description.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test(translation): add config class_path resolution tests
Validate that all class_path entries in fit.yml and predict.yml resolve
to importable classes, matching the DynaCLR test pattern.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Raw logits are now passed through sigmoid before computing binary_accuracy and
binary_f1_score
* fix the dataset statistics
* add contributing and uv to claude.md
* pydantic >2.0
* - Add `schemas.py` with shared Pydantic data models (FOVRecord, etc.)
- Add `collection.py` for ML training collection definitions
- Rename `hours_post_infection` → `hours_post_perturbation` in `_typing.py`
- Update `cell_index.py` to use `collection_path` and renamed field
- Add corresponding tests for new modules and updated cell index
* uvx precommit
* refactor(viscy-data): replace condition_balanced with stratify_by in sampler
* add norm_meta batching helpers and triplet None guard
* - Apply _radians_to_degrees to shear_range (Kornia expects degrees, calls deg2rad internally)
- Fix docstrings to reflect ZYX input order, facet naming, and unit conventions
- Remove stale timepoint_statistics lookup in _normalize.py
* extend DatasetRecord from FOVRecord in viscy-data schemas; add viscy-data dep
* - ExperimentRegistry backed by Collection for per-experiment channel/norm maps
- MultiExperimentIndex with parallel FOV loading and lineage-aware anchors
- MultiExperimentDataModule with stratify_by, num_workers_index, FOV-level split
- Dataset updates for new pipeline; remove ExperimentConfig from __init__
- Consolidate shared test helpers and constants into conftest.py
* pass batch_size to Lightning metric logging for correct step counts
* update recipes and add sampling-strategies guide; add CLAUDE.md
- Update build-cell-index, train-multi-experiment, troubleshooting recipes
- Add sampling-strategies.md documenting FlexibleBatchSampler axes
- Update README table
- Add CLAUDE.md with data pipeline design principles
* update pseudotime and linear classifier scripts for renamed fields
* add configs
* physical nomrlaiztion and fix to mlp projetion layer for adapter.
* code review fixes: remove _components/, assert→ValueError, docstrings, logging
- Remove duplicate viscy_models/_components/ (all imports already use components/)
- Replace assert with raise ValueError in conv_block_2d/3d.py
- Fix numpy docstrings in viscy_data/_utils.py (_ensure_channel_list, _collate_samples)
- Narrow broad except Exception to (OSError, ValueError) in hcs.py
- Use direct row["parent_track_id"] access in cell_index.py (column guarded above)
- Fix frame interval mode().iloc[0] IndexError risk in pseudotime/metrics.py
- Remove backwards-compat re-exports from qc/config.py
- Add logging + warn on silent AUROC ValueError in linear_classifier.py
- Add exc_info=True to PHATE/PCA warning in embedding_writer.py
- Remove resolved TODO in feature.py
- Add _logger and replace print() with logging in dynaclr/utils.py and qc/qc_metrics.py
- Warn when logger_base path does not exist in dynaclr/utils.py
- Move inline imports to top in test_sampler.py
- Add type hints to stems.py compute_stem_channels()
- Remove redundant num_channels reassignment in beta_vae_25d.py
- Fix prek reference in CLAUDE.md (was pre-commit)
- Fix input_channel comment/value: organelle→marker in example config
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix tests: remove test_smoke, update config fields, fix parquet dtype
- Remove packages/viscy-data/tests/test_smoke.py: not a real integration
test (magic __all__ count, source string matching, no functional coverage)
- Update TestLinearClassifierTrainConfig: embedding_model+wandb_project
replaced by embedding_model_name+embedding_model_version in schema
- Fix test_parquet_lineage_preserved: add check_index_type=False to handle
object vs StringDtype difference between legacy and parquet paths
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* rename organelle→marker channel type; fix ArrowString/schema test failures
- Rename "organelle" → "marker" in VALID_CHANNELS, pseudotime plotting,
and test fixtures to match the unified channel type naming
- Fix tests: set pd.options.future.infer_string=False in conftest to prevent
pandas 2.x ArrowStringArray from breaking anndata zarr writer
- Fix test_loss: torch.Generator(device=device) for CUDA compatibility
- Update TestLinearClassifierInferenceConfigOrganelle to new schema
(embedding_model_name/version + models list)
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* add experiment configs and SLURM scripts
- Update example_cell_index.yaml with real dataset paths and rename
hours_post_infection → hours_post_perturbation
- Add collection YAMLs: A549_ZIKV_multiorganelle, A549_bag_of_channels, example
- Add training fit configs for A549_ZIKV_multiorganelle and A549_bag_of_channels
- Add smoothness evaluation SLURM scripts
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* add linear classifier and pseudotime analysis scripts
- generate_classifier_inference.py: generate inference configs + SLURM
script for a model predictions folder
- generate_train_config_from_folder.py: generate training configs from
prediction folders, supports multi-dataset combine
- label_offset_sweep.py: sweep temporal label offsets for infection
classifier to find optimal onset labeling
- infection_death_remodeling.py: correlate infection, death, and
organelle remodeling event timings across tracks
- infection_onset_distribution.py: compute and plot infection onset
distributions from classifier predictions
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix test_inference_reproducibility: remove stale reference zarr dependency
Replace comparison against a pre-computed reference zarr (39170 cells,
now stale) with a self-contained determinism test: run inference twice
with the same seed and assert the outputs match within GPU tolerance.
This removes the brittle hardcoded cell count and the reference zarr
that needs to be regenerated whenever the data or pipeline changes.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* feat(translation): add sliding window volume prediction (PR #280 port)
Port PR #280 ("Predict volume") functionality to the modular architecture.
- Enhance _blend_in in prediction_writer.py to support both torch.Tensor
(5D: B,C,Z,Y,X) and np.ndarray (4D: C,Z,Y,X) with unified blending
- Add predict_sliding_windows() to AugmentedPredictionVSUNet for
in-memory Z-sliding inference with linear feathering blending
- Extract _predict_with_tta() helper from predict_step()
- Make forward_transforms/inverse_transforms optional (default identity)
- Add getattr guard for out_stack_depth (clear error for 2D models)
- Add 7 tests: blend_in consistency/edge cases, sliding window shape,
invalid input, missing attribute, optional transforms
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix examples and configs: rename organelle→marker channel type, add overwrite=True to EmbeddingWriter
- quickstart.py/ipynb: add overwrite=True to EmbeddingWriter to prevent
FileExistsError on notebook re-run
- cross_validate_example.yaml: channels [phase, sensor, organelle] → [phase, sensor, marker]
- example_linear_classifier_inference.yaml: update W&B artifact names
organelle_state-organelle-* → organelle_state-marker-*
- example_linear_classifier_train.yaml: update comment and example
embedding paths to use marker instead of organelle
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(viscy-models): port UNeXt2Stem validation guards to components/stems.py
Copy the in_stack_depth and out_channels divisibility checks from
_components/stems.py to components/stems.py before the upcoming merge
deletes _components/. Without these guards, invalid parameters cause
a silent ZeroDivisionError.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* remove ed notes
* missing pyarrow in dynaclr install. dpeend on visc-data triplet optional dependency
* fix troubleshooting.md for MultiExperiment setup
* add wandndb as core dependency
* remove the data.log
* fix(monorepo): declare missing deps, fix test infrastructure for root pytest
Dependency fixes (all were missing from pyproject.toml):
- viscy-utils: add wandb, anndata, tensorboard as core deps; remove
redundant optional-dependencies.anndata group
- viscy-data: promote pandas and pyarrow from optional/test to core deps
- viscy-models: add pytorch-metric-learning as core dep
- dynaclr: add statsmodels to eval and test dep groups
Test infrastructure:
- Extract shared test helpers/constants from conftest.py into
helpers.py so tests can import them under --import-mode=importlib
(from conftest import broke when running pytest from repo root)
- Remove pythonpath=["tests"] from dynaclr pyproject.toml (was a
workaround for running from the app dir; root config now handles it)
- Add pythonpath=["applications/dynaclr/tests"] to root pytest config
- Move pd.options.future.infer_string=False into pytest_configure hook
so imports stay at top level (fixes ruff E402)
Bug fixes:
- hcs.py: scope propagate=False to viscy_data.hcs.cache logger instead
of viscy_data parent, which was breaking caplog in downstream tests
- Remove test_small_group_emits_warning: tested log wording not behavior
- Remove test_nan_gene_name_to_ntc: tested pandas fillna not module code
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* docs(dynaclr): add tracking note for anndata ArrowString zarr bug
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* chore: remove tests/__init__.py files (importlib mode doesn't need them)
--import-mode=importlib does not use package-style imports for test
files, so __init__.py in test directories is unnecessary and can cause
import collisions.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(dynaclr): resource leak, None propagation, CSV ambiguity
- Wrap open_ome_zarr() in context managers in index.py and cell_index.py
- Raise RuntimeError in _sample_positives() when no positive found instead of silent self-positive fallback
- Raise FileNotFoundError when tracking CSV is missing (fail fast before training)
- Raise ValueError when multiple CSVs exist in a tracks dir
- Update test_empty_tracks_empty_anchors → test_missing_tracking_csv_raises
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* chore(dynaclr): minor cleanup in inspect_dataloader and test formatting
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* add package and applications folder rules for CLAUDE.md
* context managers rule
* remove __version.py
* delete gsd markdowns
* chore: remove stale applications/dynacell directory
Remove old Hydra output logs and cache files from the defunct dynacell
application. This directory was already excluded from the uv workspace.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Eduardo Hirata-Miyasaki <edhiratam@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Alexandr Kalinin <alxndrkalinin@users.noreply.github.com>
Rename the virtual staining application to match project branding. Follows Pattern A naming convention (like dynaclr, qc) — no viscy- prefix. - Directory: applications/translation → applications/cytoland - Package: viscy-translation → cytoland - Imports: from viscy_translation.engine → from cytoland.engine - Config class_paths: cytoland.engine.VSUNet - CLI: python -m cytoland fit/predict - README rewritten to match dynaclr style with paper reference - Root README updated with Cytoland in Applications table Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Rename the virtual staining application to match project branding. Follows Pattern A naming convention (like dynaclr, qc) — no viscy- prefix. - Directory: applications/translation → applications/cytoland - Package: viscy-translation → cytoland - Imports: from viscy_translation.engine → from cytoland.engine - Config class_paths: cytoland.engine.VSUNet - CLI: python -m cytoland fit/predict - README rewritten to match dynaclr style with paper reference - Root README updated with Cytoland in Applications table Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…equired,
matches runtime ultrack columns)
- Add log_embeddings_every_n_epochs to ContrastiveModule; logs UMAP colored
by condition/experiment/HPI to WandB on validation epochs
- Fix detach_sample to split channels as extra columns (landscape-friendly)
- Guard MultiExperimentTripletDataset.__getitem__ with NotImplementedError
- Add demo scripts for WandB image and UMAP logging
- Add engine tests for embedding accumulation and epoch-gating logic
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Alexandr Kalinin <alxndrkalinin@users.noreply.github.com> Co-authored-by: Eduardo Hirata-Miyasaki <edhiratam@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* refactor: clean up viscy-utils optional dependencies
* refactor: clean up dynaclr test imports
* fix: lazy import annotation dependencies
* feat: add fnet model for cytoland benchmarking
* remove deprectaed files that were left over from the monorepo restructuring
* CI to test applications/ loading each test-applications matrix job cds into the app directory before running pytest.
* fix: resolve root pytest conftest collision and lazy-import linear_classifier deps
Remove applications/*/tests from root testpaths — multiple app conftest.py
files collide as the same plugin name when collected together. Apps are now
tested per-directory via CI matrix (added in prior commit).
Lazy-import wandb and anndata in linear_classifier.py (same pattern as
annotation.py) so the module is importable without optional extras.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: add e2e FNet3D test and move unit tests to test_unet/
- Add test_fnet3d_real_datamodule_fast_dev_run: end-to-end test
exercising HCSDataModule → VSUNet(FNet3D) → Trainer
- Move test_unet3d.py from tests/ root to tests/test_unet/
matching the location of other UNet test files
- Add test_state_dict_keys verifying recursive key structure
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: extract _make_divisible_pad helper and simplify Unet3d
- Extract duplicated DivisiblePad construction into _make_divisible_pad()
- Cache 2**depth as self._divisor in Unet3d.__init__
- Make downsamples_z a class attribute instead of a @property
- Remove trivial docstring on _DoubleConv3d.forward
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: enable root pytest to collect all packages and applications
Remove pythonpath = ["tests"] from app configs and __init__.py from app
test dirs — both caused conftest plugin collisions when root pytest
collected multiple applications together. Replace all `from conftest
import` / `from .conftest import` with either inlined constants or
fixture factories so test files no longer need conftest on sys.path.
Root testpaths now includes applications/*/tests, so `uv run pytest`
runs all 836 tests (packages + applications) in a single invocation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: consolidate duplicated test constants into conftest fixtures
Replace inlined SYNTH_C/D/H/W, IMG_H/W, N_T/Z/TRACKS, FCMAE_H/W, and
MIXED_LOSS_H/W constants with synth_dims and hcs_dims fixtures returning
dicts. Single source of truth in each app's conftest.py.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: restore empty src/viscy/__init__.py for wheel build
Hatchling requires the package directory referenced in
[tool.hatch.build.targets.wheel] to exist. Without it, `uv build`
fails and `import viscy` breaks.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add Spotlight foreground-aware loss with precomputed masks (#389)
* feat: add SpotlightLoss for foreground-aware virtual staining
Implement Spotlight (Kalinin et al. 2025, arXiv:2507.05383), a
model-agnostic loss function that focuses supervision on biologically
relevant foreground regions via:
- Masked MSE using per-sample Otsu thresholding on targets
- Dice loss on soft-thresholded predictions (tunable sigmoid)
Works as a drop-in loss_function for any VSUNet architecture.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: document Spotlight paper deviations and implementation choices
- Add config comments listing all deviations from the paper: optimizer,
normalization, patch selection, target normalization, training length,
isotropic voxels
- Add code comments explaining tunable sigmoid clamping rationale and
zero-foreground fallback behavior
- Fix _otsu_threshold_batch docstring return shape
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address Spotlight paper deviations
- Normalize both source and target channels in config (paper A.4)
- Use max_steps: 50000 instead of max_epochs (paper A.3)
- Add per-sample min_foreground_fraction filtering to SpotlightLoss
(paper A.4: only patches with ≥0.1% FG voxels used for training)
- FG fraction computed per-sample, not batch-wide
- Add min_foreground_fraction: 0.001 to example config
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: simplify SpotlightLoss foreground filtering logic
- Co-locate soft_pred weighting with mask weighting in same block
- Remove misleading sample_weight is not None guard
- Use (pred * 0).sum() for zero-gradient return (maintains graph)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: validate eps, n_bins, min_foreground_fraction in SpotlightLoss
Add constructor validation for remaining parameters to catch
misconfiguration early, consistent with existing sigmoid_k/lambda_mse
checks.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: fix misleading z-score normalization comment in SpotlightLoss
The docstring incorrectly implied z-score normalization was what the
paper uses. Rewritten to acknowledge this is a deviation: the paper
subtracts the Otsu threshold, while the implementation recomputes
Otsu inside forward() on already-normalized data.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: precomputed FOV-level Otsu thresholds for Spotlight
- Add opt-in compute_otsu to generate_normalization_metadata() and
viscy preprocess CLI — stores per-FOV Otsu threshold in norm_meta
- Add min_foreground_fraction to SlidingWindowDataset/HCSDataModule
with bounded retry loop (max 10 random retries) for patch filtering
- Refactor SpotlightLoss: replace n_bins + min_foreground_fraction
with fg_threshold (None=Otsu fallback, float=fixed threshold)
- Update config: use subtrahend=otsu_threshold, fg_threshold=0.0,
min_foreground_fraction=0.001
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: generalize foreground filtering to nonzero fraction check
- Rename min_foreground_fraction → min_nonzero_fraction in
SlidingWindowDataset and HCSDataModule
- Add nonzero_threshold (default 0.0) and nonzero_channel (default
None → first target) parameters for method-agnostic filtering
- Remove coupling to Otsu thresholds in norm_meta — check is now a
simple (patch >= threshold) comparison on the configured channel
- Validate nonzero_channel against channel map in __init__
- Fix MaskTestDataset to forward **kwargs to super().__init__()
- Hoist check_key before retry loop, fix attribute ordering
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: configurable max_nonzero_retries with exhaustion warning
- Add max_nonzero_retries parameter (default 100) to
SlidingWindowDataset and HCSDataModule
- Log warning when retries are exhausted instead of silently
returning a below-threshold sample
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: remove papers directory
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: use local-mean downsampling for robust Otsu thresholding
Point-sampling at stride 32 loses spatial structure needed for clean
bimodal histograms. Replace with downscale_local_mean (default factor
4) that averages local neighborhoods, preserving the FG/BG separation
that Otsu needs. Only affects the compute_otsu path — existing
grid sampling for mean/std is unchanged.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use denser grid + median filter for Otsu instead of full-volume read
Replace _downsample_local_mean (which loaded the entire FOV at full
resolution) with a denser _grid_sample at otsu_grid_spacing=8 followed
by a median filter (size=3). This is fast (sparse tensorstore reads)
while providing enough spatial density to capture inter-cell gaps.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: hoist Otsu imports and restrict median filter to spatial dims
- Move scipy/skimage imports before the position loop (avoid repeated
import lookups per FOV)
- Fix median_filter size from (3,3,3,3) to (1,1,3,3) — only smooth
Y/X, not across timepoints or Z-slices
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: add preprocessing tests for compute_otsu path
- test_compute_otsu_stores_threshold: verifies otsu_threshold key
is written to fov_statistics when compute_otsu=True
- test_compute_otsu_threshold_separates_bimodal: verifies threshold
falls between BG and FG modes on bimodal fluorescence data
- test_compute_otsu_false_omits_threshold: verifies otsu_threshold
is NOT present when compute_otsu=False (default)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: precompute foreground masks for Spotlight training
Add `generate_fg_masks()` to precompute binary FG masks during
preprocessing (smooth + Otsu threshold), store as zarr arrays, and
thread through the data pipeline to SpotlightLoss.
This eliminates the mismatch where Otsu thresholds were computed on
smoothed/subsampled data but applied to raw patches at training time,
producing noisy FG/BG masks at boundaries.
Following pytorch_fnet's WeightedMSE weight_map_batch pattern:
- Preprocessing: `generate_fg_masks()` smooths full-res data per FOV
per timepoint and stores binary masks as "fg_mask" zarr arrays
- Data loading: `SlidingWindowDataset` loads masks alongside images,
threads through MONAI transforms via temp keys for spatial co-alignment
- Loss: `SpotlightLoss.forward()` accepts optional `fg_mask` argument
(priority: precomputed > fixed threshold > runtime Otsu)
- Engine: `VSUNet._compute_loss()` helper passes mask from batch to loss
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address PR review feedback on nonzero filtering
- Add parameter validation for min_nonzero_fraction and max_nonzero_retries
- Fix retry loop: check fraction on every attempt (including last),
warn only when all attempts fail the criterion
- Scope nonzero filtering to training only via _train_filter_settings
property (val/test/predict are deterministic, no random resampling)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: address code review — hoist imports, use context managers
- Move scipy/skimage imports to top of meta_utils.py (CLAUDE.md rule)
- Move generate_fg_masks and SlidingWindowDataset imports to top of
test files
- Use context managers for zarr stores in test fixtures
- Replace inline `import pytest` with top-level `raises` import
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: per-channel SpotlightLoss for multi-target training
Compute loss per (batch, channel) pair instead of globally:
- Masked MSE: channels with FG mask use masked MSE, channels without
fall back to regular MSE
- Dice: only channels with FG mask data contribute; channels without
are excluded from the Dice average
- Otsu: compute per-(sample, channel) thresholds instead of per-sample
This correctly handles multi-channel targets where masks exist for
only some channels (e.g., Nuclei mask but not Membrane mask).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add fg_mask_channels param to preprocess CLI
Allow specifying which channels to compute FG masks for, independently
from which channels get normalization/Otsu. Defaults to all channels
with Otsu thresholds. Enables e.g. masking only Nuclei when training
with both Nuclei and Membrane targets.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use all-ones default for non-target mask channels
Channels without explicit FG masks should get full supervision (all
voxels contribute to loss), not zero supervision. Changed from
np.zeros to np.ones for the mask array initialization.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: exclude all-ones mask channels from Dice loss
All-ones masks (placeholder for "no real FG/BG mask") should not
contribute to Dice — they would penalize the model for not predicting
everything as foreground. Now only channels with real masks (both
0s and 1s) contribute to Dice. If no channel has a real mask, Dice
is zero and a warning is logged.
Also simplified MSE path: all-ones masks naturally give regular MSE
via masked_sum / fg_per_ch = sq_err.sum() / n_spatial, eliminating
the need for a torch.where fallback.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: use warnings.warn for no-mask Dice, fix test coverage
- Replace _logger.warning with warnings.warn (built-in dedup prevents
log spam on every forward call)
- Fix test_partial_mask_ignores_placeholder_channel_in_dice to use a
real FG/BG mask on channel 0 (not all-ones which has_real_mask
correctly excludes)
- Remove redundant "what" comment
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: use _logger.warning with one-shot flag for no-mask Dice
Replace warnings.warn with _logger.warning guarded by
_warned_no_real_mask flag — fires once per SpotlightLoss instance.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address Copilot review comments on PR #389
- Remove unused num_workers param from generate_fg_masks
- Fix all-zero mask MSE: fall back to unmasked MSE via torch.where
- Use keyword arg for fg_mask in _compute_loss (avoids TypeError
with non-Spotlight losses)
- Fix check_key operator precedence (nonzero_channel was bypassing
min_nonzero_fraction=0 guard)
- Guard mask-based nonzero check to target channels only (source
channels fall back to raw threshold)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Alexandr Kalinin <alxndrkalinin@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: adapt FNet3D tests to fixture pattern after merge
FNet3D tests use hardcoded in_stack_depth=4 (not synth_dims["d"]=5)
because FNet3D requires Z divisible by 2^depth. Also fixed attribute
name (_predict_pad, not predict_pad) and removed synthetic_batch
dependency since FNet3D needs its own tensor shape.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add scipy dependency and ndim validation for Unet3d
- Add scipy as explicit dependency in viscy-utils (directly imported
in meta_utils.py and evaluation modules, not just transitive)
- Add ndim != 5 check in Unet3d.forward() to catch 4D input with a
clear error instead of a misleading spatial dimension mismatch
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: stream fg_mask writes to avoid OOM on large FOVs
Replace in-memory full-FOV allocation with streaming writes:
use pos.create_zeros() to allocate the zarr array on disk, then
write per-timepoint per-channel slices. For a typical FOV
(80×6×50×2048×2048), this reduces memory from ~96GB to ~16MB
(one Z×Y×X slice at a time).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: migrate cytoland logging from TensorBoard to W&B
Replace TensorBoard-specific _log_samples with the existing
log_image_grid() helper from log_images.py which dispatches to both
TensorBoard and W&B. Add DDP guard (is_global_zero) to prevent
duplicate image logging in distributed training.
Update example configs from TensorBoardLogger to WandbLogger.
No hard wandb dependency needed — import is lazy in log_images.py.
Tests keep TensorBoardLogger (log_image_grid handles both backends).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add SLURM submission template for cytoland training
Based on dynaclr's fit_slurm.sh pattern. Uses Lightning CLI with
config overrides for run name, save dir, and checkpoint path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add FCMAE pretrain/finetune configs and encoder-only checkpoint loading
Add encoder_only parameter to FcmaeUNet for loading only encoder weights
from a pretrained checkpoint, enabling fine-tuning with different output
channels (e.g., 1→2). Create example configs for self-supervised FCMAE
pretraining (pretrain_fcmae.yml) and supervised fine-tuning
(finetune_fcmae.yml) matching the patterns from pretrain_3d.py and
finetune_3d.py.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: review feedback — worker-safe RNG, clear fg_mask TypeError, Otsu comment
- Replace stdlib random.randint with torch.randint in DataLoader retry
loop (torch seeds each worker independently, avoiding correlated retries)
- Wrap fg_mask keyword dispatch in _compute_loss with TypeError catch
that names the misconfiguration (loss vs data config mismatch)
- Add derivation comment showing Otsu inter_class_var formula is correct
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: migrate FCMAE pipeline to batched GPU transforms
Move augmentation transforms from per-sample (Decollated → MONAI wrappers
→ StackChannelsd → collate) to batched GPU execution via
on_after_batch_transfer, matching the DynaCLR pattern.
Changes across 3 packages + cytoland:
viscy-transforms:
- Add BatchedStackChannelsd (inherits StackChannelsd, dim=1 cat)
- Add BatchedRandInvertIntensityd (per-sample randomization on batched tensors)
viscy-data:
- Add on_after_batch_transfer to GPUTransformDataModule base class
(dispatches train_gpu_transforms vs val_gpu_transforms)
- Add on_after_batch_transfer dispatcher to CombinedDataModule
(handles list batches for training, single dict for validation)
- Add gpu_augmentations param + on_after_batch_transfer to HCSDataModule
cytoland:
- Remove FcmaeUNet.train_transform_and_collate / val_transform_and_collate
- Add _merge_batches to concatenate per-dataset batches from CombinedLoader
- Update FCMAE configs to use Batched* transforms (no more Decollated)
- Update test fixtures to use BatchedStackChannelsd
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address PR review issues and Copilot comments
- Replace try/except TypeError in _compute_loss with inspect.signature
check that handles both explicit fg_mask param and **kwargs
- Add freeze_encoder to FcmaeUNet Parameters docstring
- Move inline imports to top of test_training_integration.py
- Restore weight channel limitation comment in hcs.py
- Replace O(N log N) _find_window with bisect.bisect_right (O(log N))
- Optimize generate_fg_masks: bulk write non-target channels, tile
zarr chunks at 256 for large FOVs
- Align BatchedRandInvertIntensityd with MONAI RandomizableTransform
pattern (randomize() + per-sample _do_transform tensor)
- Fix Airtable test_dataframe_columns expected columns
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address PR review issues and Copilot comments
- Replace try/except TypeError in _compute_loss with inspect.signature
check that handles both explicit fg_mask param and **kwargs
- Add freeze_encoder to FcmaeUNet Parameters docstring
- Move inline imports to top of test_training_integration.py
- Restore weight channel limitation comment in hcs.py
- Replace O(N log N) _find_window with bisect.bisect_right (O(log N))
- Optimize generate_fg_masks: bulk write non-target channels, tile
zarr chunks at 256 for large FOVs
- Align BatchedRandInvertIntensityd with MONAI RandomizableTransform
pattern (randomize() + per-sample _do_transform tensor)
- Fix Airtable test_dataframe_columns expected columns
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: make cytoland tta padding metadata-independent
* fix: propagate fg_mask through spatial augmentations
Spatial transforms (RandAffined, RandFlipd, etc.) now automatically
include fg_mask keys when fg_mask_key is set, ensuring masks stay
pixel-aligned with source/target after augmentation.
Uses an explicit _SPATIAL_TRANSFORMS allowlist — intensity transforms
(contrast, noise, etc.) are excluded to avoid corrupting binary masks.
Injection is idempotent (safe across repeated setup() calls).
Covers both CPU augmentations (_fit_transform) and GPU augmentations
(gpu_augmentations in __init__).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: restructure cytoland configs by model with composable recipes
Reorganize example configs from flat files (fit.yml, predict.yml) into
model-specific directories (vscyto2d/, vscyto3d/, vsneuromast/, fnet3d/)
with composable recipe fragments.
New infrastructure:
- viscy_utils.compose: PyYAML-based config composition via `base:` key
(recursive deep merge, lists replace, zero new deps)
- CLI integration: detect `base:` in --config/-c and auto-compose
before LightningCLI (no-op for configs without base:)
Model configs:
- vscyto2d: pretrain, finetune, predict (FcmaeUNet, in_stack_depth=1)
- vscyto3d: pretrain, finetune, train_spotlight, predict (UNeXt2, z=5)
- vsneuromast: fit, predict (UNeXt2, z=21, no pretraining)
- fnet3d: fit, predict (FNet3D, depth=4)
Recipes (reusable fragments):
- trainer/: fit_4gpu, predict_gpu
- models/: fcmae_2d, fcmae_3d, unext2_3d, unext2_neuromast, fnet3d
- data/: hcs_nuc_mem_{2d,3d,neuromast}, cached_pretrain
- modes/: spotlight (loss + fg_mask + Otsu normalization)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: port CellDiff models to viscy-models as optional dependency (#394)
* feat: port CellDiff models to viscy-models as optional dependency
Add UNetViT3D (deterministic) and CELLDiffNet (flow-matching backbone)
to viscy-models behind an optional `celldiff` extra ["diffusers", "einops"].
CELLDiff3DVS training wrapper and transport module deferred to Stage 3
(Dynacell application layer). Dead code from the fork dropped: CondConvNet,
BertPredictionHeadTransform, MLMHead, PixelShuffle3d, Upsample, Downsample,
init_weights, and unused 1D/2D positional embedding functions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: validate patch divisibility in CellDiff model constructors
Reject spatial sizes where latent dimensions are not exactly divisible
by patch_size after encoder downsampling. Previously, integer division
silently truncated remainders, causing the decoder to crash on torch.cat
with mismatched skip-connection shapes (e.g. input_spatial_size=[10,64,64]
with patch_size=4).
Also: fix positional embedding return type annotations (float32 -> float64),
add cond channel validation in CELLDiffNet.forward(), and add validation
tests for both models.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Alexandr Kalinin <alxndrkalinin@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address PR #388 review findings
- SLURM script: fix CLI command from `cytoland fit` to `viscy fit`
(cytoland has no [project.scripts] entrypoint)
- FcmaeUNet: add `ckpt_path` to save_hyperparameters(ignore=...) so
load_from_checkpoint works with encoder_only=True
- test_training_integration: move inline imports to module level
- celldiff models: convert constructor assert to if/raise ValueError
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: only allow missing keys in final crop when fg_mask is configured
Keep strict key validation (allow_missing_keys=False) when no fg_mask_key
is set, so mis-specified channel names fail fast at crop time instead of
silently skipping.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: unify 3D U-Net model API behind UNet3DBase (#396)
* feat: add generalized 3D conv blocks for unified U-Net base
Move Block, ResnetBlock from celldiff/modules/simple_diffusion.py to
unet/blocks.py with configurable norm (group/batch), activation
(silu/relu), and residual flag. Add TimestepEmbedder and
ConvBottleneck3D. Replace einops with plain PyTorch.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add UNet3DBase iterative encoder-decoder with injected bottleneck
Parametrized 3D U-Net base with configurable norm, activation, residual,
downsample_z, time conditioning, and conditioning input. Exposes
num_blocks property and downsamples_z attribute for engine compatibility.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add ViTBottleneck3D encapsulating CellDiff transformer bottleneck
Extracts PatchEmbed3D, sinusoidal positional embedding, TransformerBlock
stack, FinalLayer, and unpatchify into a single module with the unified
bottleneck interface forward(x, time_embeds=None).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: rewrite Unet3d (FNet) as thin wrapper of UNet3DBase
Replace recursive _FNetRecurse with iterative UNet3DBase configured for
BatchNorm+ReLU, non-residual blocks, all-dim downsampling, and conv
bottleneck. FNet weight init preserved via self.apply(). Add positive
FNet sliding-window test to cytoland engine tests.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: rewrite UNetViT3D + CELLDiffNet as thin wrappers, remove einops
Rewrite both CellDiff models as thin UNet3DBase wrappers with injected
ViTBottleneck3D. Delete simple_diffusion.py (Block/ResnetBlock moved to
unet/blocks.py). Remove TimestepEmbedder from celldiff/modules (moved
to unet/blocks.py). Remove einops from celldiff optional deps and test
importorskips.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: add unified parametrized tests for all 3D U-Net variants
Shared assertions for num_blocks, downsamples_z, forward pass shape
preservation, and UNet3DBase lineage across Unet3d, UNetViT3D, and
CELLDiffNet.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use consistent Tensor type annotation in CellDiff wrappers
Replace mixed torch.Tensor / Tensor usage with Tensor throughout
forward signatures and docstrings.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Dynaclr-dino (#387)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Alexandr Kalinin <alxndrkalinin@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Eduardo Hirata-Miyasaki <edhiratam@gmail.com>
* fix: remove duplicate fields from auto-merged test_database.py
Auto-merge kept both branches' additions of channel_names, marker,
and tracks_path. Remove the duplicates.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: cache loss function fg_mask compatibility check at init time
Move inspect.signature() call from _compute_loss() (hot path, every
batch) to __init__() (one-time). Stores result as _loss_accepts_fg_mask
boolean flag.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: skip fg_mask in predict stage and handle --config= form in CLI
P1: _setup_predict now strips fg_mask_key from dataset settings so
prediction works on datasets without precomputed masks.
P2: _maybe_compose_config now handles --config=path.yml and -c=path.yml
in addition to the space-separated form.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: handle base:null in config composition and constant-channel Otsu
compose.py: normalize base:null to empty list instead of crashing
with TypeError on iteration.
meta_utils.py: catch ValueError from threshold_otsu on constant-value
channels (e.g. all-zero) and default to 0.0.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: add ForegroundMaskSupport collaborator for fg_mask logic
Extract _SPATIAL_TRANSFORMS tuple and ForegroundMaskSupport class into
a new foreground_masks.py module. This collaborator will encapsulate all
fg_mask/Spotlight logic (validate, read, inject, extract, patch transforms)
so SlidingWindowDataset and HCSDataModule can delegate to it instead of
scattering mask conditionals across 21 touch points.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: extract SlidingWindowDataset and MaskTestDataset into sliding_window.py
Move both dataset classes from hcs.py to a new sliding_window.py module.
Code is moved verbatim with inline fg_mask logic preserved — the collaborator
wiring happens in the next commit. Update import paths in __init__.py and
test_hcs.py.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: wire ForegroundMaskSupport collaborator into SlidingWindowDataset
SlidingWindowDataset now delegates all fg_mask logic to a
ForegroundMaskSupport collaborator: validate_and_store (per-position),
read_window (inside retry loop), inject_into_sample (before transform),
extract_from_sample (after transform). The fg_mask_key constructor param
is preserved for backward compat — it creates the collaborator internally.
HCSDataModule._inject_mask_keys becomes a 3-line delegate to
ForegroundMaskSupport.patch_spatial_transforms. _SPATIAL_TRANSFORMS and
its viscy_transforms imports are removed from hcs.py (now live in
foreground_masks.py).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: centralize mask temp-key naming, remove extract_from_sample
Add ForegroundMaskSupport.mask_temp_keys() as single source of truth for
the __fg_mask_{ch} naming convention. Use it in __init__, _fit_transform,
and _final_crop instead of inlining the f-string pattern in 4 places.
Simplify inject_into_sample to use precomputed _mask_keys instead of
recomputing f-strings per call. Remove extract_from_sample and its
callback indirection — the caller now stacks directly via _stack_channels.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: ignore encoder_only in FcmaeUNet.save_hyperparameters
load_from_checkpoint on an encoder-only fine-tuned checkpoint would crash
with ValueError because encoder_only=True was saved but ckpt_path was not.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: generalize mask channel indexing for target-only mask arrays
ForegroundMaskSupport now auto-detects whether the mask array uses the
full image channel layout or a compact target-only layout. On the first
position, validate_and_store compares mask vs image channel counts and
sets _mask_ch_idx accordingly. read_window uses internal indices instead
of the caller-provided target_ch_idx.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use context managers for open_ome_zarr in HCSDataModule
_setup_fit, _setup_test, and _positions_maybe_single opened zarr stores
without `with` statements, leaking file handles. Zarr v2 DirectoryStore
positions survive store closure so this is safe.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add map_location="cpu" to VSUNet checkpoint loading
Prevents device mismatch when a GPU-saved checkpoint is loaded during
CPU-based config parsing. Matches FcmaeUNet._load_encoder_weights which
already uses map_location="cpu".
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: reject non-divisible input sizes in ViTBottleneck3D
Floor division silently accepted odd spatial sizes (e.g. 514) that cause
encoder/decoder shape mismatches at concat time. Now validates that each
downsampled dimension is exactly divisible by 2^num_downsamples before
computing the latent size.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: store mask channel indices per position, not globally
_mask_ch_idx was cached from the first FOV and reused for all positions.
This breaks datasets that mix full-channel and target-only mask layouts
across positions. Now stores per-position indices in _mask_ch_indices
so each position's mask array is read with the correct channel mapping.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use constant value as Otsu threshold for uniform channels
A constant nonzero channel got otsu_threshold=0.0, causing
generate_fg_masks to mark everything as foreground and Spotlight
normalization to produce huge values. Using the constant value
itself means nothing is marked foreground, which is correct —
a uniform channel has no meaningful foreground structure.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: replace assert with ValueError in unpatchify
Asserts are stripped with python -O, turning shape mismatches into
cryptic reshape errors. An explicit ValueError with expected/actual
token counts fails deterministically in all runtime modes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: add config composition tests for load_composed_config
The compose module had zero test coverage. These 12 tests cover
_deep_merge (flat, nested, list-replace, immutability) and
load_composed_config (no base, base:null, single/multiple/nested
bases, base-as-string, circular detection, ordering precedence).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: guard freeze_encoder against non-FCMAE architectures
freeze_encoder=True accessed self.model.encoder, which only exists on
FullyConvolutionalMAE. Using it with UNet3DBase or UNeXt2 would crash
with AttributeError. Now raises a clear ValueError instead.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add FNet3D and VSCyto3D training configs for dynacell SEC61B benchmark (#399)
* feat: add MinMaxSampled normalization transform
Port MinMaxSampled from the cell_diff_vs_viscy repo for percentile-
based min-max normalization to [-1, 1]. Supports p1_p99, p5_p95,
and min_max data ranges. Also extends LevelNormStats TypedDict in
both viscy-data and viscy-transforms with percentile fields (p1, p5,
p95, p99, min, max) and changes to total=False since not all zarr
stores have all stats.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use self.keys[0] in Batched* transforms for gpu_augmentations
Four Batched* dict transforms used next(iter(sample.keys())) or
next(iter(sample.values())) to get a reference tensor for
randomization. This fails when used in gpu_augmentations because
the batch dict has non-tensor keys like 'index' before the image
keys. Use self.keys[0] to always reference the first declared
transform key instead.
Also simplify DTypeLike -> type annotation in _noise.py to fix
jsonargparse introspection failure with numpy's complex DTypeLike
union type.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: log VSUNet hyperparameters to WandB Config tab
Add self.save_hyperparameters(ignore=["loss_function"]) in
VSUNet.__init__ so architecture, model_config, lr, and schedule
are logged to WandB's Config section. loss_function is excluded
because Lightning already saves nn.Module state in checkpoints.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add FNet3D and VSCyto3D training configs for SEC61B benchmark
Composable config recipes and leaf training configs for FNet3D and
VSCyto3D (UNeXt2) on AICS iPSC SEC61B (ER) dataset, targeting
architecture comparison with CellDiff UNetViT3D.
New recipes: fit_1gpu trainer, hcs_sec61b_3d data (MinMax p1_p99
normalization + GPU augmentations), fnet3d_z8 and unext2_3d_z8
model presets. Leaf configs compose these with model-appropriate
hyperparameters (FNet3D: lr=1e-3, batch=32; VSCyto3D: lr=2e-4,
batch=16). SLURM scripts target 1x H200.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: extract shared _match_image and precompute data_range keys
Deduplicate _match_image from NormalizeSampled and MinMaxSampled
into a module-level function. Precompute data_range -> (low_key,
high_key) mapping in MinMaxSampled.__init__ to avoid per-sample
string dispatch. Also fix DTypeLike docstrings in _noise.py.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add to_numpy helper for mixed-precision tensor conversion
NumPy does not support bfloat16, so bf16 tensors from AMP/autocast
crash on .numpy(). Add a shared to_numpy() helper in viscy_utils
that casts floating tensors to float32 before conversion. Integer
and boolean tensors preserve their dtype.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: replace .numpy() with to_numpy() at all external boundaries
Mixed-precision training produces bf16 tensors that crash on
.numpy() when passed to NumPy, CellPose, sklearn, wandb, zarr,
or AnnData. Replace all .detach().cpu().numpy() patterns with the
new to_numpy() helper across logging, callbacks, evaluation, and
application engines.
Also removes redundant .cpu() calls in embedding_writer and
apply_mlp_embedder since to_numpy() handles device transfer.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: narrow to_numpy to only cast bfloat16, preserve fp64
The blanket float→fp32 cast silently discarded float64 precision
in evaluation code like pairwise_distance_matrix, which explicitly
uses .double() for numerical accuracy. Only bfloat16 is unsupported
by NumPy; fp16/fp32/fp64 all have native equivalents.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: verify NormalizeSampled output values, not just shape
The test only asserted shape and metadata presence, missing actual
normalization correctness. Now computes (x - mean) / (std + eps)
on known inputs and asserts the result matches.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: restore per-batch CPU offload in MLP embedder
Removing .cpu() from the accumulation loop kept all encoded batches
on GPU until final concatenation, causing memory to grow with
dataset size. Restore immediate CPU offload so GPU memory stays
flat during large embedding exports.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: create slurm_log directory before job starts
SLURM opens --output/--error files before the script body runs.
Without the directory, jobs fail immediately on a clean checkout.
Create with mode 775 for group-write access on shared HPC storage.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: avoid GPU concatenation in embedding writer
Convert each prediction to numpy individually and concatenate on
CPU with np.concatenate instead of torch.cat on GPU. Prevents
transient GPU memory spikes during large prediction runs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Alexandr Kalinin <alxndrkalinin@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add public Dynacell benchmark app and Stage 2 training fixes (#397)
* feat: add public Dynacell benchmark application (Stage 2)
Create applications/dynacell/ as a thin supervised virtual staining
benchmark app consuming UNetViT3D and FNet3D from viscy-models.
Engine: DynacellUNet(LightningModule) with model-aware
example_input_array, fg_mask/Spotlight support, and explicit
NotImplementedError for predict_step (Stage 3 scope).
Configs: base-composition recipes for UNetViT3D and FNet3D fit,
including data, trainer, and Spotlight mode overlays.
Tests: 17 tests covering init, forward, spatial rejection,
fast_dev_run (synthetic + real OME-Zarr), Spotlight+fg_mask,
and config class_path resolution.
Workspace: remove dynacell from uv exclude, add to sources.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add else-branch for invalid schedule and use torch.stack
configure_optimizers silently fell through for unrecognized schedule
strings, causing a NameError at training time. Also replace
torch.tensor(losses) with torch.stack(losses) since elements are
already tensors.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use uv run in usage examples and clarify unsupported subcommands
CLAUDE.md requires uv run for all commands. Also distinguish predict
(explicit NotImplementedError) from test (no test_step override,
Lightning default fails on batch dict) in README.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: rename test config dicts to VIT_TEST_CONFIG/FNET_TEST_CONFIG
Clearer naming distinguishes test-size configs from production defaults.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: correct test subcommand error description in README
Lightning raises MisconfigurationException when test_step is missing,
not a batch-dict failure from the default implementation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: simplify training_step and remove logging noise
- Drop multi-batch loop in training_step: Dynacell uses a single
HCSDataModule with no CombinedDataModule, so the loop always ran
once and was speculative abstraction per CLAUDE.md
- Remove .to(self.device) on loss tensors: loss is already on the
correct device; the no-op calls are misleading
- Remove unused _logger module-level variable
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add return type annotation and fix logger comment in engine
- training_step was missing -> Tensor return annotation
- example_input_array comment said TensorBoard specifically;
W&B and other loggers also consume it
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: save hyperparameters, weight validation loss, fix scheduler, guard log_samples
- Add save_hyperparameters(ignore=[loss_function, ckpt_path]) so
load_from_checkpoint works and hparams are logged to experiment trackers
- Weight loss/validate by batch size per dataloader and across dataloaders;
unweighted mean biases toward smaller tail batches when val set is not
divisible by batch_size
- Fix WarmupCosineSchedule t_total: use estimated_stepping_batches instead
of max_epochs — the scheduler is step-based, so passing epoch count caused
LR collapse after ~1 epoch (200 steps vs 50k steps for fnet3d)
- Guard _log_samples against empty input: log_image_grid crashes via
np.concatenate([]) when log_batches_per_epoch=0
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: fix GPU device mismatch and WarmupCosine step interval
sizes_t was created on CPU while losses are on the model device in
GPU/DDP training, causing a device-mismatch crash at the end of the
first validation epoch. Fix by passing device=losses[0].device.
WarmupCosineSchedule is parameterized with estimated_stepping_batches
(a step count) but was returned as a bare scheduler, which Lightning
steps once per epoch by default. Return it as a config dict with
interval="step" so the LR decays at the correct granularity.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: document data_path requirement in README usage example
The configs ship with data_path unset (null). Without a note,
users following the README command hit a jsonargparse error before
training starts. Show the CLI override pattern explicitly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Alexandr Kalinin <alxndrkalinin@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: correct 3D positional embedding token ordering and add validation
The meshgrid used default xy indexing which produces depth-varies-
fastest ordering, but PatchEmbed3D flattens (B,C,D,H,W) in C-order
where depth varies slowest. Switch to ij indexing so depth tokens
are contiguous. Also replace assert statements with ValueError in
positional embedding functions, and add spatial divisibility
validation to UNet3DBase.forward.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: cache _divisor in UNet3DBase, remove redundant Unet3d.forward
UNet3DBase.forward recomputed 2**num_blocks on every call. Cache it
at init as _divisor. Since the base class now validates spatial
divisibility, Unet3d no longer needs its own forward override or
separate _divisor — remove both to eliminate the duplication.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: cache channel lists and use stdlib randint in SlidingWindowDataset
__getitem__ rebuilt combined channel name/index lists via .copy() +
.extend() on every call. Cache them at init since they never change.
Also replace torch.randint (allocates a tensor) with random.randint
for the nonzero-retry index — ~10x faster for scalar int sampling.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use torch RNG for retry index to respect global seed
random.randint uses a separate RNG from torch.manual_seed, making
retry sampling non-reproducible when seed_everything is set. Revert
to torch.randint but use a 0-dim tensor to minimize allocation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add deterministic predict support to Dynacell (Stage 2.5)
Replace predict_step NotImplementedError with DivisiblePad-based
tiled inference, matching Cytoland's proven pattern. FNet3D pads
all spatial dims; UNetViT3D tiles must match input_spatial_size.
Adds predict configs, integration tests for both architectures,
and HCSPredictionWriter pipeline verification.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add git workflow rules to CLAUDE.md
Codify no-amend, no-force-push, atomic commits, and explicit
staging as project-level instructions for Claude Code sessions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: correct key-sharing and axis bugs in batched GPU transforms (#400)
* fix: correct key-sharing and axis bugs in BatchedRandAffined and BatchedRand3DElasticd
BatchedRandAffined generated independent random params per key, causing
source/target misalignment during training. Also, scale_range was
incorrectly axis-inverted and Kornia sampled per-axis independently.
BatchedRand3DElasticd had the same per-key bug plus a displacement axis
swap (D↔W) when mapping to grid_sample, and a double probability gate.
Fixes:
- Generate affine params once via forward_parameters(), reuse for all keys
- Support flat (min,max) and per-axis ZYX scale_range with isotropic option
- Generate elastic displacement field once, reuse for all keys
- Correct displacement D,H,W → grid X,Y,Z axis mapping
- Remove double probability gate and unused spatial_size in elastic
- Add 12 tests covering key consistency, axis ordering, and scale behavior
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: update SEC61B training configs for H200 and external artifact storage
- Increase batch_size to 64 (FNet3D and VSCyto3D) and max_epochs to 100
- Move artifacts to /hpc/projects/comp.micro/virtual_staining/models/
- Set explicit checkpoint dirpath outside the repo
- Fix scale_range from [0.5, 1.5] to per-axis [[0.8, 1.2], [0.7, 1.3],
[0.7, 1.3]] matching original VSCyto3D augmentation ranges
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address code review — tests, imports, docs, and allow_missing_keys
- Strengthen elastic axis ordering test to actually catch D↔W swap
- Strengthen isotropic scale test to verify params directly
- Import from public viscy_transforms API in tests, not private modules
- Replace unused loop variable `b` with `_` in elastic field generation
- Document batch-level probability semantics in elastic docstring
- Raise ValueError when isotropic_scale=True combined with per-axis ranges
- Guard against KeyError when allow_missing_keys=True and no keys present
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: validate scale_range length to catch malformed 3-scalar inputs
_parse_scale_range now raises ValueError for inputs like [0.2, 0.3, 0.3]
(3 bare floats) instead of passing them to Kornia which crashes with an
unhelpful shape error.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use 2/(size-1) for align_corners=True displacement normalization
With align_corners=True, grid_sample maps [-1, 1] to pixel [0, N-1],
so 1 voxel displacement = 2/(N-1). The old 2/N formula under-scaled
displacements. Pre-existing bug, fixed while in the area.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Alexandr Kalinin <alxndrkalinin@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add shear support with Z-proportional scaling and oversized crop pipeline
- Fix shear_range: remove incorrect _radians_to_degrees and _invert_per_axis
(shear is in degrees, not radians; facets are not ZYX-ordered axes)
- Add _parse_shear_range: supports (min,max) isotropic, 3-value MONAI
shorthand [s_zy, s_zx, s_yz], and 6 per-facet (min,max) pairs
- Add scale_z_shear option (default True): scales Z-related shear facets
by z_depth/yx_size to prevent destructive shear on thin Z volumes
- Update config: yx_patch_size 512, center crop to 256 after affine,
re-enable shear [0.0, 3.0, 3.0] matching original VSCyto3D
- Use mul_ in-place for shear scaling, remove unnecessary .contiguous()
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: match original VSCyto3D normalization and noise params
Switch normalization from MinMax p1_p99 to mean/std (source) and
median/iqr (target) to match the original VSCyto3D pipeline.
Reduce Gaussian noise std from 5.0 to 1.0 to match original.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: standardize W&B run naming and grouping
* feat: add BatchedRandWeightedCropd GPU-batched weighted spatial crop
Samples crop positions proportional to a spatial importance map using
avg_pool2d for per-window weights and torch.multinomial for sampling.
Replaces the fixed center-crop approach so training can focus on
signal-rich regions within each FOV.
Pipeline: CPU 512x512 → weighted crop 384x384 → affine → center 256x256.
Registered in _SPATIAL_TRANSFORMS for fg_mask co-alignment.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add UNeXt2 architecture support to DynacellUNet
Register UNeXt2 in the _ARCHITECTURE dict so Dynacell can own
SEC61B benchmarks that use the UNeXt2 (VSCyto3D) backbone.
Includes unit tests (init, forward, predict_step), a fast_dev_run
integration test with YX=64 fixtures, and a predict-to-OME-Zarr
integration test.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add SEC61B benchmark configs and launch scripts to dynacell
Dynacell becomes the canonical launch owner for SEC61B benchmarks
(FNet3D + UNeXt2). Includes data/model/trainer recipes, leaf
configs with MixedLoss and WarmupCosine, and H200 SLURM scripts.
Cytoland copies will be marked as transitional legacy in a
follow-up commit.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: mark cytoland SEC61B configs as transitional legacy paths
Dynacell is now the canonical launch owner for SEC61B and FNet3D
benchmarks. Add legacy markers to cytoland configs pointing to
their dynacell replacements, and note the change in the cytoland
README. No files deleted — cleanup deferred until dynacell runs
are validated on H200.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use valid per-axis scale_range format in cached_pretrain config
After PR #400 tightened _parse_scale_range validation, bare 3-float
sequences like [0.2, 0.3, 0.3] raise ValueError. Convert to explicit
per-axis (min, max) tuples matching the intended scale ranges.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: add competing-locations multichannel weight test
The original multichannel test only verified sum-over-C at one spatial
location. Add a test with deltas at opposite corners (each in exactly
one crop window) to verify that higher total weight across channels
biases sampling toward the stronger location.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add paper-baseline FNet SEC61B config
* feat: finalize SEC61B FNet configs
* refactor: move spatial cropping from CPU _final_crop to GPU transforms
Remove the hardcoded CenterSpatialCropd from HCSDataModule._final_crop()
which coupled the data module to cropping logic and eliminated spatial
diversity during training (always center-cropped). Cropping now happens
on GPU in on_after_batch_transfer:
- Training with gpu_augmentations: user-specified transforms handle crop
- Training without: default BatchedRandSpatialCropd to yx_patch_size
- Validation: deterministic BatchedCenterSpatialCropd to yx_patch_size
- Test/predict: pass through unchanged
Also moves target_2d Z slicing from on_before_batch_transfer to after
the GPU crop in on_after_batch_transfer to avoid Z dimension mismatch.
Adds BatchedRandSpatialCropd to the FNet paper config's gpu_augmentations
for random spatial sampling across the full FOV.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: extract _pad_forward_crop and fix predict unpadding
Replace _predict_pad.inverse() with explicit center-crop to the
pre-pad shape for more reliable unpadding. Extract the repeated
pad→forward→crop pattern into _pad_forward_crop helper.
Also add tuple-merging support in FcmaeUNet._merge_batches so
heterogeneous batch entries (e.g. index tuples with mixed Tensor
and list elements) are correctly concatenated across sub-batches
instead of silently dropping all but the first.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: decouple HCSDataModule from viscy_transforms
Remove BatchedCenterSpatialCropd/BatchedRandSpatialCropd import and
fallback crop construction from HCSDataModule. The data module should
not depend on the transforms package — cropping is the user's
responsibility via gpu_augmentations config.
Replace fallback crops with a shape validation check that raises a
clear error when source spatial dims don't match yx_patch_size during
training/validation. Test/predict pass through unchanged (full FOV).
Also delete the no-op on_before_batch_transfer and gate the shape
check on training/validating only.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use context manager for open_ome_zarr in CachedOmeZarrDataModule
Same pattern fixed in HCSDataModule by commit 462f653 — zarr store
opened without `with` statement leaks file handles if an exception
occurs before close.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: restrict spatial shape check to training only
Validation runs at full FOV without gpu_augmentations, so the shape
check against yx_patch_size would always fail. Per-pixel MSE is
scale-invariant so full-FOV validation is valid. Training still
validates that gpu_augmentations produce the expected patch size.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: add per-worker FOV cache to SlidingWindowDataset
Consecutive sliding window samples often come from the same FOV,
causing redundant zarr chunk decompression across calls. Cache the
decompressed FOV array per worker using lru_cache so subsequent
Z windows from the same FOV are pure numpy slices (no I/O).
Each DataLoader worker fork gets its own cache. Default 5 FOVs
per worker (~1 GB at 200 MB/FOV for 2-channel SEC61B data).
Configurable via fov_cache_maxsize parameter (0 to disable).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: cache per-timepoint in FOV cache, not entire T dimension
The FOV cache was reading all timepoints into memory but only using
one per sample. For multi-timepoint datasets this wastes T× memory.
Key the cache on (arr_idx, t, ch_idx) and read a single timepoint
per entry: (1, C, Z, Y, X) instead of (T, C, Z, Y, X).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: replace predict_pad.inverse with center-crop in dynacell engine
Same fix applied to cytoland in e4171aa — DivisiblePad.inverse relies
on MONAI metadata tracking which may not be active, producing wrong
output shapes. Use explicit _center_crop_to_shape instead.
Also update dynacell test yx_patch_size from (16,16) to (32,32) to
match the test fixture FOV size, since _final_crop no longer exists
to bridge the mismatch.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: fix stale docstrings in cytoland engine
on_predict_start: remove incorrect claim about DivisiblePad.inverse
(replaced by _center_crop_to_shape in e4171aa).
_compute_loss: change "configuration time" to "training time" since
the TypeError is raised inside _compute_loss, not at init.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: move inline imports to top of file in test_hcs.py
CLAUDE.md requires top-level imports unless there is a strong reason
for inline. These monai/viscy_transforms imports have no circular
dependency or conditional import justification.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: decouple viscy-data from viscy-transforms via is_spatial
Each transform now self-declares `is_spatial = True` (spatial) or
`is_spatial = False` (intensity). foreground_masks.py checks this
attribute instead of importing concrete classes from viscy-transforms,
eliminating the undeclared runtime dependency.
An MRO-based fallback detects raw MONAI spatial transforms (from
monai.transforms.spatial or monai.transforms.croppad) without imports.
Also fixes a pre-existing gap: BatchedRand3DElasticd, BatchedZoomd,
and BatchedRandZStackShiftd are now correctly identified as spatial.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add flow matching celldiff (#402)
* feat: port flow-matching transport module to viscy-models
Port ODE/SDE solvers, coupling plans (ICPlan, VPCPlan, GVPCPlan), and
Transport/Sampler classes from the CellDiff fork into
viscy_models.celldiff.modules.transport. Add torchdiffeq to the celldiff
optional dependency group. Include 7 unit tests.
Cleanup vs fork: th→torch, type hints, numpy docstrings, dead code
removed (EasyDict, log_state), bare except fixed, class names
capitalized (ODESolver/SDESolver), eval→is_eval parameter rename,
eager diffusion evaluation replaced with lazy if/elif dispatch,
CPU-to-GPU allocations fixed (randn_like/new_ones), SDE solver
returns only final state instead of all intermediates, silent .get()
fallbacks replaced with explicit ValueError on invalid config strings.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add DynacellFlowMatching LightningModule and CellDiff configs
Add CELLDiff3DVS flow-matching wrapper (celldiff_wrapper.py) and
DynacellFlowMatching LightningModule to the dynacell application.
Includes fit/predict configs, flow-matching trainer recipe (no
validation loss monitor), and celldiff_fm model recipe.
Key fixes vs fork: save_hyperparameters added, WarmupCosine uses
estimated_stepping_batches with step interval, logger-agnostic image
logging via log_image_grid, validation_step captures batch for ODE
generation logging, configure_optimizers extracted to shared helper.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: add flow-matching tests and update dynacell README
Add 4 engine smoke tests (instantiation, forward loss, generate shape,
predict pad/crop) and 3 training integration tests (WarmupCosine,
Constant schedule, predict-to-zarr) for DynacellFlowMatching.
Config class_path resolution auto-discovers new celldiff/*.yml configs.
Update README with flow-matching usage and architecture docs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: validate path_type in create_transport
Raises ValueError with a descriptive message for invalid path_type
instead of a raw KeyError, matching the validation already done for
the prediction and loss_weight parameters.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use Callable type hint instead of lowercase callable
Replace built-in callable with collections.abc.Callable in type
annotations throughout integrators.py and transport.py. The lowercase
callable is the built-in function, not a valid type annotation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: slice validation batch at capture time in DynacellFlowMatching
Only clone log_samples_per_batch samples instead of the full batch
when capturing validation data for epoch-end ODE generation logging.
Avoids holding unused GPU memory for the entire validation epoch.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: remove dead test config constants from conftest
CELLDIFF_TEST_NET_CONFIG and CELLDIFF_TEST_TRANSPORT_CONFIG in
conftest.py were never used by any fixture. The test files define
their own copies because --import-mode=importlib prevents importing
from conftest directly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: restrict spatial shape check to training only
Validation runs at full FOV without gpu_augmentations, so the shape
check against yx_patch_size would always fail. Per-pixel MSE is
scale-invariant so full-FOV validation is valid. Training still
validates that gpu_augmentations produce the expected patch size.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: add per-worker FOV cache to SlidingWindowDataset
Consecutive sliding window samples often come from the same FOV,
causing redundant zarr chunk decompression across calls. Cache the
decompressed FOV array per worker using lru_cache so subsequent
Z windows from the same FOV are pure numpy slices (no I/O).
Each DataLoader worker fork gets its own cache. Default 5 FOVs
per worker (~1 GB at 200 MB/FOV for 2-channel SEC61B data).
Configurable via fov_cache_maxsize parameter (0 to disable).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: cache per-timepoint in FOV cache, not entire T dimension
The FOV cache was reading all timepoints into memory but only using
one per sample. For multi-timepoint datasets this wastes T× memory.
Key the cache on (arr_idx, t, ch_idx) and read a single timepoint
per entry: (1, C, Z, Y, X) instead of (T, C, Z, Y, X).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: replace predict_pad.inverse with center-crop in dynacell engine
Same fix applied to cytoland in e4171aa — DivisiblePad.inverse relies
on MONAI metadata tracking which may not be active, producing wrong
output shapes. Use explicit _center_crop_to_shape instead.
Also update dynacell test yx_patch_size from (16,16) to (32,32) to
match the test fixture FOV size, since _final_crop no longer exists
to bridge the mismatch.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: fix stale docstrings in cytoland engine
on_predict_start: remove incorrect claim about DivisiblePad.inverse
(replaced by _center_crop_to_shape in e4171aa).
_compute_loss: change "configuration time" to "training time" since
the TypeError is raised inside _compute_loss, not at init.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: move inline imports to top of file in test_hcs.py
CLAUDE.md requires top-level imports unless there is a strong reason
for inline. These monai/viscy_transforms imports have no circular
dependency or conditional import justification.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: decouple viscy-data from viscy-transforms via is_spatial
Each transform now self-declares `is_spatial = True` (spatial) or
`is_spatial = False` (intensity). foreground_masks.py checks this
attribute instead of importing concrete classes from viscy-transforms,
eliminating the undeclared runtime dependency.
An MRO-based fallback detects raw MONAI spatial transforms (from
monai.transforms.spatial or monai.transforms.croppad) without imports.
Also fixes a pre-existing gap: BatchedRand3DElasticd, BatchedZoomd,
and BatchedRandZStackShiftd are now correctly identified as spatial.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: move SDE solver time grid to input device
SDESolver created self.t and self.dt on CPU in __init__, causing
device mismatch when sample() operates on GPU tensors. Move to
input device at sample() start and pass dt explicitly to step
functions, matching the pattern already used by ODESolver.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address numerical and correctness bugs in …
edyoshikun
commented
Apr 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.