Skip to content

Applications: DynaCLR refactor#370

Closed
edyoshikun wants to merge 246 commits intomodular-viscy-stagingfrom
app-dynaclr
Closed

Applications: DynaCLR refactor#370
edyoshikun wants to merge 246 commits intomodular-viscy-stagingfrom
app-dynaclr

Conversation

@edyoshikun
Copy link
Copy Markdown
Member

No description provided.

edyoshikun and others added 30 commits February 12, 2026 13:53
- Add pyproject.toml with hatchling build, torch/timm/monai/numpy deps
- Create src layout with _components, unet, contrastive, vae subpackages
- Add PEP 561 py.typed marker
- Add test scaffolding with device fixture

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add viscy-models to root dependencies and uv sources
- Update lockfile with timm and viscy-models dependencies
- Verified: uv sync, import, and pytest collection all succeed

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add 06-01-SUMMARY.md with execution results
- Update STATE.md with position, metrics, and decisions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- stems.py: UNeXt2Stem, StemDepthtoChannels from v0.3.3 unext2.py
- heads.py: PixelToVoxelHead, UnsqueezeHead, PixelToVoxelShuffleHead
- blocks.py: icnr_init, _get_convnext_stage, UNeXt2UpStage, UNeXt2Decoder
- __init__.py: re-exports all 8 public components
- Zero imports from unet/, vae/, or contrastive/
- All attribute names preserved for state dict compatibility
- Copy ConvBlock2D from v0.3.3 source to snake_case file
- Copy ConvBlock3D from v0.3.3 source to snake_case file
- Preserve register_modules/add_module pattern for state dict key compatibility
- Update _layers/__init__.py with public re-exports
- Fix docstring formatting for ruff D-series compliance
- 6 tests for ConvBlock2D: forward pass, state dict keys, residual, filter steps, instance norm
- 4 tests for ConvBlock3D: forward pass, state dict keys, dropout registration, layer order
- All 10 tests verify shape, naming patterns, and module registration
- test_stems.py: UNeXt2Stem shape, StemDepthtoChannels shape + mismatch error
- test_heads.py: PixelToVoxelHead, UnsqueezeHead, PixelToVoxelShuffleHead shapes
- test_blocks.py: icnr_init, _get_convnext_stage, UNeXt2UpStage, UNeXt2Decoder
- 10 tests total, all passing on CPU
- SUMMARY.md with migration details and self-check
- STATE.md updated: phase 6 plan 3/3, decisions recorded
- SUMMARY.md documents 8 extracted components with 10 tests
- STATE.md updated with decisions from 06-02 execution
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Copy UNeXt2 class (~70 lines) from monolithic unext2.py
- Update imports to use viscy_models._components (stems, heads, blocks)
- Preserve all attribute names for state dict compatibility
- Export UNeXt2 from viscy_models.unet public API

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add tests: default, small backbone, multichannel, diff stack depths, deconv, stem validation
- Fix deconv decoder tuple bug in UNeXt2UpStage (trailing comma created tuple not module)
- Mark deconv test xfail: original code has channel mismatch in deconv forward path
- All 26 tests pass (25 passed, 1 xfailed) with no regressions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add 07-01-SUMMARY.md with execution results and deviation documentation
- Update STATE.md: phase 7, plan 1/2 complete, new decisions logged

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Copy FCMAE and all helper classes/functions to unet/fcmae.py
- Replace old viscy imports with viscy_models._components imports
- Remove duplicated PixelToVoxelShuffleHead (import from _components.heads)
- Fix mutable list defaults to tuples (encoder_blocks, dims)
- Export both UNeXt2 and FullyConvolutionalMAE from unet/__init__.py
- Copy all 11 test functions with zero logic changes
- Update imports from viscy.unet.networks.fcmae to viscy_models.unet.fcmae
- Import PixelToVoxelShuffleHead from viscy_models._components.heads
- All 37 tests pass across full suite (no regressions)
- Add 07-02-SUMMARY.md with execution results
- Update STATE.md: Phase 7 complete, 12 plans total, decisions logged
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add BetaVae25D with VaeUpStage, VaeEncoder, VaeDecoder helpers
- Add BetaVaeMonai wrapping MONAI VarAutoEncoder
- Fix VaeDecoder mutable list defaults to tuples (COMPAT-02)
- Change VaeEncoder pretrained default to False
- Preserve all attribute names for state dict compatibility

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…models

- Add ContrastiveEncoder with convnext/resnet50 backbone support via timm
- Add ResNet3dEncoder with MONAI ResNetFeatures backend
- Fix ResNet50 bug: use encoder.num_features instead of encoder.head.fc.in_features
- Add pretrained parameter (default False) for pure nn.Module semantics
- Preserve state dict attribute names (stem, encoder, projection)
- Share projection_mlp utility between both encoder classes
- 3 tests for ContrastiveEncoder: convnext_tiny, resnet50, custom stem
- 2 tests for ResNet3dEncoder: resnet18, resnet10
- Verify embedding and projection output shapes
- ResNet50 test uses in_stack_depth=10 for valid stem channel alignment
  - Add `collection.py` for ML training collection definitions
  - Rename `hours_post_infection` → `hours_post_perturbation` in `_typing.py`
  - Update `cell_index.py` to use `collection_path` and renamed field
  - Add corresponding tests for new modules and updated cell index
…calls deg2rad internally)

  - Fix docstrings to reflect ZYX input order, facet naming, and unit conventions
  - Remove stale timepoint_statistics lookup in _normalize.py
…/norm maps

  - MultiExperimentIndex with parallel FOV loading and lineage-aware anchors
  - MultiExperimentDataModule with stratify_by, num_workers_index, FOV-level split
  - Dataset updates for new pipeline; remove ExperimentConfig from __init__
  - Consolidate shared test helpers and constants into conftest.py
  - Update build-cell-index, train-multi-experiment, troubleshooting recipes
  - Add sampling-strategies.md documenting FlexibleBatchSampler axes
  - Update README table
  - Add CLAUDE.md with data pipeline design principles
@edyoshikun edyoshikun mentioned this pull request Mar 17, 2026
edyoshikun and others added 7 commits March 17, 2026 11:35
…, logging

- Remove duplicate viscy_models/_components/ (all imports already use components/)
- Replace assert with raise ValueError in conv_block_2d/3d.py
- Fix numpy docstrings in viscy_data/_utils.py (_ensure_channel_list, _collate_samples)
- Narrow broad except Exception to (OSError, ValueError) in hcs.py
- Use direct row["parent_track_id"] access in cell_index.py (column guarded above)
- Fix frame interval mode().iloc[0] IndexError risk in pseudotime/metrics.py
- Remove backwards-compat re-exports from qc/config.py
- Add logging + warn on silent AUROC ValueError in linear_classifier.py
- Add exc_info=True to PHATE/PCA warning in embedding_writer.py
- Remove resolved TODO in feature.py
- Add _logger and replace print() with logging in dynaclr/utils.py and qc/qc_metrics.py
- Warn when logger_base path does not exist in dynaclr/utils.py
- Move inline imports to top in test_sampler.py
- Add type hints to stems.py compute_stem_channels()
- Remove redundant num_channels reassignment in beta_vae_25d.py
- Fix prek reference in CLAUDE.md (was pre-commit)
- Fix input_channel comment/value: organelle→marker in example config

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- Remove packages/viscy-data/tests/test_smoke.py: not a real integration
  test (magic __all__ count, source string matching, no functional coverage)
- Update TestLinearClassifierTrainConfig: embedding_model+wandb_project
  replaced by embedding_model_name+embedding_model_version in schema
- Fix test_parquet_lineage_preserved: add check_index_type=False to handle
  object vs StringDtype difference between legacy and parquet paths

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…lures

- Rename "organelle" → "marker" in VALID_CHANNELS, pseudotime plotting,
  and test fixtures to match the unified channel type naming
- Fix tests: set pd.options.future.infer_string=False in conftest to prevent
  pandas 2.x ArrowStringArray from breaking anndata zarr writer
- Fix test_loss: torch.Generator(device=device) for CUDA compatibility
- Update TestLinearClassifierInferenceConfigOrganelle to new schema
  (embedding_model_name/version + models list)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- Update example_cell_index.yaml with real dataset paths and rename
  hours_post_infection → hours_post_perturbation
- Add collection YAMLs: A549_ZIKV_multiorganelle, A549_bag_of_channels, example
- Add training fit configs for A549_ZIKV_multiorganelle and A549_bag_of_channels
- Add smoothness evaluation SLURM scripts

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- generate_classifier_inference.py: generate inference configs + SLURM
  script for a model predictions folder
- generate_train_config_from_folder.py: generate training configs from
  prediction folders, supports multi-dataset combine
- label_offset_sweep.py: sweep temporal label offsets for infection
  classifier to find optimal onset labeling
- infection_death_remodeling.py: correlate infection, death, and
  organelle remodeling event timings across tracks
- infection_onset_distribution.py: compute and plot infection onset
  distributions from classifier predictions

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…dency

Replace comparison against a pre-computed reference zarr (39170 cells,
now stale) with a self-contained determinism test: run inference twice
with the same seed and assert the outputs match within GPU tolerance.
This removes the brittle hardcoded cell count and the reference zarr
that needs to be regenerated whenever the data or pipeline changes.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@edyoshikun edyoshikun marked this pull request as ready for review March 17, 2026 23:04
…verwrite=True to EmbeddingWriter

- quickstart.py/ipynb: add overwrite=True to EmbeddingWriter to prevent
  FileExistsError on notebook re-run
- cross_validate_example.yaml: channels [phase, sensor, organelle] → [phase, sensor, marker]
- example_linear_classifier_inference.yaml: update W&B artifact names
  organelle_state-organelle-* → organelle_state-marker-*
- example_linear_classifier_train.yaml: update comment and example
  embedding paths to use marker instead of organelle

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@edyoshikun
Copy link
Copy Markdown
Member Author

we will merge this to #379 and then into #373

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants