feat(mosaic): LiveStitch placement + smart canvas management#40
Merged
Conversation
Switch SaveViTResult from the Fourier-shift stitcher to ptychoml.stitch.stitch_batch_livestitch_into (nearest-integer placement that returns the touched bbox), and harden the live mosaic: - MosaicWriterOp writes the full cumulative canvas every batch (latest-wins, capacity-1 + drop policy) instead of bbox-only patches — fixes the staircase artifact on unidirectional scans. _normalise_full now returns (fill_value, array) and _normalise_sub is removed. - Smart canvas centering: when the observed range is < 50% of commanded (the slow axis has barely moved when the canvas is first sized), infer the scan direction and centre on the commanded midpoint instead of the biased observed one. Extracted to holoptycho/mosaic_canvas.estimate_canvas_mid so it's unit-testable without the Tiled-bound vit_inference import. - Pending-frame buffer: ViT batches whose scan positions are still NaN (the position stream lagging the ViT stream) are buffered and re-stitched once positions arrive, instead of dropped (cap 500). - New knobs: patch_flip (D4 on each patch via ptychoml.apply_d4, default identity), mosaic_min_overlap (default 0.5), canvas_pad default 64 -> 0. SaveViTResult/MosaicWriterOp are lifted from PR #37 verbatim (the proven live-mosaic code), re-pointed at ptychoml.stitch (per H1). Two faithful pre-existing quirks kept as-is: the start() warm-up calls stitch_batch_nearest while the hot path is livestitch (harmless, both pure numpy), and fourier_pad is now an unused vestigial param. H3 of the #37 decomposition. Stitching correctness is covered by ptychoml; the new holoptycho centering logic is tested in tests/test_mosaic_canvas.py. Co-authored-by: Himanshu Goel <4122621+himanshugoel2797@users.noreply.github.com>
…rmalise_full ptychoml#11 added normalize_mosaic (the averaging companion to the stitch accumulators). Replace MosaicWriterOp's inline _normalise_full with it and bump the ptychoml pin. Verified bit-identical to the inline version across random (canvas, counts) inputs, so no behavior change. Co-authored-by: Himanshu Goel <4122621+himanshugoel2797@users.noreply.github.com>
The NaN-position race handling (buffer ViT frames whose scan positions haven't arrived yet, re-merge once they do) was the most complex untested piece of H3. Extract the partition step into a pure mosaic_canvas.partition_pending (behavior-identical to the prior inline double list-comprehension, and evaluates the finite-position check once per frame instead of twice) and cover it in tests/test_mosaic_canvas.py: all-ready, all-waiting-on-NaN, mixed + order preservation, out-of-range index, partial-NaN row, patch-travels-with-index, and empty input. Co-authored-by: Himanshu Goel <4122621+himanshugoel2797@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
H3 of the #37 decomposition. Switches the live ViT mosaic from Fourier-shift stitching to LiveStitch (nearest-integer + bbox) and adds the canvas-management fixes from #37.
SaveViTResult/MosaicWriterOpare lifted verbatim from #37 (the proven live-mosaic code), re-pointed atptychoml.stitchper H1.Changes (
vit_inference.py,ptycho_holo.py,mosaic_canvas.py)SaveViTResultusesptychoml.stitch.stitch_batch_livestitch_into(nearest-integer placement, returns the touched bbox) instead ofstitch_batch_into.MosaicWriterOpwrites the full cumulative canvas every batch (latest-wins, capacity-1 + drop policy), not just the bbox._normalise_fullreturns(fill_value, array);_normalise_subremoved.holoptycho/mosaic_canvas.estimate_canvas_midso it's unit-testable without the Tiled-boundvit_inferenceimport._pending_framesand re-stitched once positions arrive, instead of dropped (cap 500).patch_flip(D4 per patch viaptychoml.apply_d4, defaultidentity),mosaic_min_overlap(0.5),canvas_paddefault 64 → 0.Scope boundary
Only the
SaveViTResult/MosaicWriterOp(mosaic) changes. ThePtychoViTInferenceOpchanges from #37 — orientation autodetect (H5) and ONNX inner-crop / vit-only mode (H4) — are deliberately not here.Faithful-to-#37 quirks (kept, not changed)
start()warm-up callsstitch_batch_nearestwhile the hot path isstitch_batch_livestitch_into— harmless (both pure numpy, nothing to JIT).fourier_padis now an unused vestigial param.Flagging rather than "fixing" them, since #37 is the working reference.
Tests
tests/test_mosaic_canvas.pycoversestimate_canvas_mid(observed-midpoint when well-covered or commanded unknown; direction-inferred midpoint for under-scanned + / − / flat). Stitching itself is covered by ptychoml. TheSaveViTResult/MosaicWriterOpoperators run only in the live pipeline (per memory, not worth fighting the Tiled/Holoscan harness to exercise).Depends on
NSLS2/ptychoml#11 (merged) —
MosaicWriterOpnow importsptychoml.stitch.normalize_mosaicinstead of an inline_normalise_full(verified bit-identical).pixi.lockbumped accordingly.