feat(preprocess): delegate model branch to ptychoml + geometry/DC knobs#39
Merged
Merged
Conversation
Route ImagePreprocessorOp's model-input branch through ptychoml.preprocess_diffraction (normalize -> hot-pixel mask -> sqrt -> D4 -> fftshift) and the intensity tap through ptychoml.apply_d4, replacing the hardcoded antidiag-flip/rot90/fftshift/transpose/sqrt chain. New scan-JSON knobs (all with back-compatible defaults): - tap_orient (default 'antitranspose'): D4 on the saved intensity tap; reproduces the old anti-diagonal flip. - dp_orient (default 'rot90_cw'): D4 on the model branch; verified to reproduce the old orientation chain exactly (max diff ~1e-6). - fftshift_dp (default None): DC convention; None lets ptychoml auto-detect via detect_dc_at_corner (centers DC only when at the corners — matches the old unconditional shift for typical detector input). - vit_normalization / vit_scale (1e5 / 1e4): intensity normalization passed to preprocess_diffraction so model input matches the offline training scale. This is the one intended behavior change (amplitudes rescaled by sqrt(scale/normalization)); harmless global scale for the iterative DM engine, required for the ViT model. - hot_pixel_count_threshold (default None/disabled). Also: auto_center default flipped to off (the shift isn't part of ptychoml's pipeline and displaces the beam from the model's trained position); remove the now-unused dp_transpose knob; ImageBatchOp Eiger2 flip mirrors the column ROI (extracted to crop_flipped_roi, tested). H2 of the #37 decomposition. preprocess_diffraction/apply_d4 correctness is covered by ptychoml's tests; the new holoptycho geometry (crop_flipped_roi) is tested in tests/test_preprocess_geometry.py. Co-authored-by: Himanshu Goel <4122621+himanshugoel2797@users.noreply.github.com>
ptychoml renamed PtychoViTInference's 'data_is_shifted' parameter to 'fftshift' (None = auto-detect DC via detect_dc_at_corner). holoptycho main still called PtychoViTInference(data_is_shifted=...), which raises TypeError against the pinned ptychoml (the bump in #38 crossed the rename) — the ViT op would crash on its first batch. Smoke tests missed it: the vit_inference import dies earlier on TILED_BASE_URL, and CI envs skip holoscan entirely. Rename PtychoViTInferenceOp's param data_is_shifted -> fftshift and pass fftshift= to the session; drop data_is_shifted=True in ptycho_holo.py and let it default to None (auto-detect), matching PR #37's end-to-end DC handling that pairs with this PR's preprocess_diffraction(fftshift=fftshift_dp=None). Adds tests/test_vit_session_contract.py to pin the holoptycho<->ptychoml call contract so a future rename fails in CI instead of on the beamline. Co-authored-by: Himanshu Goel <4122621+himanshugoel2797@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
H2 of the #37 decomposition. The complete DC-convention / diffraction-preprocessing change: routes
ImagePreprocessorOp's model branch throughptychoml.preprocess_diffraction+ the tap throughptychoml.apply_d4, exposes geometry/normalization as config knobs, and aligns the ViT session with ptychoml's renamedfftshiftAPI. Mirrors #37's working end-to-end DC handling (auto-detect everywhere).Unlike H1, this is intentionally a behavior change (it adds the training-pipeline normalization). The orientation/fftshift defaults reproduce #37's proven configuration.
ptychoml renamed
PtychoViTInference'sdata_is_shiftedparam →fftshift. holoptycho main still callsPtychoViTInference(data_is_shifted=...), which raisesTypeErroragainst the pinned ptychoml (the bump in #38 crossed the rename) — the ViT op would crash on its first batch. CI/smoke missed it: thevit_inferenceimport dies earlier onTILED_BASE_URL, and the CI envs skip holoscan. This PR renamesPtychoViTInferenceOp's paramdata_is_shifted→fftshift(defaultNone= auto-detect) and dropsdata_is_shifted=Trueinptycho_holo.py, matching #37. A newtests/test_vit_session_contract.pypins the holoptycho↔ptychoml call contract so a future rename fails in CI, not on the beamline.Changes (
preprocess.py,vit_inference.py,ptycho_holo.py)New scan-JSON knobs, all with back-compatible defaults:
tap_orientantitransposedp_orientrot90_cwfftshift_dpNoneNone= ptychoml auto-detect viadetect_dc_at_corner(matches #37; pairs with the session's ownfftshift=None)vit_normalization/vit_scale1e5/1e4sqrt(scale/normalization))hot_pixel_count_thresholdNonehxn_to_vit.py)Also:
auto_centerdefault flipped off — the shift isn't part of ptychoml's pipeline and displaces the beam from the model's trained position.dp_transposeknob (folded intodp_orient).crop_flipped_roi.Scope boundary
Takes only the DC-convention/preprocessing subset of #37. The orientation auto-detect buffering +
swap_xy(H5), the vit-only-mode guards + ONNX inner-crop (H4), and livestitch/canvas (H3) are deliberately not here.fftshift=Noneend-to-end is #37's working config; the further "resolve-once" streaming hardening (if any) rides with the autodetect work.Verification
dp_orient='rot90_cw'+preprocess_diffractionchecked numerically vs the old inline chain on random input — identical orientation+fftshift, max abs diff1.9e-6; only delta is the intended normalization.TypeError; post-fix the session constructs withfftshift=None.Tests
Per "test holoptycho usage, don't duplicate ptychoml":
preprocess_diffraction/apply_d4correctness lives in ptychoml. New here:tests/test_preprocess_geometry.py—crop_flipped_roi(mirrored-window values + invariantcrop_flipped_roi(img,roi) == flip(crop_to_roi(img,roi))).tests/test_vit_session_contract.py— the ptychoml session call contract (fftshiftkwarg).