Skip to content

feat(preprocess): delegate model branch to ptychoml + geometry/DC knobs#39

Merged
Garrett Bischof (gwbischof) merged 2 commits into
mainfrom
h2/preprocess-geometry-knobs
Jun 8, 2026
Merged

feat(preprocess): delegate model branch to ptychoml + geometry/DC knobs#39
Garrett Bischof (gwbischof) merged 2 commits into
mainfrom
h2/preprocess-geometry-knobs

Conversation

@gwbischof

@gwbischof Garrett Bischof (gwbischof) commented Jun 5, 2026

Copy link
Copy Markdown
Collaborator

H2 of the #37 decomposition. The complete DC-convention / diffraction-preprocessing change: routes ImagePreprocessorOp's model branch through ptychoml.preprocess_diffraction + the tap through ptychoml.apply_d4, exposes geometry/normalization as config knobs, and aligns the ViT session with ptychoml's renamed fftshift API. Mirrors #37's working end-to-end DC handling (auto-detect everywhere).

Unlike H1, this is intentionally a behavior change (it adds the training-pipeline normalization). The orientation/fftshift defaults reproduce #37's proven configuration.

⚠️ Also fixes a runtime break on main

ptychoml renamed PtychoViTInference's data_is_shifted param → fftshift. holoptycho main still calls PtychoViTInference(data_is_shifted=...), which raises TypeError against the pinned ptychoml (the bump in #38 crossed the rename) — the ViT op would crash on its first batch. CI/smoke missed it: the vit_inference import dies earlier on TILED_BASE_URL, and the CI envs skip holoscan. This PR renames PtychoViTInferenceOp's param data_is_shiftedfftshift (default None = auto-detect) and drops data_is_shifted=True in ptycho_holo.py, matching #37. A new tests/test_vit_session_contract.py pins the holoptycho↔ptychoml call contract so a future rename fails in CI, not on the beamline.

Changes (preprocess.py, vit_inference.py, ptycho_holo.py)

New scan-JSON knobs, all with back-compatible defaults:

Knob Default Effect
tap_orient antitranspose D4 on the saved intensity tap — reproduces the old anti-diagonal flip
dp_orient rot90_cw D4 on the model branch — verified to reproduce the old orientation chain (max diff ~1e-6)
fftshift_dp None DC convention; None = ptychoml auto-detect via detect_dc_at_corner (matches #37; pairs with the session's own fftshift=None)
vit_normalization / vit_scale 1e5 / 1e4 Intensity normalization → the one intended behavior change (amplitudes rescaled by sqrt(scale/normalization))
hot_pixel_count_threshold None Photon-count hot-pixel zeroing (matches hxn_to_vit.py)

Also:

  • auto_center default flipped off — the shift isn't part of ptychoml's pipeline and displaces the beam from the model's trained position.
  • Removed the now-unused dp_transpose knob (folded into dp_orient).
  • Eiger2 horizontal flip now mirrors the column ROI; extracted to crop_flipped_roi.

Scope boundary

Takes only the DC-convention/preprocessing subset of #37. The orientation auto-detect buffering + swap_xy (H5), the vit-only-mode guards + ONNX inner-crop (H4), and livestitch/canvas (H3) are deliberately not here. fftshift=None end-to-end is #37's working config; the further "resolve-once" streaming hardening (if any) rides with the autodetect work.

Verification

  • dp_orient='rot90_cw' + preprocess_diffraction checked numerically vs the old inline chain on random input — identical orientation+fftshift, max abs diff 1.9e-6; only delta is the intended normalization.
  • Live repro confirmed the pre-fix TypeError; post-fix the session constructs with fftshift=None.

Tests

Per "test holoptycho usage, don't duplicate ptychoml": preprocess_diffraction/apply_d4 correctness lives in ptychoml. New here:

  • tests/test_preprocess_geometry.pycrop_flipped_roi (mirrored-window values + invariant crop_flipped_roi(img,roi) == flip(crop_to_roi(img,roi))).
  • tests/test_vit_session_contract.py — the ptychoml session call contract (fftshift kwarg).
pixi run test   # 66 passed; the 2 vit_inference/ptycho_holo smoke failures are
                # pre-existing (module load needs a reachable TILED_BASE_URL).

Route ImagePreprocessorOp's model-input branch through
ptychoml.preprocess_diffraction (normalize -> hot-pixel mask -> sqrt -> D4
-> fftshift) and the intensity tap through ptychoml.apply_d4, replacing the
hardcoded antidiag-flip/rot90/fftshift/transpose/sqrt chain. New scan-JSON
knobs (all with back-compatible defaults):

- tap_orient (default 'antitranspose'): D4 on the saved intensity tap;
  reproduces the old anti-diagonal flip.
- dp_orient (default 'rot90_cw'): D4 on the model branch; verified to
  reproduce the old orientation chain exactly (max diff ~1e-6).
- fftshift_dp (default None): DC convention; None lets ptychoml auto-detect
  via detect_dc_at_corner (centers DC only when at the corners — matches the
  old unconditional shift for typical detector input).
- vit_normalization / vit_scale (1e5 / 1e4): intensity normalization passed
  to preprocess_diffraction so model input matches the offline training
  scale. This is the one intended behavior change (amplitudes rescaled by
  sqrt(scale/normalization)); harmless global scale for the iterative DM
  engine, required for the ViT model.
- hot_pixel_count_threshold (default None/disabled).

Also: auto_center default flipped to off (the shift isn't part of ptychoml's
pipeline and displaces the beam from the model's trained position); remove
the now-unused dp_transpose knob; ImageBatchOp Eiger2 flip mirrors the
column ROI (extracted to crop_flipped_roi, tested).

H2 of the #37 decomposition. preprocess_diffraction/apply_d4
correctness is covered by ptychoml's tests; the new holoptycho geometry
(crop_flipped_roi) is tested in tests/test_preprocess_geometry.py.

Co-authored-by: Himanshu Goel <4122621+himanshugoel2797@users.noreply.github.com>
ptychoml renamed PtychoViTInference's 'data_is_shifted' parameter to
'fftshift' (None = auto-detect DC via detect_dc_at_corner). holoptycho main
still called PtychoViTInference(data_is_shifted=...), which raises TypeError
against the pinned ptychoml (the bump in #38 crossed the rename) — the ViT op
would crash on its first batch. Smoke tests missed it: the vit_inference
import dies earlier on TILED_BASE_URL, and CI envs skip holoscan entirely.

Rename PtychoViTInferenceOp's param data_is_shifted -> fftshift and pass
fftshift= to the session; drop data_is_shifted=True in ptycho_holo.py and let
it default to None (auto-detect), matching PR #37's end-to-end DC handling
that pairs with this PR's preprocess_diffraction(fftshift=fftshift_dp=None).

Adds tests/test_vit_session_contract.py to pin the holoptycho<->ptychoml call
contract so a future rename fails in CI instead of on the beamline.

Co-authored-by: Himanshu Goel <4122621+himanshugoel2797@users.noreply.github.com>
@gwbischof Garrett Bischof (gwbischof) merged commit 4767711 into main Jun 8, 2026
5 checks passed
@gwbischof Garrett Bischof (gwbischof) deleted the h2/preprocess-geometry-knobs branch June 8, 2026 17:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant