feat(vit): derive ViT inner_crop from the ONNX probe geometry#42
Merged
Conversation
Add holoptycho/engine_probe.py:
- find_onnx_for_engine: map a compiled {name}_v{ver}.engine to its source
onnx/{name}/{ver}/*.onnx (the model_manager cache convention).
- inner_crop_from_onnx: load the ONNX, extract the complex probe
(probe_real/probe_imag, or 2-D constant graph outputs), and hand the probe
to ptychoml.inner_crop_from_probe (geometry now lives in ptychoml). Falls
back to None when onnx is unavailable or the probe can't be extracted.
Wire it in PtychoApp.compose(): inner_crop precedence is explicit config >
ONNX-probe-derived > None (SaveViTResult auto-derives min(H,W)//4).
The helpers live in their own module (not vit_inference) so they're free of
the Holoscan/Tiled import-time setup and can be unit tested. find_onnx_for_engine
is fully covered in tests/test_engine_probe.py (filename convention, sorting,
bad name, missing/empty dir) plus the graceful-None fallback; the inscribed-
square geometry is tested in ptychoml, and the ONNX *extraction* path runs in
real model loads (onnx isn't in the CI test env).
First of the two H4 PRs (true vit-only mode follows).
Co-authored-by: Himanshu Goel <4122621+himanshugoel2797@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
H4a of the #37 decomposition — derive the ViT-patch
inner_cropfrom the model's baked probe instead of always auto-deriving it. (H4b, true vit-only mode, follows separately.)Changes
New
holoptycho/engine_probe.py:find_onnx_for_engine— map a compiled{name}_v{ver}.engineto its sourceonnx/{name}/{ver}/*.onnx(themodel_managercache convention).inner_crop_from_onnx— load the ONNX, extract the complex probe (probe_real/probe_imaginitializers, or two 2-D constant graph outputs), and hand it toptychoml.inner_crop_from_probe(the inscribed-square geometry, added in feat(preprocess): add inner_crop_from_probe — ViT-patch crop from probe geometry ptychoml#12). Falls back toNonewhenonnxis unavailable or the probe can't be extracted.Wired in
PtychoApp.compose():inner_cropprecedence is explicit config > ONNX-probe-derived > None (thenSaveViTResultauto-derivesmin(H,W)//4).Why a separate module
The helpers are pure (
Path/re/onnx/ptychoml— no Holoscan/Tiled), andvit_inferencedoesn't use them internally (onlycompose()does). Putting them inengine_probe.pykeeps them out ofvit_inference's import-time Tiled setup so they're unit-testable.Tests
tests/test_engine_probe.py(7) fully coversfind_onnx_for_engine— the holoptycho-specific filename logic (match, first-sorted, multi-digit version + underscored name, bad name, missing dir, empty dir) — plus the graceful-Nonefallback. The inscribed-square geometry is tested in ptychoml; the ONNX extraction path runs in real model loads (onnxisn't in the CI test env, soinner_crop_from_onnxreturnsNonethere).Depends on
NSLS2/ptychoml#12 (merged);
pixi.lockbumped.