Skip to content

refactor: move intrinsics/extrinsics into embodiment classes#380

Closed
ElmoPA wants to merge 2 commits into
mainfrom
elmo/refactor-viz
Closed

refactor: move intrinsics/extrinsics into embodiment classes#380
ElmoPA wants to merge 2 commits into
mainfrom
elmo/refactor-viz

Conversation

@ElmoPA

@ElmoPA ElmoPA commented May 1, 2026

Copy link
Copy Markdown
Contributor

Move ARIA/MECKA/SCALE intrinsics and EXTRINSICS dict out of egomimicUtils
and into the embodiment classes (Aria, Mecka, Scale, Eva) as class attributes.
Remove CameraTransforms class and VIZ_INTRINSICS_KEY indirection; viz functions
now take an intrinsics matrix directly via cls.INTRINSICS.

ElmoPA commented May 1, 2026

Copy link
Copy Markdown
Contributor Author

This was referenced May 1, 2026
@ElmoPA ElmoPA marked this pull request as ready for review May 1, 2026 20:09
_fx, _fy = _fx0 * _sx, _fy0 * _sy
_cx, _cy = _cx0 * _sx, _cy0 * _sy

MECKA_INTRINSICS = np.array(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just type it directly into here instead of this ugly code above

ElmoPA added 2 commits June 12, 2026 16:57
Move ARIA/MECKA/SCALE intrinsics and EXTRINSICS dict out of egomimicUtils
and into the embodiment classes (Aria, Mecka, Scale, Eva) as class attributes.
Remove CameraTransforms class and VIZ_INTRINSICS_KEY indirection; viz functions
now take an intrinsics matrix directly via cls.INTRINSICS.
Add Aria.viz mode="gaze" that projects (yaw, pitch, depth) gaze vectors
through ARIA_T_RGB_CPF and Aria.INTRINSICS, plus a "gaze" keymap entry.
Update zarr_data_viz.ipynb to demonstrate the new mode.
@ElmoPA ElmoPA force-pushed the elmo/refactor-viz branch from e088ec3 to 4eb7558 Compare June 12, 2026 20:59
@github-actions

Copy link
Copy Markdown

Claude Code Review

PR Review: Move intrinsics/extrinsics into embodiment classes

Summary

Refactor moves camera intrinsics/extrinsics constants from egomimicUtils.py into embodiment classes as class attributes, and removes the CameraTransforms / VIZ_INTRINSICS_KEY indirection. The direction is good, but the implementation has several correctness bugs that will break training/rollouts.

Key concerns

🔴 1. Bogus LW_BIMANUAL enum alias — will raise ValueError

LW_BIMANUAL = 5  # alias of ARIA_BIMANUAL — routes through aria handling for now

Python Enum does not allow duplicate values to be defined as separate members — LW_BIMANUAL will silently become an alias such that EMBODIMENT.LW_BIMANUAL is EMBODIMENT.ARIA_BIMANUAL, and EMBODIMENT_ID_TO_KEY[5] will resolve to whichever name member.name iteration picks (likely ARIA_BIMANUAL). Even if intentional, this collides with the embodiment-string convention (must match exact enum name). This also looks unrelated to the PR's stated purpose — please remove it or split into a separate PR with proper enum handling (@enum.unique would have caught this; consider IntEnum with a distinct value, or aria_bimanual as the canonical string).

🔴 2. Lost extrinsics — Eva.EXTRINSICS is only x5Dec13_2

The original EXTRINSICS dict contained many calibrations (ariaJul29, ariaFeb17, ariaMar4, ariaJun7, ariaOct18_arx, x5Nov15_2, x5Nov18_3, x5Dec10_2, x5Dec13_2, etc.). This PR drops all of them and keeps only x5Dec13_2 as Eva.EXTRINSICS. Any episode/checkpoint/visualization script keyed off an older calibration is now silently using the wrong extrinsics. Since poses are stored in SLAM world frame and re-expressed at training time, this can silently corrupt training targets for older datasets without raising.

Also: _build_eva_bimanual_eef_frame_transform_list and _build_eva_bimanual_transform_list previously took extrinsics_key as an argument. Removing the parameter eliminates the ability to switch calibrations per dataset.

🔴 3. rollout.pyextrinsics_key removed from PolicyRollout signature

extrinsics_key="x5Dec13_2" was a function arg threaded through main(). Removing it hardcodes one calibration into rollouts. If multiple robot calibrations are used across rollouts, this is a regression. At minimum, keep the parameter (or a calibration enum) even if it currently maps to a single Eva.EXTRINSICS.

🟡 4. Aria class body is malformed

return super().viz(image, viz_data, mode=mode, **kwargs)
    FINGER_EDGES = [

Missing blank line + indentation is suspect — visually looks like FINGER_EDGES was meant to be at class scope but is now positioned after a return inside the viz method's source range. Re-read: actually it's at class scope (no leading whitespace inside def), but Aria.FINGER_EDGES duplicates what's already on Human. Either remove or confirm Aria override is intended. Same for FINGER_COLORS, FINGER_EDGE_RANGES, DOT_COLOR, ACTION_STRIDE=3 (already inherited).

Also missing blank line between Aria and Scale classes.

🟡 5. Scale.EXTRINSICS = None and Mecka.EXTRINSICS = None

Previously these were identity dicts ({"left": np.eye(4), "right": np.eye(4)}). If anything downstream does embodiment.EXTRINSICS["left"] it will now TypeError. Prefer keeping the identity dict for type stability, or document that None means "no extrinsics applied."

🟡 6. Notebook references undefined LightWheel

vis = LightWheel.viz_transformed_batch(batch, mode="traj")

LightWheel is not imported or defined in this diff. Notebook will NameError.

🟡 7. Circular-import risk

viz_utils.py no longer imports INTRINSICS (good), but egomimic/rldb/embodiment/human.py now defines ARIA_INTRINSICS etc., and eva.py imports from egomimic.rldb.embodiment.human import ARIA_INTRINSICS. This is fine, but the constants conceptually belong in a cameras.py / calibrations.py module rather than buried in human.pyEva is not a human embodiment and importing from human for camera intrinsics is confusing.

🟡 8. Test coverage

No new tests. At minimum:

  • A test asserting each embodiment class exposes a valid INTRINSICS (3x4 numpy array) and EXTRINSICS (dict with left/right 4x4 matrices, or explicitly None).
  • A regression test that Eva.EXTRINSICS["left"] matches the previous EXTRINSICS["x5Dec13_2"]["left"] bit-exactly.

Suggestions

  1. Drop the LW_BIMANUAL change from this PR. If needed, do it properly in a separate PR with a unique enum value.
  2. Preserve all extrinsics calibrations. Move them into a CALIBRATIONS: dict[str, dict] class attribute on Eva (and a

Reviewed by Claude · Review workflow

@ElmoPA ElmoPA closed this Jun 16, 2026
@ElmoPA ElmoPA deleted the elmo/refactor-viz branch June 16, 2026 21:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants