Add Noitom full-body teleop, recording, and replay support

.pre-commit-config.yaml

@@ -8,12 +8,16 @@ repos:
       # Run the linter
       - id: ruff
         args: ["--fix"]
+        # The bundled Noitom MocapApi SDK is vendor code; keep it byte-stable.
+        exclude: ^src/plugins/noitom_mocap/third_party/mocap_api/
       # Run the formatter
       - id: ruff-format
+        exclude: ^src/plugins/noitom_mocap/third_party/mocap_api/
   - repo: https://github.com/pre-commit/pre-commit-hooks
     rev: v6.0.0
     hooks:
       - id: trailing-whitespace
+        exclude: ^src/plugins/noitom_mocap/third_party/mocap_api/
       - id: check-symlinks
       - id: destroyed-symlinks
       - id: check-added-large-files
@@ -24,6 +28,7 @@ repos:
       - id: check-executables-have-shebangs
       - id: check-toml
       - id: end-of-file-fixer
+        exclude: ^src/plugins/noitom_mocap/third_party/mocap_api/
       - id: check-shebang-scripts-are-executable
       - id: detect-private-key
       - id: debug-statements

CMakeLists.txt

@@ -59,6 +59,7 @@ endif()
 # Options
 option(BUILD_PLUGINS "Build plugins" ON)
 option(BUILD_PLUGIN_OAK_CAMERA "Build OAK camera plugin (requires vcpkg for DepthAI v3.x)" OFF)
+option(BUILD_PLUGIN_NOITOM_MOCAP "Build Noitom mocap plugin (uses bundled MocapApi SDK)" OFF)
 option(BUILD_EXAMPLES "Build examples" ON)
 option(BUILD_EXAMPLE_TELEOP_ROS2 "Build only the teleop_ros2 example (e.g. for Docker)" OFF)
 option(BUILD_TESTING "Build unit tests" ON)
@@ -187,6 +188,9 @@ if(BUILD_PLUGINS)
     if(BUILD_PLUGIN_OAK_CAMERA)
         add_subdirectory(src/plugins/oak)
     endif()
+    if(BUILD_PLUGIN_NOITOM_MOCAP)
+        add_subdirectory(src/plugins/noitom_mocap)
+    endif()
 endif()
 
 # Formatting enforcement (runs on Linux by default)

LICENSES/LicenseRef-Noitom-MocapApi-SDK.txt

@@ -0,0 +1,9 @@
+Noitom MocapApi SDK License Reference
+
+The files under src/plugins/noitom_mocap/third_party/mocap_api/ are bundled
+third-party Noitom MocapApi SDK artifacts.
+
+The copied SDK drop in this repository does not include a redistributable
+license text or copyright notice. Keep those files annotated with
+LicenseRef-Noitom-MocapApi-SDK in REUSE.toml instead of assigning NVIDIA
+copyright or Apache-2.0 metadata to the vendor SDK.

REUSE.toml

@@ -41,3 +41,12 @@ path = "examples/cloudxr_mujoco_teleop/**"
 precedence = "override"
 SPDX-FileCopyrightText = "Copyright (c) 2026 HTC Corporation"
 SPDX-License-Identifier = "Apache-2.0"
+
+# The bundled Noitom MocapApi SDK does not ship SPDX metadata in the copied
+# headers/bindings; keep the vendor path annotated instead of adding NVIDIA
+# copyright headers to third-party SDK files.
+[[annotations]]
+path = "src/plugins/noitom_mocap/third_party/mocap_api/**"
+precedence = "override"
+SPDX-FileCopyrightText = "NOASSERTION"
+SPDX-License-Identifier = "LicenseRef-Noitom-MocapApi-SDK"

docs/source/device/haptikos.rst

@@ -4,6 +4,19 @@
 Haptikos Exoskeletons
 =====================
 
+.. raw:: html
+
+   <div style="margin-top: 1rem;"></div>
+
+.. image:: ../_static/haptikos-exoskeletons.gif
+   :alt: Haptikos exoskeletons used with Isaac Teleop
+   :class: align-center
+   :width: 100%
+
+.. raw:: html
+
+   <div style="margin-bottom: 1.5rem;"></div>
+
 Use the `Haptikos <https://haptikos.tech/>`_ Exoskeletons with the Isaac Teleop framework. Currently only Linux is supported. Tested on Meta Quest headsets. Other headsets with controllers may work as well.
 
 .. contents:: On this page

examples/noitom/README.md

@@ -0,0 +1,203 @@
+<!--
+SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+SPDX-License-Identifier: Apache-2.0
+-->
+
+# Noitom G1 Test Task
+
+This example registers an external Isaac Lab task that is based on
+`Isaac-PickPlace-Locomanipulation-G1-Abs-v0` and swaps its IsaacTeleop pipeline
+to read the `noitom_mocap` OpenXR tensor collection as `FullBodyPosePN` data
+defined in `src/core/schema/fbs/full_body_pn.fbs`.
+
+Noitom data is converted before it reaches this task. The plugin publishes the
+Perception Neuron full-body schema on tensor identifier `full_body_pn`; this
+example consumes it with `FullBodyTrackerPN("noitom_mocap")`.
+
+The registered task id is:
+
+```bash
+Isaac-PickPlace-Locomanipulation-G1-Noitom-Abs-v0
+```
+
+## Examples
+
+This README has two workflows:
+
+- Teleop: stream live Noitom full-body data into the G1 IsaacLab task.
+- Record and replay: capture the same `full_body_pn` stream to MCAP and play it
+  back without running the teleop task.
+
+### Example 1: Teleop
+
+![Noitom G1 live teleoperation](assets/teleop.gif)
+
+Terminal 1, keep CloudXR running:
+
+```bash
+conda activate teleop
+cd ~/IsaacTeleop
+python -m isaacteleop.cloudxr
+```
+
+Terminal 2, publish Noitom data into CloudXR/OpenXR:
+
+```bash
+source ~/.cloudxr/run/cloudxr.env
+cd ~/IsaacTeleop
+./install/plugins/noitom_mocap/noitom_mocap_plugin --host=<hds-host> --port=<hds-port>
+```
+
+The default plugin collection id is `noitom_mocap`, which matches
+`NoitomG1Settings.collection_id`. If you pass `--collection-id` to the plugin,
+update `NoitomG1Settings.collection_id` to the same value.
+
+Terminal 3, run IsaacLab with the external task registration callback:
+
+```bash
+conda activate teleop
+cd ~/dependence/IsaacLab3-0
+PYTHONPATH=~/IsaacTeleop/examples/noitom:$PYTHONPATH \
+./isaaclab.sh -p scripts/environments/teleoperation/teleop_se3_agent.py \
+  --task Isaac-PickPlace-Locomanipulation-G1-Noitom-Abs-v0 \
+  --visualizer kit \
+  --xr \
+  --external_callback noitom_tasks.register_tasks
+```
+
+If CloudXR is already running in Terminal 1 and IsaacLab should not auto-launch
+another runtime, add:
+
+```bash
+--cloudxr_env none --no-auto_launch_cloudxr
+```
+
+### Example 2: Record and replay
+
+Record and replay use the same Noitom OpenXR tensor collection as the teleop
+task. Keep CloudXR and `noitom_mocap_plugin` running as in Example 1.
+
+Before recording from a development checkout, rebuild the Python package and
+make sure this shell imports the rebuilt package, not an older package installed
+in the conda environment:
+
+```bash
+cd ~/IsaacTeleop
+cmake --build build --target python_package noitom_mocap_plugin --parallel
+
+PYTHONPATH=$PWD/build/python_package/Release \
+python - <<'PY'
+import isaacteleop.deviceio as deviceio
+deviceio.FullBodyTrackerPN("noitom_mocap", 16 * 1024)
+print(deviceio.__file__)
+PY
+```
+
+Record the Noitom full-body stream:
+
+![Noitom full-body recording](assets/record.gif)
+
+```bash
+PYTHONPATH=$PWD/build/python_package/Release \
+python examples/noitom/record_noitom_full_body.py --duration 10
+```
+
+By default the recorder reads collection `noitom_mocap`, writes MCAP channel
+base `full_body_pn`, and stores the file under `examples/noitom/recordings/`.
+Override paths or ids as needed:
+
+```bash
+python examples/noitom/record_noitom_full_body.py \
+  --collection-id noitom_mocap \
+  --channel-name full_body_pn \
+  --duration 10 \
+  --output /tmp/noitom_full_body.mcap
+```
+
+Replay the newest recording:
+
+![Noitom MCAP replay](assets/replay.gif)
+
+```bash
+PYTHONPATH=$PWD/build/python_package/Release \
+python examples/noitom/replay_noitom_full_body.py
+```
+
+Replay an explicit file:
+
+```bash
+python examples/noitom/replay_noitom_full_body.py /tmp/noitom_full_body.mcap
+```
+
+`replay_noitom_full_body.py` uses the Python `mcap` package to infer playback
+duration from the file summary. If that package is unavailable in your test
+environment, pass a duration explicitly:
+
+```bash
+python examples/noitom/replay_noitom_full_body.py /tmp/noitom_full_body.mcap --duration 10
+```
+
+The resulting MCAP stores `core.FullBodyPosePNRecord` on the `full_body_pn`
+channel path. Replay it with `examples/noitom/replay_noitom_full_body.py`, not
+the PICO `FullBodyPosePicoRecord` replay path.
+
+If the recorder prints a "waiting for Noitom frames" message, check that the
+plugin is still running in the same OpenXR/CloudXR runtime and that both sides
+use the same `collection_id`.
+
+## Behavior
+
+Retargeting lives in `noitom_retargeting.py` and is wired by `noitom_tasks.py`.
+It maps Noitom `FullBodyPosePN` shoulder / elbow / wrist bones into G1 **arm joint
+deltas** for direct joint-position teleop (no Pink IK).
+
+Pipeline:
+
+```text
+FullBodyPosePN
+  → torso frame (pelvis, SPINE3, shoulders)
+  → left/right wrist SE3 deltas (shoulder, elbow, wrist)
+  → arm-length scale + two-link reach clamp
+  → left/right arm joint deltas (shoulder, elbow, wrist; 7+7)
+  → action [left_arm(7), right_arm(7), hand_joints(14), locomotion(4)]
+```
+
+Upper body uses :class:`~isaaclab.envs.mdp.actions.JointPositionAction` (not Pink IK).
+Each arm joint is retargeted from the corresponding mocap bone using the same
+torso-relative delta idea as the legacy wrist-only path.
+
+The G1 action layout is:
+
+```text
+[left_arm(7), right_arm(7), hand_joints(14), locomotion(4)]
+```
+
+Arm deltas use shoulder / elbow / wrist mocap in a torso-relative frame (position
+for shoulder–elbow–wrist chain, orientation for wrist). Hand joints and
+locomotion stay fixed for bring-up unless extended later.
+Finger retargeting is intentionally omitted.
+
+Calibration:
+
+1. After teleop **reset** (`execution_events.reset`), retargeting clears its
+   neutral reference.
+2. Hold a stable upper-body pose; the next valid frame becomes the neutral
+   reference (`calibrated=ready` in logs).
+3. Motion is applied relative to that neutral pose.
+
+Default settings:
+
+- `NoitomG1Settings` in `noitom_tasks.py` (collection, visualization)
+- `NoitomRetargetingSettings` in `noitom_retargeting.py` (scale, smoothing,
+  robot arm lengths, workspace clamps)
+
+When Kit visualization is enabled, the Noitom frame is shown as a cyan stick
+figure anchored to the robot pelvis (orange crosses mark retargeted wrist
+targets). The visualizer uses the same Y-up to Z-up conversion as the retargeter
+(`noitom_position_to_isaac`).
+
+**Calibration tip:** At reset, hold a pose close to the G1 default upper-body
+posture (elbows bent, hands near chest height). Calibration maps your neutral
+mocap pose to the robot nominal wrist targets; a large mismatch at calibration
+will show the cyan skeleton far from the orange wrist targets even when you are
+not moving.