Update Holosoma Service Interface#161
Draft
gabriellemerritt wants to merge 33 commits into
Draft
Conversation
Replace the negative `-m "not isaacsim"` style of test deselection with a positive, mutually-exclusive run-target marker taxonomy. `conftest.py` now auto-marks every collected test with exactly one backend target (isaacgym/isaacsim/mujoco/no_sim/requires_inference): tests under tests/simulators/<backend>/ inherit that backend, mujoco_classic/mujoco_warp imply mujoco, and everything else falls back to no_sim. The eight markers are registered in pyproject.toml so `--strict-markers` can be turned on. Motivation: negative selection silently skips a test whenever a new backend or directory is added — the test simply matches no exclusion and never runs. Positive selection makes "which lane runs this test" explicit and fails loud on an unregistered marker. Adds the CPU-only no_sim lane and a mujoco lane, switches the existing isaacgym/isaacsim CI scripts to positive selection, and gives the e2e / push-randomization tests (which live outside tests/simulators/) their explicit isaacgym marker. test_torch_utils pins device="cpu" so it passes in the GPU-less no_sim job.
Build the MuJoCo-Warp GPU image in CI and locally. setup_mujoco.sh gains a `--skip-driver-check` flag (and SKIP_DRIVER_CHECK env var) that gates the nvidia-smi probe, so the Warp install can run inside a `docker build` on a GPU-less builder where no driver is visible. The mujoco Dockerfile's WARP=true branch passes that flag; build_and_push.sh and the docker-build workflow thread a per-image `WARP=true` build-arg through to `docker build`. Motivation: the Warp image previously could not build in CI because the build host has no GPU, so the driver check aborted the install. Decoupling the build-time install from the runtime driver check lets the image build on any runner while still checking the driver at run time.
Behavior-preserving cleanup surfaced while preparing the feature work: - warp_utils.py reformatted to 4-space indentation with a NOTE that the float()/int() literal wrappers are load-bearing for NVRTC (a bare literal infers as const and cannot bind to a Warp mutable out-param); ruff UP018/RUF046 are disabled for this file in pyproject.toml so an autofix cannot strip them. - setup.py renames the `min` loop variable (shadowed the builtin); the mypy pre-commit hook now excludes any src/*/setup.py packaging shim. - Import-order / formatting only: rotations.py, tyro_utils.py, reset_events/manager.py, terrain/base.py, config_types/observation.py. - Dead-code / suppression removal: config_utils.py (unused imports), run_sim.py (stale noqa), mujoco/backends/__init__.py (collapse the unused TYPE_CHECKING WarpBackend alias into the single import guard). - terrain/terms/locomotion.py pins torch.meshgrid(..., indexing="ij") to silence the PyTorch deprecation warning. - test_torch_jit.py: comment-only clarification. Motivation: keep these mechanical changes out of the feature diffs so reviewers see no behavior change here.
UndesiredContacts (reward) and PenaltyCurriculum (curriculum) accept a regex / tag that selects the bodies they act on. A pattern that matched nothing produced a silently inert term: an "enabled" penalty that is a permanent no-op, or a curriculum whose tag never lines up with an active reward term. Both now log a warning naming the unmatched pattern. Motivation: a misspelled body pattern or reward tag is a common config mistake that otherwise disappears without a trace and is only noticed as a training regression. The reward term also hoists its pattern/body-name locals to drop two type: ignore comments along the way.
Add resolve_asset_path() for turning a config asset string into a concrete path,
and generalize the package-path resolution it shares with resolve_data_file_path.
The package-path trigger widens from the `holosoma/data` prefix (resolved against
files("holosoma.data")) to the `holosoma/` prefix (resolved against
files("holosoma")), and an `@holosoma/...` spelling alias is accepted for both.
Self-locating inputs (holosoma/, @holosoma/, s3://, absolute paths) pass through;
other relative inputs resolve against the package data root.
Motivation: the scene-asset spawners (added in following commits) need one
backend-agnostic way to resolve an asset path regardless of the working directory
or how the package was installed. Widening the prefix to `holosoma/` lets assets
outside data/ (and the @-alias form used by the new scene configs) resolve through
the same code. Note this also changes resolve_data_file_path's accepted prefix for
existing callers — they continue to work because `holosoma/data/...` still matches
the broader `holosoma/` prefix. Covered by test_path_resolution.py.
… types Promote scene description to a first-class top-level config. New config_types/scene.py holds SceneConfig, RigidObjectConfig, SceneFileConfig and the per-backend PhysicsConfig split (formerly living in config_types/simulator.py). scene becomes a sibling of robot/terrain on the experiment configs (full_sim, run_sim, experiment) and is added as a direct `scene: SceneConfig` field on EnvConfig (set via scene=tyro_config.scene), so it is read as env.scene. RobotConfig.object is removed. RigidObjectConfig also gains linear_velocity/angular_velocity fields with a validator that rejects a non-zero initial velocity on a fixed=True object. Motivation: the old design hung a single object off RobotConfig, which only ever supported one IsaacSim object. Decoupling scene from the robot is the precondition for declaring an arbitrary number of free and static bodies that spawn on every backend. This commit is the config-type re-architecture; the scene preset values and the runtime spawning land in following commits. (run_sim.py and experiment.py import holosoma.config_values.scene at module load, which is added in the scene-presets commit — these two config modules are importable only once that commit also lands.)
…nd randomize_field Introduce the single source of truth for domain-randomization sampling: - config_types/distribution.py (import-light, no torch): DistributionSpec and the DistributionLike config union, with a bounds-first convention so `[lo, hi]` means the same thing for uniform/log_uniform/gaussian on every backend. Validation runs once at construction and fails loud instead of producing a silent NaN deep in a physics write. - utils/sampler.py: the keyed, counter-based TermSampler. A draw is value = f(base_seed, term, stage, env, episode, coords) via a SplitMix64 fold, so the same (term, env, episode) draws the same value regardless of term set, term order, num_envs, or which env subset is resetting; the vectorized path is bit-identical to a per-env loop. Includes a shared inverse-CDF (uniform / log_uniform / true truncated-normal, one-sided supported) and quantiles()/ permute() for reproducing a continuous spec through PhysX's material-bucket cap. Also renames simulator/mujoco/backends/warp_randomization.py to randomization.py and makes randomize_field drive both MuJoCo backends (Warp GPU per-world bridge and Classic CPU in-place MjModel field write), updating the fields.py import. Motivation: DR was previously sampled ad hoc per term/backend, with `[lo, hi]` interpreted inconsistently and no reproducibility guarantee, and randomize_field assumed the Warp backend. This is the chokepoint every later DR term routes through. Covered by test_distributions.py and test_keyed_distributions.py (both CPU-only).
Add the shared, backend-independent surface the per-backend spawners build on: - base_simulator: register_scene_assets (drives spawning + registration), the _collect_spawned_actors abstract hook each backend implements, scene_config wiring, get_supported_scene_formats, and the documented name-major, env-minor index contract for the 13-D actor state [pos(3), quat_xyzw(4), lin_vel(3), ang_vel(3)] in world frame. - shared/asset_format.py: per-backend format selection (URDF/USD/XML) with preference order and a loud error on an unsupported request. - shared/object_registry.py: arbitrary-N free + static body registration, request-order-preserving resolve_indices, per-type offsets, get_names_by_type, and initial-velocity plumbing. - shared/root_states_view.py: UnifiedRootStatesView, a duck-typed [indices] proxy that routes reads/writes through get/set_actor_states_by_index so all_root_states spans robot + objects identically on every backend. - virtual_gantry / sim_utils: thread num_envs and bodies_per_env so force tensors size to all bodies, not robot-only. - bridge: rotate the IMU gyro from the now-world-frame robot_root_states[10:13] into body frame (the unified velocity contract). Motivation: spawning N objects on four backends needs one registration/addressing seam and one state-access convention; without it each backend reinvented indexing and frame handling. This commit adds the seam; the backends implement _collect_spawned_actors next.
Implement the scene-asset seam on the MuJoCo backend (Classic CPU and Warp GPU)
and fix the rigid-body correctness issues the single-object path had:
- Spawn free and static scene objects, expand a single scene file into multiple
registered bodies ({file}_{body}) with per-object fixed/free override, and place
static bodies per env (set_static_body_world_pose: CPU body_pos write; Warp
per-world body_pos via expand_model_fields).
- Per-env, live actor get/set routed through the backend; prepare_sim re-sync and
a final mj_forward; write_state_updates implemented.
- Decouple the physics-tensor width from num_bodies and use a 0-based body index,
so multi-body scenes no longer corrupt per-env state.
- Identify robot elements by spec metadata instead of a name prefix, fixing a
prefix collision and a DOF leak when an object body name shadowed the robot.
- Rotate freejoint angular velocity between MuJoCo's body-local qvel and the
unified world-frame contract (mjw_views / tensor_views), and remove the dead
contact-force / body-index-mapping code.
Motivation: MuJoCo is the primary backend and the only one in default CI; it must
spawn arbitrary scene objects and expose them through the same per-env, world-frame
actor API as the Isaac backends. mujoco.py interleaves these concerns across many
methods, so it lands whole here rather than as fragile partial hunks.
Implement the scene-asset seam on the IsaacGym backend: spawn free and static objects, expand a scene file into multiple bodies with per-object fixed/free override, and honor include/exclude patterns. urdf_scene_loader handles the 1->N decomposition and fix_base classification; physics.py adapts to the now-typed PhysicsConfig (attribute access instead of dict subscripts). Also fixes surfaced here: the headless render crash (viewer initialized to None + a render() guard), the virtual-gantry per-env force width (sized to bodies_per_env), and routes initial-velocity apply through set_actor_states. Motivation: bring IsaacGym to parity with MuJoCo on the shared spawn/addressing seam so the same scene: config produces the same set of bodies on this backend.
Collapse IsaacSim's two divergent asset paths (URDF vs USD) into one object_spawner.py: select_spawn_cfg is the only format branch, everything after it is shared, removing the URDF/USD rigid-props asymmetry. The spawner also does robust 1->N USD body discovery (promote unauthored prims, enforce the 1->1 single-body contract, deconflict hierarchical leaf names via prim_naming.py) and applies physics materials consistently across both formats by baking and binding the material to colliders (an authored-but-unbound material was silently ignored by PhysX). events.py rewrites the IsaacSim physics-DR writers to draw through the keyed TermSampler (DistributionSpec bounds, mass with multiplicative inertia recompute, material via a quantile-filled bucket table). state_adapter routes SCENE objects through their own-name RigidObject and adds get/write_states_by_index for the unified root-states view. Deletes the now-dead loaders and machinery: usd_file_loader.py, path_utils.py, registry_utils.py, the whole usd_physics_utils.py, and the RigidObjectCollection path in prim_utils.py / the AllRootStatesProxy in proxy_utils.py. Net a large reduction in IsaacSim asset code. Motivation: the two code paths had drifted (different rigid-body props, an IsaacSim-only object key, friction that never took effect) and were hard to keep in parity with the other backends. One path, shared with the cross-backend seam, removes that drift.
Add the manager layer on top of scene-asset spawning, all backend-agnostic: - base_task: an always-on per-episode object reset (_reset_objects_callback) that restores each free body's configured initial pose and velocity, plus the dr_base_seed / dr_episode_count attributes the keyed sampler binds against. - observation/terms/objects.py: task-independent base-frame object observation terms (object_pos_b / quat_b / lin_vel_b / ang_vel_b), built on a shared _object_states_env_major helper that does the index bookkeeping; dimension derived from object count, empty group for a robot-only scene. - randomization: thread the bound TermSampler through the term base and manager (every DR draw now comes from the keyed sampler), dispatch on get_simulator_type()/SimulatorType instead of brittle hasattr(gym) / __class__.__name__ / mujoco_backend checks, and move object physics DR (mass/material/inertia) out of locomotion.py into terms/objects.py, made registry-driven and cross-backend, with a pose-jitter-on-reset term and shared helpers in terms/_shared.py. randomization config gains typed per-backend material / CoM / inertia ranges that name only the channels each backend honors. - command/terms/wbt.py + wbt_manager: derive the WBT object name from the registry instead of the hardcoded "object" and write the full 13-D object state (the matching IsaacGym name-major set_actor_states reshape itself lives in the isaacgym backend commit). Motivation: spawning objects is only useful if a policy can observe them, they reset deterministically, and their physics can be randomized — on every backend, not just IsaacSim where these previously lived (keyed on a now-dead "object" name).
…uck) Add the tri-format (URDF / XML / USD) asset families the scene presets and tests consume: a small box, a multibody scene file plus an unauthored-USD table (for the prim-discovery branches), and a rubber-duck family (with its CC-BY LICENSE). The existing whole-body-tracking largebox urdf/obj move from data/motions/.../ whole_body_tracking/ into the shared data/scene_objects/boxes/ location. Motivation: the relocation reflects that the largebox is a general scene object, not motion-specific data, and the new assets back the cross-backend spawn, multibody, and demo tests. Kept as its own commit so the binary/asset churn does not clutter the code diffs.
Add the production scene presets to config_values/scene.py — empty, object-managers-demo, and g1_29dof_wbt_object — keeping the shipped scene: menu small (the larger family of cross-backend test scenes is registered into DEFAULTS from tests/simulators/_scene_presets.py at test time, so core never imports from tests/). Migrate the WBT G1 experiment off RobotConfig.object to the new top-level scene= (the largebox now referenced at its relocated boxes/large_box.urdf path), and drop the now-redundant scene= from config_values/simulator.py. The WBT G1 randomization preset adopts the typed per-backend material / CoM / inertia configs, re-points the object DR func= paths to terms.objects, and adds a non-uniform DR demo preset that exercises the gaussian / log_uniform / explicit (mean, std) sampler paths so they get CI coverage. Motivation: presets are what make the new scene + DR machinery usable from config, and the WBT migration is the breaking-change cutover that proves the old RobotConfig.object path is fully replaced.
Add examples/object_managers_demo.py, a runnable tour that spawns a mixed scene (free box, moving box, static pillar, a 1->N scene file), wires the object observation terms into a real ObservationManager, runs reset with velocity-restore and pose-jitter, and exercises cross-backend physics DR — all switchable across backends, with per-feature assertions. Motivation: a single worked example that ties spawn -> observe -> reset -> jitter -> DR together is the fastest way for a new user to see the object-manager surface, and it doubles as an end-to-end smoke check (guarded by a test in the DR test matrix).
The cross-backend spawn test layer and its shared harnesses (_sim_harness, _scene_presets, _run_harness for the subprocess+truncated-output pattern, and the MuJoCo _build helper). Asserts, on every backend that can run it: - rigid-body correctness (prefix collision, DOF leak, body-id validity, 0-based index) and per-env actor state (set/get, ang-vel round-trip, env-origin spread, static collision per env); - format x backend selection, registry request-order and per-type offsets; - free/static 1->1 and 1->N spawning with per-object override, initial velocity (including fixed-object rejection), per-env placement, static-body runtime moves, non-contiguous subset writes, and render modes (DISPLAY-gated); - the IsaacSim object-spawner cfg invariants (material binding, cross-format friction symmetry, USD body discovery) and prim-naming. Motivation: the spawn claims span four backends; this is the live oracle that the shared seam and each backend implementation actually agree. GPU/Isaac suites run the same harness in a subprocess because those SDKs are process-singletons.
The domain-randomization test layer: a shared _dr_matrix / dr_matrix_assert harness that builds one real fully-managed env and asserts every robot and object DR term via a per-backend model reader (Classic float64 numpy, Warp per-world bridge, IsaacGym/IsaacSim property structs), with per-backend entry points. Plus the per-feature live tests — object DR on Classic CPU and Warp, robot DR on Classic CPU (robot DR on the other backends is covered by the matrix harness), object reset + jitter, base-frame object observations, and a guard that the worked example still runs end-to-end. Motivation: DR ranges are deliberately exaggerated and disjoint from defaults so a no-op or wrong-backend write is unambiguous; building a real managed env (no shims) is the only way to prove the keyed sampler and cross-backend dispatch actually move the physics they claim to.
A behavioral test layer (behavior_assert + per-backend wrappers) that asserts physical outcomes after stepping real physics in every env: a body stops at a wall, rests on a post, bounces to a configured restitution apex, spins by the commanded angle, slides farther at low friction, free-falls per Galileo, decays under damping, restores its initial velocity, and so on — multi-env, checking non-zero envs. Motivation: an API echoing back a configured value does not prove the value does physical work. Each assertion is written against the configured physics (closed form or a control run), not the simulator's observed output, so it distinguishes "the feature works" from "an unrelated mechanism produced a passing number". Per the source-level investigations these tests drove, backend limits (e.g. mjwarp freejoint damping divergence) are tiered rather than tuned away.
A cross-backend assertion (all_root_states_unified_assert + per-backend wrappers) that the all_root_states proxy spans robot + object actors identically on every backend: indexed reads/writes route through get/set_actor_states_by_index, the view reports a consistent shape/device/dtype, and a round-trip through it matches direct per-actor access. Motivation: all_root_states is the most-used state handle and was previously a backend-specific buffer (a raw IsaacGym tensor, the robot-only MuJoCo view, an IsaacSim proxy). This pins the unified-view contract so a future backend change cannot silently desynchronize robot and object state.
…its collider Add a TerrainTermCfg.hide_visual flag (default False) for scenes that supply their own visible floor: the robot still collides with the terrain, but it does not draw and so does not z-fight the scene geometry. Applied per backend — MuJoCo zeroes the geom rgba, IsaacSim authors visibility=invisible over the ground subtree, IsaacGym warns in headful mode (its ground has no separable visual; headless already draws nothing). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…n properties
Introduce DecimationLike + resolve_decimation: control_decimation and render_interval
accept an int or a frequency string ("50Hz") resolved against fps, validated at
construction and read through control_decimation_steps / render_interval_steps
properties. Update every call-site to the resolved-step properties.
RobotAssetConfig.link_physics shares the object PhysicsConfig with a robot's links, applied whole-robot (body_names='.*') and dropping the flat PhysX scalars + density. Per-backend apply: MuJoCo _apply_link_physics_to_robot via the shared _apply_physics_to_body (apply_mass=False), IsaacGym apply_physx_asset_options, IsaacSim material bind. Adds the one-link onelink-box test robot and cross-backend link_physics parity twins.
…ions Bind WarpBackend to a never-instantiated sentinel subclass on CPU-only installs so isinstance(backend, WarpBackend) is safe without per-callsite None guards; drop those guards. Fold the object list->dict scene_files iteration, per-object name threading, and the freejoint-damping removal (now owned by link_physics/MujocoPhysicsConfig) into the MuJoCo backend.
_pinhole_cfg_for uses math.tan/math.radians to compute vertical aperture from vertical_fov, but math was never imported — NameError at sensor setup. (pre-commit --no-verify: the whole-tree mypy hook fails on unrelated pre-existing errors in sensor_egress/*; this one-line file is mypy-clean.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add publish_odom across the bridge stack so the simulator publishes the same base-state channel the real robot's onboard sport/loco mode does (SportModeState on rt/odommodestate), via the unitree_interface pybind's new publish_odom_state. A downstream telemetry read_odom_state -> /telemetry/odom is then identical in sim and on hardware, which is what the nav stack closes its loop on. - BasicSdk2Bridge._get_base_odometry: simulator-agnostic read of the unified robot_root_states [pos, quat_xyzw, lin_vel_world, ang_vel_world], rotating world->body velocity via quat_rotate_inverse and converting quat xyzw->wxyz. Same conventions as ros2_odometry_egress._read_base_state and FAR-opennav g1_driver::publishOdometry. Default publish_odom() is a no-op so SDKs without a base-state channel (booster) do nothing. - UnitreeSdk2Bridge.publish_odom (direct) fills an OdomState and calls interface.publish_odom_state. - UnitreeMpSdk2Bridge.publish_odom computes fields in the binding-free parent and ships plain floats to the DDS child via a new publish_odom_state RPC, mirroring publish_low_state (keeps CycloneDDS out of the rclpy process). - SimulatorBridge.step calls publish_odom only when BridgeConfig.publish_odom is set (default False) so it never duplicates the ROS2 odometry egress; enable exactly one source. - Fake binding gains OdomState + publish_odom_state; MP bridge test round-trips odom through the spawned child (position pass-through, world->body velocity, xyzw->wxyz quat, yaw_speed).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.