diff --git a/AGENTS.md b/AGENTS.md index 227770bd4..c96a0e92c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -34,7 +34,7 @@ UniLab 是一个 **高性能、模块化、contract 驱动** 的 RL infrastructu - **WARNING_LIST**：`reward.*`、`env.control_config.simulate_action_latency`、`env.ctrl_dt`。 - **ALLOWLIST**（自由覆盖）：`training.sim_backend`、`env.scene`、`training.play_steps`、`env.domain_rand`、`env.noise_config`、`env.commands.vel_limit`。 -训练时 `ExperimentTracker.start()` 把上述字段写入 `run_config.json` 的 `contract_snapshot`（不改 checkpoint 格式，旧 run 无 snapshot 时 fallback + warning）；五个 play 入口在建 env 前调用 `resolve_sim2sim_config` 校验，并用 `policy_load_dim_guard` 包裹 checkpoint 加载以把维度不匹配的隐晦报错重抛为显式诊断。设 `training.sim2sim_strict=false` 可把 DENYLIST 差异降级为 warning（默认 `true`）。DENYLIST 字段应通过 task 的 `base.yaml` 共享（范例：`conf/ppo/task/g1_walk_flat/{base,mujoco,motrix}.yaml`）；跨后端契约审计见 `scripts/audit_sim2sim_contracts.py`。 +训练时 `ExperimentTracker.start()` 把上述字段写入 `run_config.json` 的 `contract_snapshot`（不改 checkpoint 格式，旧 run 无 snapshot 时 fallback + warning）；五个 play 入口在建 env 前调用 `resolve_sim2sim_config` 校验，并用 `policy_load_dim_guard` 包裹 checkpoint 加载以把维度不匹配的隐晦报错重抛为显式诊断。设 `training.sim2sim_strict=false` 可把 DENYLIST 差异降级为 warning（默认 `true`）。DENYLIST 字段在每个后端 owner 配置中显式声明并保持跨后端一致（范例：`conf/ppo/task/g1_walk_flat/{mujoco,motrix}.yaml`）；跨后端契约审计见 `scripts/audit_sim2sim_contracts.py`。 ## Pointers diff --git a/README.md b/README.md index deb33f76e..5058bd966 100644 --- a/README.md +++ b/README.md @@ -132,7 +132,7 @@ uv run eval --algo appo --task go2_joystick_flat --sim motrix --load-run -1 uv run eval --algo appo --task go2_joystick_flat --sim motrix --load-run -1 --render-mode record ``` -This routes through the `go2_joystick_flat/motrix` task owner config and keeps backend selection explicit. +This routes through the `go2_joystick_flat/motrix` task owner config and keeps backend selection explicit. Each backend owner carries an optional `play_profile` block that layers render-only overrides at eval time (`training.play_only=true`) without affecting training. On macOS / MacBook, the UniLab CLI routes Motrix interactive playback through `mxpython` when needed. Motrix defaults to interactive playback; use `--render-mode record` for headless video export or `--render-mode none` to skip playback. Detailed script-level commands are in the [Training Guide](https://unilabsim.github.io/UniLab-doc/en/2-user_guide/1-training/0-index.html). diff --git a/README_zh.md b/README_zh.md index e1e9be84f..39a50dc1a 100644 --- a/README_zh.md +++ b/README_zh.md @@ -134,7 +134,7 @@ uv run eval --algo appo --task go2_joystick_flat --sim motrix --load-run -1 uv run eval --algo appo --task go2_joystick_flat --sim motrix --load-run -1 --render-mode record ``` -这会路由到 `go2_joystick_flat/motrix` 任务 owner 配置，并保持后端选择显式化。 +这会路由到 `go2_joystick_flat/motrix` 任务 owner 配置，并保持后端选择显式化。每个后端 owner 带一个可选的 `play_profile` 块，在 eval 时（`training.play_only=true`）叠加仅渲染相关的覆盖，不影响训练。 在 macOS / MacBook 上，UniLab CLI 在需要时会通过 `mxpython` 路由 Motrix 交互式回放。Motrix 默认使用交互式回放；要导出无头视频请使用 `--render-mode record`，要跳过回放请使用 `--render-mode none`。更细的脚本级命令请参阅 [训练指南](https://unilabsim.github.io/UniLab-doc/zh_CN/2-user_guide/1-training/0-index.html)。 diff --git a/conf/ppo/task/allegro_inhand/motrix.yaml b/conf/ppo/task/allegro_inhand/motrix.yaml index d4d68902b..3608d9743 100644 --- a/conf/ppo/task/allegro_inhand/motrix.yaml +++ b/conf/ppo/task/allegro_inhand/motrix.yaml @@ -50,3 +50,7 @@ env: joint_noise: 0.0 ball_vel_noise: 0.0 ball_z_offset: 0.0 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/allegro_inhand/mujoco.yaml b/conf/ppo/task/allegro_inhand/mujoco.yaml index 3bfed4175..17d639913 100644 --- a/conf/ppo/task/allegro_inhand/mujoco.yaml +++ b/conf/ppo/task/allegro_inhand/mujoco.yaml @@ -54,3 +54,7 @@ env: joint_noise: 0.0 ball_vel_noise: 0.0 ball_z_offset: 0.0 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/allegro_inhand_grasp/motrix.yaml b/conf/ppo/task/allegro_inhand_grasp/motrix.yaml index 8dfe5c1d4..6b5cdca31 100644 --- a/conf/ppo/task/allegro_inhand_grasp/motrix.yaml +++ b/conf/ppo/task/allegro_inhand_grasp/motrix.yaml @@ -31,3 +31,7 @@ env: push_robots: false ball_vel_noise: 0.0 joint_noise: 0.25 # random sampling of the grasp poses +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/allegro_inhand_grasp/mujoco.yaml b/conf/ppo/task/allegro_inhand_grasp/mujoco.yaml index 20d565f51..e6e62f2f6 100644 --- a/conf/ppo/task/allegro_inhand_grasp/mujoco.yaml +++ b/conf/ppo/task/allegro_inhand_grasp/mujoco.yaml @@ -31,3 +31,7 @@ env: push_robots: false ball_vel_noise: 0.0 joint_noise: 0.25 # random sampling of the grasp poses +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_box_tracking/mujoco.yaml b/conf/ppo/task/g1_box_tracking/mujoco.yaml index bb39b7ee4..278a8b8f3 100644 --- a/conf/ppo/task/g1_box_tracking/mujoco.yaml +++ b/conf/ppo/task/g1_box_tracking/mujoco.yaml @@ -42,3 +42,7 @@ reward: std_joint_vel: 1.0 std_object_pos: 0.2 std_object_ori: 0.3 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_climb_tracking/motrix.yaml b/conf/ppo/task/g1_climb_tracking/motrix.yaml index 7c6407e17..0c30db38e 100644 --- a/conf/ppo/task/g1_climb_tracking/motrix.yaml +++ b/conf/ppo/task/g1_climb_tracking/motrix.yaml @@ -78,3 +78,7 @@ reward: std_body_ang_vel: 3.14 std_joint_pos: 0.2 std_joint_vel: 1.0 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_climb_tracking/mujoco.yaml b/conf/ppo/task/g1_climb_tracking/mujoco.yaml index 9697dc3fc..b7111e85a 100644 --- a/conf/ppo/task/g1_climb_tracking/mujoco.yaml +++ b/conf/ppo/task/g1_climb_tracking/mujoco.yaml @@ -78,3 +78,7 @@ reward: std_body_ang_vel: 3.14 std_joint_pos: 0.2 std_joint_vel: 1.0 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_flip_tracking/motrix.yaml b/conf/ppo/task/g1_flip_tracking/motrix.yaml index 1d17d213c..3435bd57a 100644 --- a/conf/ppo/task/g1_flip_tracking/motrix.yaml +++ b/conf/ppo/task/g1_flip_tracking/motrix.yaml @@ -32,3 +32,7 @@ reward: std_body_ang_vel: 3.14 std_joint_pos: 0.2 std_joint_vel: 1.0 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_flip_tracking/mujoco.yaml b/conf/ppo/task/g1_flip_tracking/mujoco.yaml index b9d736e2a..1edd403da 100644 --- a/conf/ppo/task/g1_flip_tracking/mujoco.yaml +++ b/conf/ppo/task/g1_flip_tracking/mujoco.yaml @@ -78,3 +78,7 @@ reward: std_body_ang_vel: 3.14 std_joint_pos: 0.2 std_joint_vel: 1.0 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_motion_tracking/mujoco.yaml b/conf/ppo/task/g1_motion_tracking/mujoco.yaml index f257cd9e6..7301e4b68 100644 --- a/conf/ppo/task/g1_motion_tracking/mujoco.yaml +++ b/conf/ppo/task/g1_motion_tracking/mujoco.yaml @@ -32,3 +32,7 @@ reward: std_body_ang_vel: 3.14 std_joint_pos: 0.2 std_joint_vel: 1.0 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_walk_flat/base.yaml b/conf/ppo/task/g1_walk_flat/base.yaml deleted file mode 100644 index 962e1ecf5..000000000 --- a/conf/ppo/task/g1_walk_flat/base.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ -# Shared cross-backend contract for g1_walk_flat. See AGENTS.md (Sim2Sim). -algo: - empirical_normalization: false - obs_groups: - actor: - - actor - policy: - actor_hidden_dims: [512, 256, 128] - critic_hidden_dims: [512, 256, 128] -env: - control_config: - action_scale: 0.25 diff --git a/conf/ppo/task/g1_walk_flat/motrix.yaml b/conf/ppo/task/g1_walk_flat/motrix.yaml index fddd5f51b..88c971936 100644 --- a/conf/ppo/task/g1_walk_flat/motrix.yaml +++ b/conf/ppo/task/g1_walk_flat/motrix.yaml @@ -1,10 +1,7 @@ # @package _global_ -# Inherits base.yaml, then overrides contract fields for Motrix-specific tuning +# Standalone Motrix owner config: carries the shared contract inline, then +# overrides contract fields for Motrix-specific tuning # (intentionally non-transferable from MuJoCo; drop overrides to restore parity). -defaults: - - /task/g1_walk_flat/base - - _self_ - training: task_name: G1WalkFlat sim_backend: motrix @@ -18,6 +15,8 @@ algo: critic: - critic policy: + actor_hidden_dims: [512, 256, 128] + critic_hidden_dims: [512, 256, 128] init_noise_std: 0.5 algorithm: learning_rate: 3.0e-4 @@ -67,3 +66,7 @@ reward: min_forward_speed_for_gait_reward: 0.05 min_base_height: 0.5 max_tilt_deg: 35.0 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_walk_flat/mujoco.yaml b/conf/ppo/task/g1_walk_flat/mujoco.yaml index 54d670f5d..843d7b85f 100644 --- a/conf/ppo/task/g1_walk_flat/mujoco.yaml +++ b/conf/ppo/task/g1_walk_flat/mujoco.yaml @@ -1,16 +1,22 @@ # @package _global_ -# Inherits the shared contract from base.yaml; backend-specific tuning only. -defaults: - - /task/g1_walk_flat/base - - _self_ - +# Standalone MuJoCo owner config: carries the shared cross-backend contract inline +# (formerly base.yaml), plus backend-specific tuning. training: task_name: G1WalkFlat sim_backend: mujoco algo: num_envs: 2048 max_iterations: 2200 + empirical_normalization: false + obs_groups: + actor: + - actor + policy: + actor_hidden_dims: [512, 256, 128] + critic_hidden_dims: [512, 256, 128] env: + control_config: + action_scale: 0.25 curriculum: enabled: false noise_config: @@ -37,3 +43,7 @@ reward: min_base_height: 0.55 max_tilt_deg: 25.0 pose_weights: [0.01, 1.0, 5.0, 0.01, 5.0, 5.0, 0.01, 1.0, 5.0, 0.01, 5.0, 5.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0] +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_wall_flip_tracking/mujoco.yaml b/conf/ppo/task/g1_wall_flip_tracking/mujoco.yaml index 1dd234f91..e11d9b05b 100644 --- a/conf/ppo/task/g1_wall_flip_tracking/mujoco.yaml +++ b/conf/ppo/task/g1_wall_flip_tracking/mujoco.yaml @@ -78,3 +78,7 @@ reward: std_body_ang_vel: 3.14 std_joint_pos: 0.2 std_joint_vel: 1.0 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go1_joystick_flat/motrix.yaml b/conf/ppo/task/go1_joystick_flat/motrix.yaml index b699946ab..d84ba1b03 100644 --- a/conf/ppo/task/go1_joystick_flat/motrix.yaml +++ b/conf/ppo/task/go1_joystick_flat/motrix.yaml @@ -43,3 +43,7 @@ reward: swing_feet_z: 4.0 tracking_sigma: 0.25 base_height_target: 0.3 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go1_joystick_flat/mujoco.yaml b/conf/ppo/task/go1_joystick_flat/mujoco.yaml index e6edb90b6..794266ad3 100644 --- a/conf/ppo/task/go1_joystick_flat/mujoco.yaml +++ b/conf/ppo/task/go1_joystick_flat/mujoco.yaml @@ -34,3 +34,7 @@ reward: swing_feet_z: 4.0 tracking_sigma: 0.25 base_height_target: 0.3 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go1_joystick_rough/motrix.yaml b/conf/ppo/task/go1_joystick_rough/motrix.yaml index b24e9ccd6..21be8340a 100644 --- a/conf/ppo/task/go1_joystick_rough/motrix.yaml +++ b/conf/ppo/task/go1_joystick_rough/motrix.yaml @@ -115,3 +115,7 @@ reward: feet_gait_max_err: 0.2 feet_gait_velocity_threshold: 0.5 feet_gait_command_threshold: 0.1 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go1_joystick_rough/mujoco.yaml b/conf/ppo/task/go1_joystick_rough/mujoco.yaml index 05a43dd17..04a9dc179 100644 --- a/conf/ppo/task/go1_joystick_rough/mujoco.yaml +++ b/conf/ppo/task/go1_joystick_rough/mujoco.yaml @@ -116,3 +116,7 @@ reward: feet_gait_max_err: 0.2 feet_gait_velocity_threshold: 0.5 feet_gait_command_threshold: 0.1 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2_arm_manip_loco/mujoco.yaml b/conf/ppo/task/go2_arm_manip_loco/mujoco.yaml index a5b998089..162e0a256 100644 --- a/conf/ppo/task/go2_arm_manip_loco/mujoco.yaml +++ b/conf/ppo/task/go2_arm_manip_loco/mujoco.yaml @@ -106,3 +106,7 @@ env: freeze_arm_joints: false disable_ee_goal_trajectory: false fixed_ee_goal_cart: [0.15, 0.0, 0.25] +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2_joystick_flat/motrix.yaml b/conf/ppo/task/go2_joystick_flat/motrix.yaml index 682935bc5..aca13216d 100644 --- a/conf/ppo/task/go2_joystick_flat/motrix.yaml +++ b/conf/ppo/task/go2_joystick_flat/motrix.yaml @@ -35,3 +35,7 @@ reward: swing_feet_z: 4.0 tracking_sigma: 0.25 base_height_target: 0.3 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2_joystick_flat/mujoco.yaml b/conf/ppo/task/go2_joystick_flat/mujoco.yaml index d27260e44..35cf09bed 100644 --- a/conf/ppo/task/go2_joystick_flat/mujoco.yaml +++ b/conf/ppo/task/go2_joystick_flat/mujoco.yaml @@ -27,3 +27,7 @@ reward: swing_feet_z: 4.0 tracking_sigma: 0.25 base_height_target: 0.3 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2_joystick_rough/motrix.yaml b/conf/ppo/task/go2_joystick_rough/motrix.yaml index 93db22561..46992b1f0 100644 --- a/conf/ppo/task/go2_joystick_rough/motrix.yaml +++ b/conf/ppo/task/go2_joystick_rough/motrix.yaml @@ -114,3 +114,7 @@ reward: feet_gait_max_err: 0.2 feet_gait_velocity_threshold: 0.5 feet_gait_command_threshold: 0.1 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2_joystick_rough/mujoco.yaml b/conf/ppo/task/go2_joystick_rough/mujoco.yaml index 00d8cf446..ea665a5fa 100644 --- a/conf/ppo/task/go2_joystick_rough/mujoco.yaml +++ b/conf/ppo/task/go2_joystick_rough/mujoco.yaml @@ -116,3 +116,7 @@ reward: feet_gait_max_err: 0.2 feet_gait_velocity_threshold: 0.5 feet_gait_command_threshold: 0.1 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2w_joystick_flat/motrix.yaml b/conf/ppo/task/go2w_joystick_flat/motrix.yaml index a2b4256f7..0e86b63b0 100644 --- a/conf/ppo/task/go2w_joystick_flat/motrix.yaml +++ b/conf/ppo/task/go2w_joystick_flat/motrix.yaml @@ -45,3 +45,7 @@ reward: upward: 1.0 tracking_sigma: 0.25 base_height_target: 0.4 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2w_joystick_flat/mujoco.yaml b/conf/ppo/task/go2w_joystick_flat/mujoco.yaml index 311b6017a..602b95a27 100644 --- a/conf/ppo/task/go2w_joystick_flat/mujoco.yaml +++ b/conf/ppo/task/go2w_joystick_flat/mujoco.yaml @@ -44,3 +44,7 @@ reward: upward: 1.0 tracking_sigma: 0.25 base_height_target: 0.4 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2w_joystick_rough/motrix.yaml b/conf/ppo/task/go2w_joystick_rough/motrix.yaml index 77fc3dd72..d15620ddb 100644 --- a/conf/ppo/task/go2w_joystick_rough/motrix.yaml +++ b/conf/ppo/task/go2w_joystick_rough/motrix.yaml @@ -93,3 +93,7 @@ reward: tracking_sigma: 0.25 base_height_target: 0.4 only_positive_rewards: false +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2w_joystick_rough/mujoco.yaml b/conf/ppo/task/go2w_joystick_rough/mujoco.yaml index 6f29fc7fe..5bcc31038 100644 --- a/conf/ppo/task/go2w_joystick_rough/mujoco.yaml +++ b/conf/ppo/task/go2w_joystick_rough/mujoco.yaml @@ -93,3 +93,7 @@ reward: tracking_sigma: 0.25 base_height_target: 0.4 only_positive_rewards: false +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/sharpa_inhand/motrix.yaml b/conf/ppo/task/sharpa_inhand/motrix.yaml index a71dea555..91cbf1436 100644 --- a/conf/ppo/task/sharpa_inhand/motrix.yaml +++ b/conf/ppo/task/sharpa_inhand/motrix.yaml @@ -20,3 +20,7 @@ env: randomize_gravity: false randomize_gravity_direction: true randomize_pd_gains: true +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/sharpa_inhand/mujoco.yaml b/conf/ppo/task/sharpa_inhand/mujoco.yaml index 7ee973e50..638b8aaa4 100644 --- a/conf/ppo/task/sharpa_inhand/mujoco.yaml +++ b/conf/ppo/task/sharpa_inhand/mujoco.yaml @@ -99,3 +99,7 @@ env: joint_noise_scale: 0.02 contact_latency: 0.005 contact_sensor_noise: 0.01 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/sharpa_inhand_grasp/motrix.yaml b/conf/ppo/task/sharpa_inhand_grasp/motrix.yaml index 8002dfe89..86168005b 100644 --- a/conf/ppo/task/sharpa_inhand_grasp/motrix.yaml +++ b/conf/ppo/task/sharpa_inhand_grasp/motrix.yaml @@ -12,3 +12,7 @@ env: domain_rand: randomize_gravity: false randomize_gravity_direction: false +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/sharpa_inhand_grasp/mujoco.yaml b/conf/ppo/task/sharpa_inhand_grasp/mujoco.yaml index 690c9c288..871b4be0a 100644 --- a/conf/ppo/task/sharpa_inhand_grasp/mujoco.yaml +++ b/conf/ppo/task/sharpa_inhand_grasp/mujoco.yaml @@ -100,3 +100,7 @@ env: - contact_right_pinky_elastomer_force disable_tactile_ids: [] grasp_collection_target: 10000 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/stewart_balance/motrix.yaml b/conf/ppo/task/stewart_balance/motrix.yaml index 311be10be..f018225f8 100644 --- a/conf/ppo/task/stewart_balance/motrix.yaml +++ b/conf/ppo/task/stewart_balance/motrix.yaml @@ -46,3 +46,7 @@ reward: progress: 0.6 still: 3.0 fall_penalty: -6.0 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/stewart_balance/mujoco.yaml b/conf/ppo/task/stewart_balance/mujoco.yaml index 4d3acc67f..f4facc798 100644 --- a/conf/ppo/task/stewart_balance/mujoco.yaml +++ b/conf/ppo/task/stewart_balance/mujoco.yaml @@ -11,3 +11,7 @@ defaults: training: task_name: StewartBalance sim_backend: mujoco +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/docs/sphinx/source/en/3-deployment/2-sim_to_sim/0-index.md b/docs/sphinx/source/en/3-deployment/2-sim_to_sim/0-index.md index f0f725fb0..09fbc9c19 100644 --- a/docs/sphinx/source/en/3-deployment/2-sim_to_sim/0-index.md +++ b/docs/sphinx/source/en/3-deployment/2-sim_to_sim/0-index.md @@ -42,6 +42,12 @@ Understand renderer and snapshot capability differences. Document unsupported backend features with evidence. ::: +:::{grid-item-card} Config guard +:link: 7-config_guard +:link-type: doc +Auto-check policy contract compatibility on cross-backend replay. +::: + :::: ```{toctree} @@ -53,4 +59,5 @@ Document unsupported backend features with evidence. 4-reward_parity 5-playback_and_snapshot_differences 6-capability_gaps +7-config_guard ``` diff --git a/docs/sphinx/source/en/3-deployment/2-sim_to_sim/7-config_guard.md b/docs/sphinx/source/en/3-deployment/2-sim_to_sim/7-config_guard.md new file mode 100644 index 000000000..3a02f4e97 --- /dev/null +++ b/docs/sphinx/source/en/3-deployment/2-sim_to_sim/7-config_guard.md @@ -0,0 +1,47 @@ +# Cross-Backend Config Guard + +When replaying across backends (train on one backend, `eval` on another), UniLab automatically checks whether the target backend config is compatible with the policy contract captured at training time, so a mismatched config can't silently load a broken policy. This happens with no manual steps. + +## An example that replays successfully + +For `go2_joystick_flat`, the MuJoCo and Motrix owners agree on every guarded field, so a cross-backend replay passes directly: + +```bash +# 1) Train in MuJoCo, producing a checkpoint +uv run train --algo ppo --task go2_joystick_flat --sim mujoco + +# 2) Replay the same checkpoint across backends in Motrix — the guard passes, playback runs +uv run eval --algo ppo --task go2_joystick_flat --sim motrix --load-run -1 +``` + +## How the guard chain works + +1. **At train time**: `ExperimentTracker` snapshots the contract fields that define policy I/O into `contract_snapshot` in `run_config.json` (the checkpoint format is untouched, so historical checkpoints stay compatible). +2. **At replay time**: `eval` loads the **target backend** owner config selected by `--sim` (e.g. `conf/ppo/task/go2_joystick_flat/motrix.yaml`) and injects `training.play_only=true`. +3. **Before env creation**: the five play entrypoints (rsl_rl / appo / offpolicy / mlx_ppo / him_ppo) call `resolve_sim2sim_config`, comparing the target config against the source run's contract snapshot field by field. +4. **At weight load**: `policy_load_dim_guard` wraps checkpoint loading, re-raising cryptic tensor shape-mismatch errors as a clear sim2sim diagnostic. + +## What the guard covers + +Fields are classified by dotted path into three tiers (see `src/unilab/training/sim2sim.py`): + +| Tier | Behavior | Fields | +|---|---|---| +| **DENYLIST** | Mismatch → `CrossBackendIncompatibleError`, aborts | `algo.obs_groups`, `env.control_config.action_scale`, `algo.policy.actor_hidden_dims` / `critic_hidden_dims`, `algo.empirical_normalization` / `algo.obs_normalization`, `env.sampling_mode` | +| **WARNING_LIST** | Prints a warning, continues | `reward.*`, `env.control_config.simulate_action_latency`, `env.ctrl_dt` | +| **ALLOWLIST** | Free to override, not checked | `training.sim_backend`, `env.scene`, `training.play_steps`, `env.domain_rand`, `env.noise_config`, `env.commands.vel_limit` | + +## When DENYLIST fields differ + +If the target backend's DENYLIST fields differ from training (e.g. a task whose two owners use different `action_scale` values), the guard aborts before env creation and lists the diverging fields. Two ways to resolve: + +- **Align the contract** (recommended): make the target owner's DENYLIST fields match the training backend, then replay. +- **Force through** (at your own risk): `uv run eval ... training.sim2sim_strict=false` downgrades DENYLIST mismatches to warnings. + +> Legacy runs: if `run_config.json` has no `contract_snapshot` (older training), the guard skips with a warning instead of breaking your workflow. + +## See also + +- {doc}`1-backend_swap` +- {doc}`4-reward_parity` +- {doc}`../../4-developer_guide/9-sim2sim_contract_status` diff --git a/docs/sphinx/source/zh_CN/3-deployment/2-sim_to_sim/0-index.md b/docs/sphinx/source/zh_CN/3-deployment/2-sim_to_sim/0-index.md index 24ba0ed5b..43c65be22 100644 --- a/docs/sphinx/source/zh_CN/3-deployment/2-sim_to_sim/0-index.md +++ b/docs/sphinx/source/zh_CN/3-deployment/2-sim_to_sim/0-index.md @@ -41,6 +41,12 @@ 用证据记录不受支持的后端特性。 ::: +:::{grid-item-card} 配置守卫 +:link: 7-config_guard +:link-type: doc +跨后端回放时自动校验策略契约兼容性。 +::: + :::: ```{toctree} @@ -52,4 +58,5 @@ 4-reward_parity 5-playback_and_snapshot_differences 6-capability_gaps +7-config_guard ``` diff --git a/docs/sphinx/source/zh_CN/3-deployment/2-sim_to_sim/7-config_guard.md b/docs/sphinx/source/zh_CN/3-deployment/2-sim_to_sim/7-config_guard.md new file mode 100644 index 000000000..c01a631fc --- /dev/null +++ b/docs/sphinx/source/zh_CN/3-deployment/2-sim_to_sim/7-config_guard.md @@ -0,0 +1,47 @@ +# 跨后端配置守卫 + +跨后端回放（在一个后端训练、到另一个后端 `eval`）时，UniLab 会自动校验目标后端配置与训练时的策略契约是否兼容，避免用错配置静默加载出行为异常的策略。整个过程无需手动干预。 + +## 一个可成功回放的例子 + +`go2_joystick_flat` 的 MuJoCo 与 Motrix owner 在守卫字段上完全一致，因此跨后端回放可以直接通过： + +```bash +# 1) MuJoCo 训练，产生 checkpoint +uv run train --algo ppo --task go2_joystick_flat --sim mujoco + +# 2) Motrix 跨后端回放同一个 checkpoint —— 守卫校验通过，正常播放 +uv run eval --algo ppo --task go2_joystick_flat --sim motrix --load-run -1 +``` + +## 生效链路 + +1. **训练时**：`ExperimentTracker` 把决定策略 I/O 的契约字段快照进 `run_config.json` 的 `contract_snapshot`（不改动 checkpoint 格式，历史 checkpoint 天然兼容）。 +2. **回放时**：`eval` 读取 `--sim` 指定的**目标后端** owner 配置（如 `conf/ppo/task/go2_joystick_flat/motrix.yaml`），并注入 `training.play_only=true`。 +3. **建 env 前**：五个 play 入口（rsl_rl / appo / offpolicy / mlx_ppo / him_ppo）调用 `resolve_sim2sim_config`，把目标配置与源 run 的契约快照逐字段比对。 +4. **加载权重时**：`policy_load_dim_guard` 包裹 checkpoint 加载，把底层 tensor 维度不匹配的晦涩报错重抛为清晰的 sim2sim 诊断。 + +## 守卫的字段 + +守卫按 dotted path 分三档（定义见 `src/unilab/training/sim2sim.py`）： + +| 档位 | 行为 | 字段 | +|---|---|---| +| **DENYLIST** | 差异即 `CrossBackendIncompatibleError`，中断 | `algo.obs_groups`、`env.control_config.action_scale`、`algo.policy.actor_hidden_dims` / `critic_hidden_dims`、`algo.empirical_normalization` / `algo.obs_normalization`、`env.sampling_mode` | +| **WARNING_LIST** | 仅打印 warning，继续 | `reward.*`、`env.control_config.simulate_action_latency`、`env.ctrl_dt` | +| **ALLOWLIST** | 自由覆盖，不检查 | `training.sim_backend`、`env.scene`、`training.play_steps`、`env.domain_rand`、`env.noise_config`、`env.commands.vel_limit` | + +## 当 DENYLIST 字段不一致时 + +若目标后端的 DENYLIST 字段与训练时不同（例如某 task 的两端 owner 在 `action_scale` 上取值不同），守卫会在建 env 前中断并列出差异字段。处理方式二选一： + +- **对齐契约**（推荐）：让目标后端 owner 的 DENYLIST 字段与训练后端一致，再回放。 +- **强制放行**（自担风险）：`uv run eval ... training.sim2sim_strict=false`，把 DENYLIST 差异降级为 warning。 + +> 兼容旧 run：若 `run_config.json` 没有 `contract_snapshot`（早期训练），守卫自动跳过并打印 warning，不会中断现有工作流。 + +## 另请参阅 + +- {doc}`1-backend_swap` +- {doc}`4-reward_parity` +- {doc}`../../4-developer_guide/9-sim2sim_contract_status` diff --git a/docs/sphinx/source/zh_CN/4-developer_guide/9-sim2sim_contract_status.md b/docs/sphinx/source/zh_CN/4-developer_guide/9-sim2sim_contract_status.md index 9505ff003..18459b02d 100644 --- a/docs/sphinx/source/zh_CN/4-developer_guide/9-sim2sim_contract_status.md +++ b/docs/sphinx/source/zh_CN/4-developer_guide/9-sim2sim_contract_status.md @@ -10,7 +10,7 @@ uv run scripts/audit_sim2sim_contracts.py ``` -该脚本只读，对每个 task 用 hydra `compose` 得到有效配置（展开 `defaults` / `base.yaml` / +该脚本只读，对每个 task 用 hydra `compose` 得到有效配置（展开 `defaults` / `# @package _global_`），再按守卫同一套归一化逐字段比对，输出本页相同的判定。 判定分三档： @@ -64,11 +64,11 @@ uv run scripts/audit_sim2sim_contracts.py | 字段 | 是否可仅改 YAML | 说明 | |---|---|---| -| `obs_groups` | 可（多数情况） | 命名差异统一进 `base.yaml`，actor 部署不变 | +| `obs_groups` | 可（多数情况） | 命名差异在两后端 owner 中统一，actor 部署不变 | | `sampling_mode` | 可（取决于 task） | 两后端运行时已同值时只需补齐显式声明 | | `action_scale` | **不可** | 改值即改训练动力学，必须 owner 决策 + 重训 | | `empirical_normalization` | **不可** | 改变网络结构，必须重训 | -试点示例：`conf/ppo/task/g1_walk_flat/{base,mujoco,motrix}.yaml`。`base.yaml` 承载共享契约， -`mujoco.yaml` 直接继承，`motrix.yaml` 为单后端调参 override 了若干契约字段——这种 override +试点示例：`conf/ppo/task/g1_walk_flat/{mujoco,motrix}.yaml`。每个后端 owner 自包含完整契约， +`motrix.yaml` 为单后端调参 override 了若干契约字段——这种 override 即令该 task 在该后端不可 sim2sim 迁移，去掉 override 即可恢复。 diff --git a/tests/envs/test_stewart.py b/tests/envs/test_stewart.py index 24f3ba640..a6a5e2b78 100644 --- a/tests/envs/test_stewart.py +++ b/tests/envs/test_stewart.py @@ -19,6 +19,7 @@ def _make_env(num_envs: int = 2): + pytest.importorskip("motrixsim", reason="motrixsim not installed") ensure_registries() return registry.make("StewartBalance", sim_backend="motrix", num_envs=num_envs) diff --git a/tests/training/test_sim2sim_resolver.py b/tests/training/test_sim2sim_resolver.py index 733160105..ef31e4137 100644 --- a/tests/training/test_sim2sim_resolver.py +++ b/tests/training/test_sim2sim_resolver.py @@ -329,7 +329,7 @@ def _compose_task(task: str) -> Any: def test_g1_walk_flat_mujoco_inherits_base_contract(): - # The MuJoCo owner must inherit the shared contract from base.yaml verbatim. + # The MuJoCo owner carries the full contract in its standalone owner config. mujoco = _compose_task("g1_walk_flat/mujoco") assert OmegaConf.select(mujoco, "env.control_config.action_scale") == 0.25 assert OmegaConf.select(mujoco, "algo.empirical_normalization") is False