From e3d629d4460723dd95e088402e544678622c82f6 Mon Sep 17 00:00:00 2001 From: LeeLeno <1846301824@qq.com> Date: Wed, 1 Jul 2026 13:34:33 +0800 Subject: [PATCH 1/3] =?UTF-8?q?feat(sim2sim):=20=E5=8F=8C=E5=90=8E?= =?UTF-8?q?=E7=AB=AF=20base.yaml=20=E9=87=8D=E6=9E=84=20+=20train/play=20?= =?UTF-8?q?=E9=85=8D=E7=BD=AE=E8=A7=A3=E8=80=A6?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 为所有双后端 task 提取共享 base.yaml,mujoco/motrix 训练配置继承之(compose 有效值不变) - 为 29 个双后端 task 新增 _play.yaml:继承同后端训练契约,仅叠加 play_only 与渲染层,实现 train/play 解耦 - eval 入口在 _play 存在时自动路由,train 不变;显式 --profile play 优先 - 跨后端 play 由 resolve_sim2sim_config 只读校验,差异显式报错,sim2sim_strict=false 可强制 --- README.md | 2 +- README_zh.md | 2 +- conf/appo/task/allegro_inhand/base.yaml | 73 +++++++++ conf/appo/task/allegro_inhand/motrix.yaml | 61 +------ .../appo/task/allegro_inhand/motrix_play.yaml | 8 + conf/appo/task/allegro_inhand/mujoco.yaml | 65 +------- .../appo/task/allegro_inhand/mujoco_play.yaml | 8 + conf/appo/task/g1_climb_tracking/base.yaml | 71 +++++++++ conf/appo/task/g1_climb_tracking/motrix.yaml | 71 +-------- .../task/g1_climb_tracking/motrix_play.yaml | 8 + conf/appo/task/g1_climb_tracking/mujoco.yaml | 70 +------- .../task/g1_climb_tracking/mujoco_play.yaml | 8 + conf/appo/task/g1_flip_tracking/base.yaml | 92 +++++++++++ conf/appo/task/g1_flip_tracking/motrix.yaml | 92 +---------- .../task/g1_flip_tracking/motrix_play.yaml | 8 + conf/appo/task/g1_flip_tracking/mujoco.yaml | 91 +---------- .../task/g1_flip_tracking/mujoco_play.yaml | 8 + conf/appo/task/g1_motion_tracking/base.yaml | 29 ++++ conf/appo/task/g1_motion_tracking/motrix.yaml | 29 +--- .../task/g1_motion_tracking/motrix_play.yaml | 8 + conf/appo/task/g1_motion_tracking/mujoco.yaml | 34 +--- .../task/g1_motion_tracking/mujoco_play.yaml | 8 + .../appo/task/g1_wall_flip_tracking/base.yaml | 50 ++++++ .../task/g1_wall_flip_tracking/motrix.yaml | 38 +---- .../g1_wall_flip_tracking/motrix_play.yaml | 8 + .../task/g1_wall_flip_tracking/mujoco.yaml | 105 ++++-------- .../g1_wall_flip_tracking/mujoco_play.yaml | 8 + conf/appo/task/go1_joystick_flat/base.yaml | 16 ++ conf/appo/task/go1_joystick_flat/motrix.yaml | 23 +-- .../task/go1_joystick_flat/motrix_play.yaml | 8 + conf/appo/task/go1_joystick_flat/mujoco.yaml | 21 +-- .../task/go1_joystick_flat/mujoco_play.yaml | 8 + conf/appo/task/go2_joystick_flat/base.yaml | 19 +++ conf/appo/task/go2_joystick_flat/motrix.yaml | 18 +-- .../task/go2_joystick_flat/motrix_play.yaml | 8 + conf/appo/task/go2_joystick_flat/mujoco.yaml | 19 +-- .../task/go2_joystick_flat/mujoco_play.yaml | 8 + conf/appo/task/sharpa_inhand/motrix_play.yaml | 8 + conf/appo/task/sharpa_inhand/mujoco_play.yaml | 8 + .../sac/g1_motion_tracking/motrix_play.yaml | 8 + .../sac/g1_motion_tracking/mujoco_play.yaml | 8 + .../offpolicy/task/sac/g1_walk_flat/base.yaml | 75 +++++++++ .../task/sac/g1_walk_flat/motrix.yaml | 31 +--- .../task/sac/g1_walk_flat/motrix_play.yaml | 8 + .../task/sac/g1_walk_flat/mujoco.yaml | 52 +----- .../task/sac/g1_walk_flat/mujoco_play.yaml | 8 + .../task/sac/g1_walk_rough/base.yaml | 75 +++++++++ .../task/sac/g1_walk_rough/motrix.yaml | 31 +--- .../task/sac/g1_walk_rough/motrix_play.yaml | 8 + .../task/sac/g1_walk_rough/mujoco.yaml | 52 +----- .../task/sac/g1_walk_rough/mujoco_play.yaml | 8 + conf/ppo/task/allegro_inhand/base.yaml | 58 +++++++ conf/ppo/task/allegro_inhand/motrix.yaml | 52 +----- conf/ppo/task/allegro_inhand/motrix_play.yaml | 13 ++ conf/ppo/task/allegro_inhand/mujoco.yaml | 57 +------ conf/ppo/task/allegro_inhand/mujoco_play.yaml | 13 ++ .../allegro_inhand_grasp/motrix_play.yaml | 13 ++ .../allegro_inhand_grasp/mujoco_play.yaml | 13 ++ conf/ppo/task/g1_box_tracking/base.yaml | 39 +++++ conf/ppo/task/g1_box_tracking/motrix.yaml | 41 ++--- .../ppo/task/g1_box_tracking/motrix_play.yaml | 8 + conf/ppo/task/g1_box_tracking/mujoco.yaml | 45 +----- .../ppo/task/g1_box_tracking/mujoco_play.yaml | 13 ++ conf/ppo/task/g1_climb_tracking/base.yaml | 80 ++++++++++ conf/ppo/task/g1_climb_tracking/motrix.yaml | 80 +--------- .../task/g1_climb_tracking/motrix_play.yaml | 13 ++ conf/ppo/task/g1_climb_tracking/mujoco.yaml | 79 +--------- .../task/g1_climb_tracking/mujoco_play.yaml | 13 ++ conf/ppo/task/g1_flip_tracking/base.yaml | 34 ++++ conf/ppo/task/g1_flip_tracking/motrix.yaml | 26 +-- .../task/g1_flip_tracking/motrix_play.yaml | 13 ++ conf/ppo/task/g1_flip_tracking/mujoco.yaml | 95 +++++------ .../task/g1_flip_tracking/mujoco_play.yaml | 13 ++ conf/ppo/task/g1_motion_tracking/base.yaml | 34 ++++ conf/ppo/task/g1_motion_tracking/motrix.yaml | 43 ++--- .../task/g1_motion_tracking/motrix_play.yaml | 8 + conf/ppo/task/g1_motion_tracking/mujoco.yaml | 33 +--- .../task/g1_motion_tracking/mujoco_play.yaml | 13 ++ conf/ppo/task/g1_walk_flat/motrix_play.yaml | 13 ++ conf/ppo/task/g1_walk_flat/mujoco_play.yaml | 13 ++ conf/ppo/task/g1_wall_flip_tracking/base.yaml | 80 ++++++++++ .../task/g1_wall_flip_tracking/motrix.yaml | 91 ++--------- .../g1_wall_flip_tracking/motrix_play.yaml | 8 + .../task/g1_wall_flip_tracking/mujoco.yaml | 79 +--------- .../g1_wall_flip_tracking/mujoco_play.yaml | 13 ++ conf/ppo/task/go1_joystick_flat/base.yaml | 32 ++++ conf/ppo/task/go1_joystick_flat/motrix.yaml | 45 ++---- .../task/go1_joystick_flat/motrix_play.yaml | 12 ++ conf/ppo/task/go1_joystick_flat/mujoco.yaml | 32 +--- .../task/go1_joystick_flat/mujoco_play.yaml | 12 ++ conf/ppo/task/go1_joystick_rough/base.yaml | 131 +++++++++++++++ conf/ppo/task/go1_joystick_rough/motrix.yaml | 113 +------------ .../task/go1_joystick_rough/motrix_play.yaml | 13 ++ conf/ppo/task/go1_joystick_rough/mujoco.yaml | 115 +------------- .../task/go1_joystick_rough/mujoco_play.yaml | 13 ++ conf/ppo/task/go2_arm_manip_loco/base.yaml | 149 ++++++++++++++++++ conf/ppo/task/go2_arm_manip_loco/motrix.yaml | 115 ++------------ .../task/go2_arm_manip_loco/motrix_play.yaml | 8 + conf/ppo/task/go2_arm_manip_loco/mujoco.yaml | 107 +------------ .../task/go2_arm_manip_loco/mujoco_play.yaml | 13 ++ conf/ppo/task/go2_joystick_flat/base.yaml | 28 ++++ conf/ppo/task/go2_joystick_flat/motrix.yaml | 37 ++--- .../task/go2_joystick_flat/motrix_play.yaml | 12 ++ conf/ppo/task/go2_joystick_flat/mujoco.yaml | 28 +--- .../task/go2_joystick_flat/mujoco_play.yaml | 12 ++ conf/ppo/task/go2_joystick_rough/base.yaml | 131 +++++++++++++++ conf/ppo/task/go2_joystick_rough/motrix.yaml | 110 +------------ .../task/go2_joystick_rough/motrix_play.yaml | 13 ++ conf/ppo/task/go2_joystick_rough/mujoco.yaml | 115 +------------- .../task/go2_joystick_rough/mujoco_play.yaml | 13 ++ conf/ppo/task/go2w_joystick_flat/base.yaml | 49 ++++++ conf/ppo/task/go2w_joystick_flat/motrix.yaml | 45 +----- .../task/go2w_joystick_flat/motrix_play.yaml | 13 ++ conf/ppo/task/go2w_joystick_flat/mujoco.yaml | 45 +----- .../task/go2w_joystick_flat/mujoco_play.yaml | 13 ++ conf/ppo/task/go2w_joystick_rough/base.yaml | 113 +++++++++++++ conf/ppo/task/go2w_joystick_rough/motrix.yaml | 88 +---------- .../task/go2w_joystick_rough/motrix_play.yaml | 13 ++ conf/ppo/task/go2w_joystick_rough/mujoco.yaml | 94 +---------- .../task/go2w_joystick_rough/mujoco_play.yaml | 13 ++ conf/ppo/task/sharpa_inhand/motrix_play.yaml | 13 ++ conf/ppo/task/sharpa_inhand/mujoco_play.yaml | 13 ++ .../task/sharpa_inhand_grasp/motrix_play.yaml | 13 ++ .../task/sharpa_inhand_grasp/mujoco_play.yaml | 13 ++ .../ppo/task/stewart_balance/motrix_play.yaml | 13 ++ .../ppo/task/stewart_balance/mujoco_play.yaml | 13 ++ src/unilab/cli.py | 9 ++ tests/envs/test_stewart.py | 1 + 128 files changed, 2409 insertions(+), 2396 deletions(-) create mode 100644 conf/appo/task/allegro_inhand/base.yaml create mode 100644 conf/appo/task/allegro_inhand/motrix_play.yaml create mode 100644 conf/appo/task/allegro_inhand/mujoco_play.yaml create mode 100644 conf/appo/task/g1_climb_tracking/base.yaml create mode 100644 conf/appo/task/g1_climb_tracking/motrix_play.yaml create mode 100644 conf/appo/task/g1_climb_tracking/mujoco_play.yaml create mode 100644 conf/appo/task/g1_flip_tracking/base.yaml create mode 100644 conf/appo/task/g1_flip_tracking/motrix_play.yaml create mode 100644 conf/appo/task/g1_flip_tracking/mujoco_play.yaml create mode 100644 conf/appo/task/g1_motion_tracking/base.yaml create mode 100644 conf/appo/task/g1_motion_tracking/motrix_play.yaml create mode 100644 conf/appo/task/g1_motion_tracking/mujoco_play.yaml create mode 100644 conf/appo/task/g1_wall_flip_tracking/base.yaml create mode 100644 conf/appo/task/g1_wall_flip_tracking/motrix_play.yaml create mode 100644 conf/appo/task/g1_wall_flip_tracking/mujoco_play.yaml create mode 100644 conf/appo/task/go1_joystick_flat/base.yaml create mode 100644 conf/appo/task/go1_joystick_flat/motrix_play.yaml create mode 100644 conf/appo/task/go1_joystick_flat/mujoco_play.yaml create mode 100644 conf/appo/task/go2_joystick_flat/base.yaml create mode 100644 conf/appo/task/go2_joystick_flat/motrix_play.yaml create mode 100644 conf/appo/task/go2_joystick_flat/mujoco_play.yaml create mode 100644 conf/appo/task/sharpa_inhand/motrix_play.yaml create mode 100644 conf/appo/task/sharpa_inhand/mujoco_play.yaml create mode 100644 conf/offpolicy/task/sac/g1_motion_tracking/motrix_play.yaml create mode 100644 conf/offpolicy/task/sac/g1_motion_tracking/mujoco_play.yaml create mode 100644 conf/offpolicy/task/sac/g1_walk_flat/base.yaml create mode 100644 conf/offpolicy/task/sac/g1_walk_flat/motrix_play.yaml create mode 100644 conf/offpolicy/task/sac/g1_walk_flat/mujoco_play.yaml create mode 100644 conf/offpolicy/task/sac/g1_walk_rough/base.yaml create mode 100644 conf/offpolicy/task/sac/g1_walk_rough/motrix_play.yaml create mode 100644 conf/offpolicy/task/sac/g1_walk_rough/mujoco_play.yaml create mode 100644 conf/ppo/task/allegro_inhand/base.yaml create mode 100644 conf/ppo/task/allegro_inhand/motrix_play.yaml create mode 100644 conf/ppo/task/allegro_inhand/mujoco_play.yaml create mode 100644 conf/ppo/task/allegro_inhand_grasp/motrix_play.yaml create mode 100644 conf/ppo/task/allegro_inhand_grasp/mujoco_play.yaml create mode 100644 conf/ppo/task/g1_box_tracking/base.yaml create mode 100644 conf/ppo/task/g1_box_tracking/motrix_play.yaml create mode 100644 conf/ppo/task/g1_box_tracking/mujoco_play.yaml create mode 100644 conf/ppo/task/g1_climb_tracking/base.yaml create mode 100644 conf/ppo/task/g1_climb_tracking/motrix_play.yaml create mode 100644 conf/ppo/task/g1_climb_tracking/mujoco_play.yaml create mode 100644 conf/ppo/task/g1_flip_tracking/base.yaml create mode 100644 conf/ppo/task/g1_flip_tracking/motrix_play.yaml create mode 100644 conf/ppo/task/g1_flip_tracking/mujoco_play.yaml create mode 100644 conf/ppo/task/g1_motion_tracking/base.yaml create mode 100644 conf/ppo/task/g1_motion_tracking/motrix_play.yaml create mode 100644 conf/ppo/task/g1_motion_tracking/mujoco_play.yaml create mode 100644 conf/ppo/task/g1_walk_flat/motrix_play.yaml create mode 100644 conf/ppo/task/g1_walk_flat/mujoco_play.yaml create mode 100644 conf/ppo/task/g1_wall_flip_tracking/base.yaml create mode 100644 conf/ppo/task/g1_wall_flip_tracking/motrix_play.yaml create mode 100644 conf/ppo/task/g1_wall_flip_tracking/mujoco_play.yaml create mode 100644 conf/ppo/task/go1_joystick_flat/base.yaml create mode 100644 conf/ppo/task/go1_joystick_flat/motrix_play.yaml create mode 100644 conf/ppo/task/go1_joystick_flat/mujoco_play.yaml create mode 100644 conf/ppo/task/go1_joystick_rough/base.yaml create mode 100644 conf/ppo/task/go1_joystick_rough/motrix_play.yaml create mode 100644 conf/ppo/task/go1_joystick_rough/mujoco_play.yaml create mode 100644 conf/ppo/task/go2_arm_manip_loco/base.yaml create mode 100644 conf/ppo/task/go2_arm_manip_loco/motrix_play.yaml create mode 100644 conf/ppo/task/go2_arm_manip_loco/mujoco_play.yaml create mode 100644 conf/ppo/task/go2_joystick_flat/base.yaml create mode 100644 conf/ppo/task/go2_joystick_flat/motrix_play.yaml create mode 100644 conf/ppo/task/go2_joystick_flat/mujoco_play.yaml create mode 100644 conf/ppo/task/go2_joystick_rough/base.yaml create mode 100644 conf/ppo/task/go2_joystick_rough/motrix_play.yaml create mode 100644 conf/ppo/task/go2_joystick_rough/mujoco_play.yaml create mode 100644 conf/ppo/task/go2w_joystick_flat/base.yaml create mode 100644 conf/ppo/task/go2w_joystick_flat/motrix_play.yaml create mode 100644 conf/ppo/task/go2w_joystick_flat/mujoco_play.yaml create mode 100644 conf/ppo/task/go2w_joystick_rough/base.yaml create mode 100644 conf/ppo/task/go2w_joystick_rough/motrix_play.yaml create mode 100644 conf/ppo/task/go2w_joystick_rough/mujoco_play.yaml create mode 100644 conf/ppo/task/sharpa_inhand/motrix_play.yaml create mode 100644 conf/ppo/task/sharpa_inhand/mujoco_play.yaml create mode 100644 conf/ppo/task/sharpa_inhand_grasp/motrix_play.yaml create mode 100644 conf/ppo/task/sharpa_inhand_grasp/mujoco_play.yaml create mode 100644 conf/ppo/task/stewart_balance/motrix_play.yaml create mode 100644 conf/ppo/task/stewart_balance/mujoco_play.yaml diff --git a/README.md b/README.md index deb33f76e..b3455cef8 100644 --- a/README.md +++ b/README.md @@ -132,7 +132,7 @@ uv run eval --algo appo --task go2_joystick_flat --sim motrix --load-run -1 uv run eval --algo appo --task go2_joystick_flat --sim motrix --load-run -1 --render-mode record ``` -This routes through the `go2_joystick_flat/motrix` task owner config and keeps backend selection explicit. +This routes through the `go2_joystick_flat/motrix` task owner config and keeps backend selection explicit. In eval mode, the CLI automatically prefers the decoupled play config `/_play` when it exists (train never does); pass `--profile play` to force it. On macOS / MacBook, the UniLab CLI routes Motrix interactive playback through `mxpython` when needed. Motrix defaults to interactive playback; use `--render-mode record` for headless video export or `--render-mode none` to skip playback. Detailed script-level commands are in the [Training Guide](https://unilabsim.github.io/UniLab-doc/en/2-user_guide/1-training/0-index.html). diff --git a/README_zh.md b/README_zh.md index e1e9be84f..e60b76001 100644 --- a/README_zh.md +++ b/README_zh.md @@ -134,7 +134,7 @@ uv run eval --algo appo --task go2_joystick_flat --sim motrix --load-run -1 uv run eval --algo appo --task go2_joystick_flat --sim motrix --load-run -1 --render-mode record ``` -这会路由到 `go2_joystick_flat/motrix` 任务 owner 配置，并保持后端选择显式化。 +这会路由到 `go2_joystick_flat/motrix` 任务 owner 配置，并保持后端选择显式化。eval 模式下，CLI 会在解耦的 play 配置 `/_play` 存在时自动优先使用它（train 不会），可用 `--profile play` 强制指定。 在 macOS / MacBook 上，UniLab CLI 在需要时会通过 `mxpython` 路由 Motrix 交互式回放。Motrix 默认使用交互式回放；要导出无头视频请使用 `--render-mode record`，要跳过回放请使用 `--render-mode none`。更细的脚本级命令请参阅 [训练指南](https://unilabsim.github.io/UniLab-doc/zh_CN/2-user_guide/1-training/0-index.html)。 diff --git a/conf/appo/task/allegro_inhand/base.yaml b/conf/appo/task/allegro_inhand/base.yaml new file mode 100644 index 000000000..10a27347d --- /dev/null +++ b/conf/appo/task/allegro_inhand/base.yaml @@ -0,0 +1,73 @@ +# @package _global_ +# Shared cross-backend config for allegro_inhand (mujoco/motrix). See AGENTS.md (Sim2Sim). + +training: + play_steps: 200 + render_spacing: 0.5 + cam_distance: 0.75 + cam_lookat: + - 0.0 + - 0.0 + - 0.15 + cam_elevation: -25.0 +algo: + num_envs: 1024 + steps_per_env: 8 + max_iterations: 3000 + save_interval: 500 + algorithm: + value_loss_coef: 4.0 + entropy_coef: 0.01 + learning_rate: 0.001 + desired_kl: 0.025 + adaptive_kl_factor: 2.0 + adaptive_lr_factor: 1.5 + num_learning_epochs: 5 + num_mini_batches: 4 + clip_param: 0.2 + gamma: 0.99 + lam: 0.95 + max_grad_norm: 1.0 + use_clipped_value_loss: true + schedule: adaptive + actor: + hidden_dims: + - 512 + - 256 + - 128 + activation: elu + obs_normalization: true + distribution_cfg: + class_name: rsl_rl.modules.distribution.GaussianDistribution + init_std: 1.0 + std_type: scalar + critic: + hidden_dims: + - 512 + - 256 + - 128 + activation: elu + obs_normalization: true +reward: + scales: + rotate: 1.25 + obj_linvel: -0.3 + pose_diff: -0.3 + torque: -0.1 + work: -2.0 + drop: 0.0 + angvel_clip_min: -0.5 + angvel_clip_max: 0.5 + reset_z_threshold: 0.125 +env: + gen_grasp: false + max_episode_seconds: 20.0 + grasp_cache_path: caches/allegro_grasp_50k.npy + domain_rand: + randomize_base_mass: false + random_com: false + randomize_gravity: false + push_robots: false + joint_noise: 0.0 + ball_vel_noise: 0.0 + ball_z_offset: 0.0 diff --git a/conf/appo/task/allegro_inhand/motrix.yaml b/conf/appo/task/allegro_inhand/motrix.yaml index c027729a8..b2f67f1a7 100644 --- a/conf/appo/task/allegro_inhand/motrix.yaml +++ b/conf/appo/task/allegro_inhand/motrix.yaml @@ -1,65 +1,20 @@ # @package _global_ +defaults: + - /task/allegro_inhand/base + - _self_ + training: task_name: AllegroInhandRotation sim_backend: motrix - play_steps: 200 - render_spacing: 0.5 cam_distance: 1.5 - cam_lookat: [0.75, 0.75, 0] + cam_lookat: + - 0.75 + - 0.75 + - 0 cam_elevation: -20.0 algo: num_envs: 16384 - steps_per_env: 8 max_iterations: 201 save_interval: 100 algorithm: - value_loss_coef: 4.0 - entropy_coef: 0.01 - learning_rate: 0.001 desired_kl: 0.02 - adaptive_kl_factor: 2.0 - adaptive_lr_factor: 1.5 - num_learning_epochs: 5 - num_mini_batches: 4 - clip_param: 0.2 - gamma: 0.99 - lam: 0.95 - max_grad_norm: 1.0 - use_clipped_value_loss: true - schedule: adaptive - actor: - hidden_dims: [512, 256, 128] - activation: elu - obs_normalization: true - distribution_cfg: - class_name: rsl_rl.modules.distribution.GaussianDistribution - init_std: 1.0 - std_type: scalar - critic: - hidden_dims: [512, 256, 128] - activation: elu - obs_normalization: true -reward: - scales: - rotate: 1.25 - obj_linvel: -0.3 - pose_diff: -0.3 - torque: -0.1 - work: -2.0 - drop: 0.0 - angvel_clip_min: -0.5 - angvel_clip_max: 0.5 - reset_z_threshold: 0.125 -env: - gen_grasp: false - max_episode_seconds: 20.0 - grasp_cache_path: caches/allegro_grasp_50k.npy - # Keep only grasp/pose reset variation. All online DR terms stay disabled. - domain_rand: - randomize_base_mass: false - random_com: false - randomize_gravity: false - push_robots: false - joint_noise: 0.0 - ball_vel_noise: 0.0 - ball_z_offset: 0.0 diff --git a/conf/appo/task/allegro_inhand/motrix_play.yaml b/conf/appo/task/allegro_inhand/motrix_play.yaml new file mode 100644 index 000000000..a64bff78b --- /dev/null +++ b/conf/appo/task/allegro_inhand/motrix_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/allegro_inhand/motrix + - _self_ + +training: + play_only: true diff --git a/conf/appo/task/allegro_inhand/mujoco.yaml b/conf/appo/task/allegro_inhand/mujoco.yaml index c24a442bc..222a16aec 100644 --- a/conf/appo/task/allegro_inhand/mujoco.yaml +++ b/conf/appo/task/allegro_inhand/mujoco.yaml @@ -1,67 +1,10 @@ # @package _global_ +defaults: + - /task/allegro_inhand/base + - _self_ + training: task_name: AllegroInhandRotation sim_backend: mujoco - play_steps: 200 - render_spacing: 0.5 - cam_distance: 0.75 - cam_lookat: [0.0, 0.0, 0.15] - cam_elevation: -25.0 cam_azimuth: 45.0 replay_queue_size: 4 -algo: - num_envs: 1024 - steps_per_env: 8 - max_iterations: 3000 - save_interval: 500 - algorithm: - value_loss_coef: 4.0 - entropy_coef: 0.01 - learning_rate: 0.001 - desired_kl: 0.025 - adaptive_kl_factor: 2.0 - adaptive_lr_factor: 1.5 - num_learning_epochs: 5 - num_mini_batches: 4 - clip_param: 0.2 - gamma: 0.99 - lam: 0.95 - max_grad_norm: 1.0 - use_clipped_value_loss: true - schedule: adaptive - actor: - hidden_dims: [512, 256, 128] - activation: elu - obs_normalization: true - distribution_cfg: - class_name: rsl_rl.modules.distribution.GaussianDistribution - init_std: 1.0 - std_type: scalar - critic: - hidden_dims: [512, 256, 128] - activation: elu - obs_normalization: true -reward: - scales: - rotate: 1.25 - obj_linvel: -0.3 - pose_diff: -0.3 - torque: -0.1 - work: -2.0 - drop: 0.0 - angvel_clip_min: -0.5 - angvel_clip_max: 0.5 - reset_z_threshold: 0.125 -env: - gen_grasp: false - max_episode_seconds: 20.0 - grasp_cache_path: caches/allegro_grasp_50k.npy - # Keep only grasp/pose reset variation. All online DR terms stay disabled. - domain_rand: - randomize_base_mass: false - random_com: false - randomize_gravity: false - push_robots: false - joint_noise: 0.0 - ball_vel_noise: 0.0 - ball_z_offset: 0.0 diff --git a/conf/appo/task/allegro_inhand/mujoco_play.yaml b/conf/appo/task/allegro_inhand/mujoco_play.yaml new file mode 100644 index 000000000..99969b4c6 --- /dev/null +++ b/conf/appo/task/allegro_inhand/mujoco_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/allegro_inhand/mujoco + - _self_ + +training: + play_only: true diff --git a/conf/appo/task/g1_climb_tracking/base.yaml b/conf/appo/task/g1_climb_tracking/base.yaml new file mode 100644 index 000000000..e071bef77 --- /dev/null +++ b/conf/appo/task/g1_climb_tracking/base.yaml @@ -0,0 +1,71 @@ +# @package _global_ +# Shared cross-backend config for g1_climb_tracking (mujoco/motrix). See AGENTS.md (Sim2Sim). + +training: + play_steps: 1000 +algo: + num_envs: 1024 + max_iterations: 20000 + save_interval: 500 +env: + sampling_mode: adaptive + truncate_on_clip_end: false + sim_dt: 0.005 + control_config: + action_scale: + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + anchor_pos_z_threshold: 0.3 + ee_body_pos_z_threshold: 0.3 + terminate_on_undesired_contacts: true + noise_config: + level: 0.0 +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 2.0 + motion_body_ori: 1.5 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_ee_body_pos_z: 2.0 + motion_joint_pos: 0.5 + motion_joint_vel: 0.25 + action_rate_l2: -0.005 + joint_limit: -10.0 + undesired_contacts: -0.1 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 diff --git a/conf/appo/task/g1_climb_tracking/motrix.yaml b/conf/appo/task/g1_climb_tracking/motrix.yaml index ec5684fe2..9077f325d 100644 --- a/conf/appo/task/g1_climb_tracking/motrix.yaml +++ b/conf/appo/task/g1_climb_tracking/motrix.yaml @@ -1,71 +1,8 @@ # @package _global_ +defaults: + - /task/g1_climb_tracking/base + - _self_ + training: task_name: G1ClimbTracking sim_backend: motrix - play_steps: 1000 -algo: - num_envs: 1024 - max_iterations: 20000 - save_interval: 500 -env: - sampling_mode: adaptive - truncate_on_clip_end: false - sim_dt: 0.005 - control_config: - action_scale: - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - anchor_pos_z_threshold: 0.3 - ee_body_pos_z_threshold: 0.3 - terminate_on_undesired_contacts: true - noise_config: - level: 0.0 -reward: - scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 2.0 - motion_body_ori: 1.5 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_ee_body_pos_z: 2.0 - motion_joint_pos: 0.5 - motion_joint_vel: 0.25 - action_rate_l2: -0.005 - joint_limit: -10.0 - undesired_contacts: -0.1 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 diff --git a/conf/appo/task/g1_climb_tracking/motrix_play.yaml b/conf/appo/task/g1_climb_tracking/motrix_play.yaml new file mode 100644 index 000000000..5ec021233 --- /dev/null +++ b/conf/appo/task/g1_climb_tracking/motrix_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/g1_climb_tracking/motrix + - _self_ + +training: + play_only: true diff --git a/conf/appo/task/g1_climb_tracking/mujoco.yaml b/conf/appo/task/g1_climb_tracking/mujoco.yaml index bff7b13c8..1914cb9d2 100644 --- a/conf/appo/task/g1_climb_tracking/mujoco.yaml +++ b/conf/appo/task/g1_climb_tracking/mujoco.yaml @@ -1,71 +1,7 @@ # @package _global_ +defaults: + - /task/g1_climb_tracking/base + - _self_ training: task_name: G1ClimbTracking sim_backend: mujoco - play_steps: 1000 -algo: - num_envs: 1024 - max_iterations: 20000 - save_interval: 500 -env: - sampling_mode: adaptive - truncate_on_clip_end: false - sim_dt: 0.005 - control_config: - action_scale: - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - anchor_pos_z_threshold: 0.3 - ee_body_pos_z_threshold: 0.3 - terminate_on_undesired_contacts: true - noise_config: - level: 0.0 -reward: - scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 2.0 - motion_body_ori: 1.5 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_ee_body_pos_z: 2.0 - motion_joint_pos: 0.5 - motion_joint_vel: 0.25 - action_rate_l2: -0.005 - joint_limit: -10.0 - undesired_contacts: -0.1 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 diff --git a/conf/appo/task/g1_climb_tracking/mujoco_play.yaml b/conf/appo/task/g1_climb_tracking/mujoco_play.yaml new file mode 100644 index 000000000..92c075a73 --- /dev/null +++ b/conf/appo/task/g1_climb_tracking/mujoco_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/g1_climb_tracking/mujoco + - _self_ + +training: + play_only: true diff --git a/conf/appo/task/g1_flip_tracking/base.yaml b/conf/appo/task/g1_flip_tracking/base.yaml new file mode 100644 index 000000000..ffe0b96ba --- /dev/null +++ b/conf/appo/task/g1_flip_tracking/base.yaml @@ -0,0 +1,92 @@ +# @package _global_ +# Shared cross-backend config for g1_flip_tracking (mujoco/motrix). See AGENTS.md (Sim2Sim). + +training: + play_steps: 1000 +algo: + num_envs: 1024 + steps_per_env: 24 + max_iterations: 3500 + save_interval: 500 + algorithm: + num_learning_epochs: 10 + num_mini_batches: 8 + clip_param: 0.2 + gamma: 0.99 + lam: 0.95 + value_loss_coef: 1.0 + entropy_coef: 0.005 + learning_rate: 0.001 + max_grad_norm: 1.0 + use_clipped_value_loss: true + schedule: adaptive + desired_kl: 0.01 + adaptive_kl_factor: 2.0 + adaptive_lr_factor: 1.5 + optimizer: adam + tau: 1.0 + target_update_freq: 1 + vtrace_clip_rho: 1.0 + vtrace_clip_c: 1.0 + enable_compile: true +env: + sampling_mode: start + truncate_on_clip_end: true + sim_dt: 0.005 + control_config: + action_scale: + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + anchor_pos_z_threshold: 0.5 + ee_body_pos_z_threshold: 0.5 + terminate_on_undesired_contacts: true + noise_config: + level: 0.0 +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 2.0 + motion_body_ori: 1.5 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_ee_body_pos_z: 2.0 + motion_joint_pos: 0.0 + motion_joint_vel: 0.0 + action_rate_l2: -0.005 + joint_limit: -10.0 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 diff --git a/conf/appo/task/g1_flip_tracking/motrix.yaml b/conf/appo/task/g1_flip_tracking/motrix.yaml index 9b2fd0c57..e3c99fbb5 100644 --- a/conf/appo/task/g1_flip_tracking/motrix.yaml +++ b/conf/appo/task/g1_flip_tracking/motrix.yaml @@ -1,92 +1,8 @@ # @package _global_ +defaults: + - /task/g1_flip_tracking/base + - _self_ + training: task_name: G1FlipTracking sim_backend: motrix - play_steps: 1000 -algo: - num_envs: 1024 - steps_per_env: 24 - max_iterations: 3500 - save_interval: 500 - algorithm: - num_learning_epochs: 10 - num_mini_batches: 8 - clip_param: 0.2 - gamma: 0.99 - lam: 0.95 - value_loss_coef: 1.0 - entropy_coef: 0.005 - learning_rate: 1.0e-3 - max_grad_norm: 1.0 - use_clipped_value_loss: true - schedule: adaptive - desired_kl: 0.01 - adaptive_kl_factor: 2.0 - adaptive_lr_factor: 1.5 - optimizer: adam - tau: 1.0 - target_update_freq: 1 - vtrace_clip_rho: 1.0 - vtrace_clip_c: 1.0 - enable_compile: true -env: - sampling_mode: start - truncate_on_clip_end: true - sim_dt: 0.005 - control_config: - action_scale: - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - anchor_pos_z_threshold: 0.5 - ee_body_pos_z_threshold: 0.5 - terminate_on_undesired_contacts: true - noise_config: - level: 0.0 -reward: - scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 2.0 - motion_body_ori: 1.5 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_ee_body_pos_z: 2.0 - motion_joint_pos: 0.0 - motion_joint_vel: 0.0 - action_rate_l2: -0.005 - joint_limit: -10.0 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 diff --git a/conf/appo/task/g1_flip_tracking/motrix_play.yaml b/conf/appo/task/g1_flip_tracking/motrix_play.yaml new file mode 100644 index 000000000..00fc916f9 --- /dev/null +++ b/conf/appo/task/g1_flip_tracking/motrix_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/g1_flip_tracking/motrix + - _self_ + +training: + play_only: true diff --git a/conf/appo/task/g1_flip_tracking/mujoco.yaml b/conf/appo/task/g1_flip_tracking/mujoco.yaml index 628bdbb12..ebcfe80ee 100644 --- a/conf/appo/task/g1_flip_tracking/mujoco.yaml +++ b/conf/appo/task/g1_flip_tracking/mujoco.yaml @@ -1,92 +1,7 @@ # @package _global_ +defaults: + - /task/g1_flip_tracking/base + - _self_ training: task_name: G1FlipTracking sim_backend: mujoco - play_steps: 1000 -algo: - num_envs: 1024 - steps_per_env: 24 - max_iterations: 3500 - save_interval: 500 - algorithm: - num_learning_epochs: 10 - num_mini_batches: 8 - clip_param: 0.2 - gamma: 0.99 - lam: 0.95 - value_loss_coef: 1.0 - entropy_coef: 0.005 - learning_rate: 1.0e-3 - max_grad_norm: 1.0 - use_clipped_value_loss: true - schedule: adaptive - desired_kl: 0.01 - adaptive_kl_factor: 2.0 - adaptive_lr_factor: 1.5 - optimizer: adam - tau: 1.0 - target_update_freq: 1 - vtrace_clip_rho: 1.0 - vtrace_clip_c: 1.0 - enable_compile: true -env: - sampling_mode: start - truncate_on_clip_end: true - sim_dt: 0.005 - control_config: - action_scale: - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - anchor_pos_z_threshold: 0.5 - ee_body_pos_z_threshold: 0.5 - terminate_on_undesired_contacts: true - noise_config: - level: 0.0 -reward: - scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 2.0 - motion_body_ori: 1.5 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_ee_body_pos_z: 2.0 - motion_joint_pos: 0.0 - motion_joint_vel: 0.0 - action_rate_l2: -0.005 - joint_limit: -10.0 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 \ No newline at end of file diff --git a/conf/appo/task/g1_flip_tracking/mujoco_play.yaml b/conf/appo/task/g1_flip_tracking/mujoco_play.yaml new file mode 100644 index 000000000..fb1db6ec2 --- /dev/null +++ b/conf/appo/task/g1_flip_tracking/mujoco_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/g1_flip_tracking/mujoco + - _self_ + +training: + play_only: true diff --git a/conf/appo/task/g1_motion_tracking/base.yaml b/conf/appo/task/g1_motion_tracking/base.yaml new file mode 100644 index 000000000..e9d8ab265 --- /dev/null +++ b/conf/appo/task/g1_motion_tracking/base.yaml @@ -0,0 +1,29 @@ +# @package _global_ +# Shared cross-backend config for g1_motion_tracking (mujoco/motrix). See AGENTS.md (Sim2Sim). + +training: + play_steps: 1000 +algo: + num_envs: 1024 + max_iterations: 5000 + save_interval: 500 +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 1.0 + motion_body_ori: 1.0 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_joint_pos: 0.0 + motion_joint_vel: 0.0 + action_rate_l2: -0.1 + joint_limit: -10.0 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 diff --git a/conf/appo/task/g1_motion_tracking/motrix.yaml b/conf/appo/task/g1_motion_tracking/motrix.yaml index 567b0d745..01043ec56 100644 --- a/conf/appo/task/g1_motion_tracking/motrix.yaml +++ b/conf/appo/task/g1_motion_tracking/motrix.yaml @@ -1,29 +1,8 @@ # @package _global_ +defaults: + - /task/g1_motion_tracking/base + - _self_ + training: task_name: G1MotionTracking sim_backend: motrix - play_steps: 1000 -algo: - num_envs: 1024 - max_iterations: 5000 - save_interval: 500 -reward: - scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 1.0 - motion_body_ori: 1.0 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_joint_pos: 0.0 - motion_joint_vel: 0.0 - action_rate_l2: -0.1 - joint_limit: -10.0 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 diff --git a/conf/appo/task/g1_motion_tracking/motrix_play.yaml b/conf/appo/task/g1_motion_tracking/motrix_play.yaml new file mode 100644 index 000000000..9cebd0be8 --- /dev/null +++ b/conf/appo/task/g1_motion_tracking/motrix_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/g1_motion_tracking/motrix + - _self_ + +training: + play_only: true diff --git a/conf/appo/task/g1_motion_tracking/mujoco.yaml b/conf/appo/task/g1_motion_tracking/mujoco.yaml index 4f2dc4843..2718d83ea 100644 --- a/conf/appo/task/g1_motion_tracking/mujoco.yaml +++ b/conf/appo/task/g1_motion_tracking/mujoco.yaml @@ -1,32 +1,12 @@ # @package _global_ -training: - task_name: G1MotionTracking - sim_backend: mujoco - play_steps: 1000 +defaults: + - /task/g1_motion_tracking/base + - _self_ + algo: - num_envs: 1024 - max_iterations: 5000 - save_interval: 500 algorithm: adaptive_kl_factor: 2.0 adaptive_lr_factor: 1.5 -reward: - scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 1.0 - motion_body_ori: 1.0 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_joint_pos: 0.0 - motion_joint_vel: 0.0 - action_rate_l2: -0.1 - joint_limit: -10.0 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 +training: + task_name: G1MotionTracking + sim_backend: mujoco diff --git a/conf/appo/task/g1_motion_tracking/mujoco_play.yaml b/conf/appo/task/g1_motion_tracking/mujoco_play.yaml new file mode 100644 index 000000000..a6212d376 --- /dev/null +++ b/conf/appo/task/g1_motion_tracking/mujoco_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/g1_motion_tracking/mujoco + - _self_ + +training: + play_only: true diff --git a/conf/appo/task/g1_wall_flip_tracking/base.yaml b/conf/appo/task/g1_wall_flip_tracking/base.yaml new file mode 100644 index 000000000..c0bb67912 --- /dev/null +++ b/conf/appo/task/g1_wall_flip_tracking/base.yaml @@ -0,0 +1,50 @@ +# @package _global_ +# Shared cross-backend config for g1_wall_flip_tracking (mujoco/motrix). See AGENTS.md (Sim2Sim). + +training: + play_steps: 1000 +algo: + num_envs: 1024 + max_iterations: 7000 + save_interval: 500 + algorithm: + num_learning_epochs: 6 + num_mini_batches: 8 + clip_param: 0.2 + gamma: 0.99 + lam: 0.95 + value_loss_coef: 1.0 + entropy_coef: 0.005 + learning_rate: 0.001 + max_grad_norm: 1.0 + use_clipped_value_loss: true + schedule: adaptive + desired_kl: 0.008 + adaptive_kl_factor: 2.0 + adaptive_lr_factor: 1.5 + optimizer: adam + tau: 1.0 + target_update_freq: 1 + vtrace_clip_rho: 1.0 + vtrace_clip_c: 1.0 + enable_compile: true +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 2.0 + motion_body_ori: 1.5 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_joint_pos: 0.0 + motion_joint_vel: 0.0 + action_rate_l2: -0.005 + joint_limit: -10.0 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 diff --git a/conf/appo/task/g1_wall_flip_tracking/motrix.yaml b/conf/appo/task/g1_wall_flip_tracking/motrix.yaml index 83cca7b39..495c7b876 100644 --- a/conf/appo/task/g1_wall_flip_tracking/motrix.yaml +++ b/conf/appo/task/g1_wall_flip_tracking/motrix.yaml @@ -1,50 +1,20 @@ # @package _global_ +defaults: + - /task/g1_wall_flip_tracking/base + - _self_ + training: task_name: G1WallFlipTracking sim_backend: motrix - play_steps: 1000 algo: - num_envs: 1024 max_iterations: 5000 - save_interval: 500 algorithm: num_learning_epochs: 5 num_mini_batches: 4 - clip_param: 0.2 - gamma: 0.99 - lam: 0.95 - value_loss_coef: 1.0 entropy_coef: 0.01 - learning_rate: 1.0e-3 - max_grad_norm: 1.0 - use_clipped_value_loss: true - schedule: adaptive desired_kl: 0.01 - adaptive_kl_factor: 2.0 - adaptive_lr_factor: 1.5 - optimizer: adam - tau: 1.0 - target_update_freq: 1 - vtrace_clip_rho: 1.0 - vtrace_clip_c: 1.0 - enable_compile: true reward: scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 motion_body_pos: 1.0 motion_body_ori: 1.0 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_joint_pos: 0.0 - motion_joint_vel: 0.0 action_rate_l2: -0.1 - joint_limit: -10.0 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 diff --git a/conf/appo/task/g1_wall_flip_tracking/motrix_play.yaml b/conf/appo/task/g1_wall_flip_tracking/motrix_play.yaml new file mode 100644 index 000000000..78ea34f66 --- /dev/null +++ b/conf/appo/task/g1_wall_flip_tracking/motrix_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/g1_wall_flip_tracking/motrix + - _self_ + +training: + play_only: true diff --git a/conf/appo/task/g1_wall_flip_tracking/mujoco.yaml b/conf/appo/task/g1_wall_flip_tracking/mujoco.yaml index 0083729c0..712156a02 100644 --- a/conf/appo/task/g1_wall_flip_tracking/mujoco.yaml +++ b/conf/appo/task/g1_wall_flip_tracking/mujoco.yaml @@ -1,70 +1,49 @@ # @package _global_ +defaults: + - /task/g1_wall_flip_tracking/base + - _self_ + training: task_name: G1WallFlipTracking sim_backend: mujoco - play_steps: 1000 replay_queue_size: 5 algo: - num_envs: 1024 steps_per_env: 20 - max_iterations: 7000 - save_interval: 500 - algorithm: - num_learning_epochs: 6 - num_mini_batches: 8 - clip_param: 0.2 - gamma: 0.99 - lam: 0.95 - value_loss_coef: 1.0 - entropy_coef: 0.005 - learning_rate: 1.0e-3 - max_grad_norm: 1.0 - use_clipped_value_loss: true - schedule: adaptive - desired_kl: 0.008 - adaptive_kl_factor: 2.0 - adaptive_lr_factor: 1.5 - optimizer: adam - tau: 1.0 - target_update_freq: 1 - vtrace_clip_rho: 1.0 - vtrace_clip_c: 1.0 - enable_compile: true env: sampling_mode: start truncate_on_clip_end: true sim_dt: 0.005 control_config: action_scale: - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 anchor_pos_z_threshold: 0.5 ee_body_pos_z_threshold: 0.5 terminate_on_undesired_contacts: true @@ -72,22 +51,4 @@ env: level: 0.0 reward: scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 2.0 - motion_body_ori: 1.5 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 motion_ee_body_pos_z: 2.0 - motion_joint_pos: 0.0 - motion_joint_vel: 0.0 - action_rate_l2: -0.005 - joint_limit: -10.0 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 \ No newline at end of file diff --git a/conf/appo/task/g1_wall_flip_tracking/mujoco_play.yaml b/conf/appo/task/g1_wall_flip_tracking/mujoco_play.yaml new file mode 100644 index 000000000..b2ccd7aac --- /dev/null +++ b/conf/appo/task/g1_wall_flip_tracking/mujoco_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/g1_wall_flip_tracking/mujoco + - _self_ + +training: + play_only: true diff --git a/conf/appo/task/go1_joystick_flat/base.yaml b/conf/appo/task/go1_joystick_flat/base.yaml new file mode 100644 index 000000000..2f15f0884 --- /dev/null +++ b/conf/appo/task/go1_joystick_flat/base.yaml @@ -0,0 +1,16 @@ +# @package _global_ +# Shared cross-backend config for go1_joystick_flat (mujoco/motrix). See AGENTS.md (Sim2Sim). + +algo: + max_iterations: 150 +reward: + scales: + tracking_lin_vel: 1.0 + tracking_ang_vel: 0.2 + lin_vel_z: -5.0 + ang_vel_xy: -0.1 + base_height: -100.0 + action_rate: -0.005 + similar_to_default: -0.1 + tracking_sigma: 0.25 + base_height_target: 0.3 diff --git a/conf/appo/task/go1_joystick_flat/motrix.yaml b/conf/appo/task/go1_joystick_flat/motrix.yaml index f7343fd5d..c3dcebb93 100644 --- a/conf/appo/task/go1_joystick_flat/motrix.yaml +++ b/conf/appo/task/go1_joystick_flat/motrix.yaml @@ -1,4 +1,8 @@ # @package _global_ +defaults: + - /task/go1_joystick_flat/base + - _self_ + training: task_name: Go1JoystickFlat sim_backend: motrix @@ -10,25 +14,22 @@ algo: distribution_cfg: init_std: 0.5 algorithm: - learning_rate: 5.0e-4 - entropy_coef: 1.0e-3 + learning_rate: 0.0005 + entropy_coef: 0.001 desired_kl: 0.008 env: sim_dt: 0.01 commands: vel_limit: - - [0.5, 0.0, 0.0] - - [0.5, 0.0, 0.0] + - - 0.5 + - 0.0 + - 0.0 + - - 0.5 + - 0.0 + - 0.0 reward: scales: - tracking_lin_vel: 1.0 - tracking_ang_vel: 0.2 - lin_vel_z: -5.0 - ang_vel_xy: -0.1 - base_height: -100.0 action_rate: -0.015 action_smooth: -0.01 similar_to_default: -0.15 swing_feet_z: 2.0 - tracking_sigma: 0.25 - base_height_target: 0.3 diff --git a/conf/appo/task/go1_joystick_flat/motrix_play.yaml b/conf/appo/task/go1_joystick_flat/motrix_play.yaml new file mode 100644 index 000000000..ecb36fb63 --- /dev/null +++ b/conf/appo/task/go1_joystick_flat/motrix_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/go1_joystick_flat/motrix + - _self_ + +training: + play_only: true diff --git a/conf/appo/task/go1_joystick_flat/mujoco.yaml b/conf/appo/task/go1_joystick_flat/mujoco.yaml index 8646817a5..2e57b1246 100644 --- a/conf/appo/task/go1_joystick_flat/mujoco.yaml +++ b/conf/appo/task/go1_joystick_flat/mujoco.yaml @@ -1,18 +1,11 @@ # @package _global_ -training: - task_name: Go1JoystickFlat - sim_backend: mujoco -algo: - max_iterations: 150 +defaults: + - /task/go1_joystick_flat/base + - _self_ + reward: scales: - tracking_lin_vel: 1.0 - tracking_ang_vel: 0.2 - lin_vel_z: -5.0 - ang_vel_xy: -0.1 - base_height: -100.0 - action_rate: -0.005 - similar_to_default: -0.1 contact: 0.24 - tracking_sigma: 0.25 - base_height_target: 0.3 +training: + task_name: Go1JoystickFlat + sim_backend: mujoco diff --git a/conf/appo/task/go1_joystick_flat/mujoco_play.yaml b/conf/appo/task/go1_joystick_flat/mujoco_play.yaml new file mode 100644 index 000000000..a2ead3142 --- /dev/null +++ b/conf/appo/task/go1_joystick_flat/mujoco_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/go1_joystick_flat/mujoco + - _self_ + +training: + play_only: true diff --git a/conf/appo/task/go2_joystick_flat/base.yaml b/conf/appo/task/go2_joystick_flat/base.yaml new file mode 100644 index 000000000..ce06048fb --- /dev/null +++ b/conf/appo/task/go2_joystick_flat/base.yaml @@ -0,0 +1,19 @@ +# @package _global_ +# Shared cross-backend config for go2_joystick_flat (mujoco/motrix). See AGENTS.md (Sim2Sim). + +algo: + max_iterations: 150 +reward: + scales: + tracking_lin_vel: 1.0 + tracking_ang_vel: 0.2 + lin_vel_z: -5.0 + ang_vel_xy: -0.1 + base_height: -100.0 + action_rate: -0.005 + similar_to_default: -0.1 + alive: 0.0 + contact: 0.24 + swing_feet_z: 4.0 + tracking_sigma: 0.25 + base_height_target: 0.3 diff --git a/conf/appo/task/go2_joystick_flat/motrix.yaml b/conf/appo/task/go2_joystick_flat/motrix.yaml index fc4597bbd..a5e12f27f 100644 --- a/conf/appo/task/go2_joystick_flat/motrix.yaml +++ b/conf/appo/task/go2_joystick_flat/motrix.yaml @@ -1,4 +1,8 @@ # @package _global_ +defaults: + - /task/go2_joystick_flat/base + - _self_ + training: task_name: Go2JoystickFlat sim_backend: motrix @@ -8,17 +12,3 @@ algo: max_iterations: 180 env: sim_dt: 0.015 -reward: - scales: - tracking_lin_vel: 1.0 - tracking_ang_vel: 0.2 - lin_vel_z: -5.0 - ang_vel_xy: -0.1 - base_height: -100.0 - action_rate: -0.005 - similar_to_default: -0.1 - alive: 0.0 - contact: 0.24 - swing_feet_z: 4.0 - tracking_sigma: 0.25 - base_height_target: 0.3 diff --git a/conf/appo/task/go2_joystick_flat/motrix_play.yaml b/conf/appo/task/go2_joystick_flat/motrix_play.yaml new file mode 100644 index 000000000..3d2f98124 --- /dev/null +++ b/conf/appo/task/go2_joystick_flat/motrix_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/go2_joystick_flat/motrix + - _self_ + +training: + play_only: true diff --git a/conf/appo/task/go2_joystick_flat/mujoco.yaml b/conf/appo/task/go2_joystick_flat/mujoco.yaml index d25c356ad..9217af855 100644 --- a/conf/appo/task/go2_joystick_flat/mujoco.yaml +++ b/conf/appo/task/go2_joystick_flat/mujoco.yaml @@ -1,20 +1,7 @@ # @package _global_ +defaults: + - /task/go2_joystick_flat/base + - _self_ training: task_name: Go2JoystickFlat sim_backend: mujoco -algo: - max_iterations: 150 -reward: - scales: - tracking_lin_vel: 1.0 - tracking_ang_vel: 0.2 - lin_vel_z: -5.0 - ang_vel_xy: -0.1 - base_height: -100.0 - action_rate: -0.005 - similar_to_default: -0.1 - alive: 0.0 - contact: 0.24 - swing_feet_z: 4.0 - tracking_sigma: 0.25 - base_height_target: 0.3 diff --git a/conf/appo/task/go2_joystick_flat/mujoco_play.yaml b/conf/appo/task/go2_joystick_flat/mujoco_play.yaml new file mode 100644 index 000000000..4708d5243 --- /dev/null +++ b/conf/appo/task/go2_joystick_flat/mujoco_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/go2_joystick_flat/mujoco + - _self_ + +training: + play_only: true diff --git a/conf/appo/task/sharpa_inhand/motrix_play.yaml b/conf/appo/task/sharpa_inhand/motrix_play.yaml new file mode 100644 index 000000000..98779fc8e --- /dev/null +++ b/conf/appo/task/sharpa_inhand/motrix_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/sharpa_inhand/motrix + - _self_ + +training: + play_only: true diff --git a/conf/appo/task/sharpa_inhand/mujoco_play.yaml b/conf/appo/task/sharpa_inhand/mujoco_play.yaml new file mode 100644 index 000000000..3b23bf337 --- /dev/null +++ b/conf/appo/task/sharpa_inhand/mujoco_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/sharpa_inhand/mujoco + - _self_ + +training: + play_only: true diff --git a/conf/offpolicy/task/sac/g1_motion_tracking/motrix_play.yaml b/conf/offpolicy/task/sac/g1_motion_tracking/motrix_play.yaml new file mode 100644 index 000000000..8fa9f0499 --- /dev/null +++ b/conf/offpolicy/task/sac/g1_motion_tracking/motrix_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/sac/g1_motion_tracking/motrix + - _self_ + +training: + play_only: true diff --git a/conf/offpolicy/task/sac/g1_motion_tracking/mujoco_play.yaml b/conf/offpolicy/task/sac/g1_motion_tracking/mujoco_play.yaml new file mode 100644 index 000000000..10b404ff6 --- /dev/null +++ b/conf/offpolicy/task/sac/g1_motion_tracking/mujoco_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/sac/g1_motion_tracking/mujoco + - _self_ + +training: + play_only: true diff --git a/conf/offpolicy/task/sac/g1_walk_flat/base.yaml b/conf/offpolicy/task/sac/g1_walk_flat/base.yaml new file mode 100644 index 000000000..c2dcc00f6 --- /dev/null +++ b/conf/offpolicy/task/sac/g1_walk_flat/base.yaml @@ -0,0 +1,75 @@ +# @package _global_ +# Shared cross-backend config for g1_walk_flat (mujoco/motrix). See AGENTS.md (Sim2Sim). + +algo: + num_envs: 2048 + learning_starts: 10 + max_iterations: 5000 + save_interval: 1000 + updates_per_step: 8 + use_symmetry: true + algo_params: + alpha_init: 0.001 + target_entropy_ratio: 0.0 +env: + control_config: + action_scale: 1.0 + gait_phase_init_mode: offset_phase + reset_base_qvel_limit: 0.5 + curriculum: + enabled: true + initial_scale: 0.5 + min_scale: 0.5 + max_scale: 1.0 + level_down_threshold: 150.0 + level_up_threshold: 750.0 + degree: 0.001 +reward: + scales: + tracking_lin_vel: 2.0 + tracking_ang_vel: 1.5 + penalty_ang_vel_xy: -1.0 + penalty_orientation: -10.0 + penalty_action_rate: -4.0 + pose: -0.5 + penalty_feet_ori: -20.0 + feet_phase: 5.0 + alive: 10.0 + tracking_sigma: 0.25 + base_height_target: 0.754 + min_base_height: 0.3 + max_tilt_deg: 65.0 + gait_frequency: 1.5 + feet_phase_swing_height: 0.09 + feet_phase_tracking_sigma: 0.04 + close_feet_threshold: 0.15 + pose_weights: + - 0.01 + - 1.0 + - 5.0 + - 0.01 + - 5.0 + - 5.0 + - 0.01 + - 1.0 + - 5.0 + - 0.01 + - 5.0 + - 5.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 diff --git a/conf/offpolicy/task/sac/g1_walk_flat/motrix.yaml b/conf/offpolicy/task/sac/g1_walk_flat/motrix.yaml index 9698d1d77..a17dc4556 100644 --- a/conf/offpolicy/task/sac/g1_walk_flat/motrix.yaml +++ b/conf/offpolicy/task/sac/g1_walk_flat/motrix.yaml @@ -1,33 +1,18 @@ # @package _global_ +defaults: + - /task/sac/g1_walk_flat/base + - _self_ + training: task_name: G1WalkFlat sim_backend: motrix algo: - num_envs: 2048 learning_starts: 1 - max_iterations: 5000 - save_interval: 1000 - updates_per_step: 8 use_symmetry: false - algo_params: - alpha_init: 0.001 - target_entropy_ratio: 0.0 env: - control_config: - action_scale: 1.0 - gait_phase_init_mode: "offset_phase" - reset_base_qvel_limit: 0.5 domain_rand: randomize_kp: false randomize_kd: false - curriculum: - enabled: true - initial_scale: 0.5 - min_scale: 0.5 - max_scale: 1.0 - level_down_threshold: 150.0 - level_up_threshold: 750.0 - degree: 0.001 reward: scales: tracking_lin_vel: 2.2 @@ -39,12 +24,4 @@ reward: penalty_feet_ori: -5.0 feet_phase: 6.0 alive: 12.0 - tracking_sigma: 0.25 - base_height_target: 0.754 - min_base_height: 0.3 - max_tilt_deg: 65.0 - gait_frequency: 1.5 - feet_phase_swing_height: 0.09 feet_phase_tracking_sigma: 0.008 - close_feet_threshold: 0.15 - pose_weights: [0.01, 1.0, 5.0, 0.01, 5.0, 5.0, 0.01, 1.0, 5.0, 0.01, 5.0, 5.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0] diff --git a/conf/offpolicy/task/sac/g1_walk_flat/motrix_play.yaml b/conf/offpolicy/task/sac/g1_walk_flat/motrix_play.yaml new file mode 100644 index 000000000..a857ecae4 --- /dev/null +++ b/conf/offpolicy/task/sac/g1_walk_flat/motrix_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/sac/g1_walk_flat/motrix + - _self_ + +training: + play_only: true diff --git a/conf/offpolicy/task/sac/g1_walk_flat/mujoco.yaml b/conf/offpolicy/task/sac/g1_walk_flat/mujoco.yaml index 37abe2d99..e6f385199 100644 --- a/conf/offpolicy/task/sac/g1_walk_flat/mujoco.yaml +++ b/conf/offpolicy/task/sac/g1_walk_flat/mujoco.yaml @@ -1,30 +1,9 @@ # @package _global_ -training: - task_name: G1WalkFlat - sim_backend: mujoco -algo: - num_envs: 2048 - learning_starts: 10 - max_iterations: 5000 - save_interval: 1000 - updates_per_step: 8 - use_symmetry: true - algo_params: - alpha_init: 0.001 - target_entropy_ratio: 0.0 +defaults: + - /task/sac/g1_walk_flat/base + - _self_ + env: - control_config: - action_scale: 1.0 - gait_phase_init_mode: "offset_phase" - reset_base_qvel_limit: 0.5 - curriculum: - enabled: true - initial_scale: 0.5 - min_scale: 0.5 - max_scale: 1.0 - level_down_threshold: 150.0 - level_up_threshold: 750.0 - degree: 0.001 noise_config: level: 1.0 scale_gyro: 0.0 @@ -32,23 +11,6 @@ env: scale_joint_angle: 0.01 scale_joint_vel: 0.1 scale_linvel: 0.0 -reward: - scales: - tracking_lin_vel: 2.0 - tracking_ang_vel: 1.5 - penalty_ang_vel_xy: -1.0 - penalty_orientation: -10.0 - penalty_action_rate: -4.0 - pose: -0.5 - penalty_feet_ori: -20.0 - feet_phase: 5.0 - alive: 10.0 - tracking_sigma: 0.25 - base_height_target: 0.754 - min_base_height: 0.3 - max_tilt_deg: 65.0 - gait_frequency: 1.5 - feet_phase_swing_height: 0.09 - feet_phase_tracking_sigma: 0.04 - close_feet_threshold: 0.15 - pose_weights: [0.01, 1.0, 5.0, 0.01, 5.0, 5.0, 0.01, 1.0, 5.0, 0.01, 5.0, 5.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0] +training: + task_name: G1WalkFlat + sim_backend: mujoco diff --git a/conf/offpolicy/task/sac/g1_walk_flat/mujoco_play.yaml b/conf/offpolicy/task/sac/g1_walk_flat/mujoco_play.yaml new file mode 100644 index 000000000..5b65b2b12 --- /dev/null +++ b/conf/offpolicy/task/sac/g1_walk_flat/mujoco_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/sac/g1_walk_flat/mujoco + - _self_ + +training: + play_only: true diff --git a/conf/offpolicy/task/sac/g1_walk_rough/base.yaml b/conf/offpolicy/task/sac/g1_walk_rough/base.yaml new file mode 100644 index 000000000..a987eb7a9 --- /dev/null +++ b/conf/offpolicy/task/sac/g1_walk_rough/base.yaml @@ -0,0 +1,75 @@ +# @package _global_ +# Shared cross-backend config for g1_walk_rough (mujoco/motrix). See AGENTS.md (Sim2Sim). + +algo: + num_envs: 2048 + learning_starts: 10 + max_iterations: 5000 + save_interval: 1000 + updates_per_step: 8 + use_symmetry: true + algo_params: + alpha_init: 0.001 + target_entropy_ratio: 0.0 +env: + control_config: + action_scale: 1.0 + gait_phase_init_mode: offset_phase + reset_base_qvel_limit: 0.5 + curriculum: + enabled: true + initial_scale: 0.5 + min_scale: 0.5 + max_scale: 1.0 + level_down_threshold: 150.0 + level_up_threshold: 750.0 + degree: 0.001 +reward: + scales: + tracking_lin_vel: 2.0 + tracking_ang_vel: 1.5 + penalty_ang_vel_xy: -1.0 + penalty_orientation: -10.0 + penalty_action_rate: -4.0 + pose: -0.5 + penalty_feet_ori: -20.0 + feet_phase: 5.0 + alive: 10.0 + tracking_sigma: 0.25 + base_height_target: 0.754 + min_base_height: 0.3 + max_tilt_deg: 65.0 + gait_frequency: 1.5 + feet_phase_swing_height: 0.09 + feet_phase_tracking_sigma: 0.04 + close_feet_threshold: 0.15 + pose_weights: + - 0.01 + - 1.0 + - 5.0 + - 0.01 + - 5.0 + - 5.0 + - 0.01 + - 1.0 + - 5.0 + - 0.01 + - 5.0 + - 5.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 diff --git a/conf/offpolicy/task/sac/g1_walk_rough/motrix.yaml b/conf/offpolicy/task/sac/g1_walk_rough/motrix.yaml index 4144b1713..a37acc1a4 100644 --- a/conf/offpolicy/task/sac/g1_walk_rough/motrix.yaml +++ b/conf/offpolicy/task/sac/g1_walk_rough/motrix.yaml @@ -1,34 +1,19 @@ # @package _global_ +defaults: + - /task/sac/g1_walk_rough/base + - _self_ + training: task_name: G1WalkRough sim_backend: motrix algo: - num_envs: 2048 learning_starts: 1 - max_iterations: 5000 - save_interval: 1000 - updates_per_step: 8 use_symmetry: false - algo_params: - alpha_init: 0.001 - target_entropy_ratio: 0.0 env: sim_dt: 0.01 - control_config: - action_scale: 1.0 - gait_phase_init_mode: "offset_phase" - reset_base_qvel_limit: 0.5 domain_rand: randomize_kp: false randomize_kd: false - curriculum: - enabled: true - initial_scale: 0.5 - min_scale: 0.5 - max_scale: 1.0 - level_down_threshold: 150.0 - level_up_threshold: 750.0 - degree: 0.001 reward: scales: tracking_lin_vel: 2.2 @@ -40,12 +25,4 @@ reward: penalty_feet_ori: -5.0 feet_phase: 6.0 alive: 12.0 - tracking_sigma: 0.25 - base_height_target: 0.754 - min_base_height: 0.3 - max_tilt_deg: 65.0 - gait_frequency: 1.5 - feet_phase_swing_height: 0.09 feet_phase_tracking_sigma: 0.008 - close_feet_threshold: 0.15 - pose_weights: [0.01, 1.0, 5.0, 0.01, 5.0, 5.0, 0.01, 1.0, 5.0, 0.01, 5.0, 5.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0] diff --git a/conf/offpolicy/task/sac/g1_walk_rough/motrix_play.yaml b/conf/offpolicy/task/sac/g1_walk_rough/motrix_play.yaml new file mode 100644 index 000000000..06338b01e --- /dev/null +++ b/conf/offpolicy/task/sac/g1_walk_rough/motrix_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/sac/g1_walk_rough/motrix + - _self_ + +training: + play_only: true diff --git a/conf/offpolicy/task/sac/g1_walk_rough/mujoco.yaml b/conf/offpolicy/task/sac/g1_walk_rough/mujoco.yaml index bd9a1282a..8dc372210 100644 --- a/conf/offpolicy/task/sac/g1_walk_rough/mujoco.yaml +++ b/conf/offpolicy/task/sac/g1_walk_rough/mujoco.yaml @@ -1,30 +1,9 @@ # @package _global_ -training: - task_name: G1WalkRough - sim_backend: mujoco -algo: - num_envs: 2048 - learning_starts: 10 - max_iterations: 5000 - save_interval: 1000 - updates_per_step: 8 - use_symmetry: true - algo_params: - alpha_init: 0.001 - target_entropy_ratio: 0.0 +defaults: + - /task/sac/g1_walk_rough/base + - _self_ + env: - control_config: - action_scale: 1.0 - gait_phase_init_mode: "offset_phase" - reset_base_qvel_limit: 0.5 - curriculum: - enabled: true - initial_scale: 0.5 - min_scale: 0.5 - max_scale: 1.0 - level_down_threshold: 150.0 - level_up_threshold: 750.0 - degree: 0.001 noise_config: level: 1.0 scale_gyro: 0.0 @@ -32,23 +11,6 @@ env: scale_joint_angle: 0.01 scale_joint_vel: 0.1 scale_linvel: 0.0 -reward: - scales: - tracking_lin_vel: 2.0 - tracking_ang_vel: 1.5 - penalty_ang_vel_xy: -1.0 - penalty_orientation: -10.0 - penalty_action_rate: -4.0 - pose: -0.5 - penalty_feet_ori: -20.0 - feet_phase: 5.0 - alive: 10.0 - tracking_sigma: 0.25 - base_height_target: 0.754 - min_base_height: 0.3 - max_tilt_deg: 65.0 - gait_frequency: 1.5 - feet_phase_swing_height: 0.09 - feet_phase_tracking_sigma: 0.04 - close_feet_threshold: 0.15 - pose_weights: [0.01, 1.0, 5.0, 0.01, 5.0, 5.0, 0.01, 1.0, 5.0, 0.01, 5.0, 5.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0] +training: + task_name: G1WalkRough + sim_backend: mujoco diff --git a/conf/offpolicy/task/sac/g1_walk_rough/mujoco_play.yaml b/conf/offpolicy/task/sac/g1_walk_rough/mujoco_play.yaml new file mode 100644 index 000000000..08bdabf81 --- /dev/null +++ b/conf/offpolicy/task/sac/g1_walk_rough/mujoco_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/sac/g1_walk_rough/mujoco + - _self_ + +training: + play_only: true diff --git a/conf/ppo/task/allegro_inhand/base.yaml b/conf/ppo/task/allegro_inhand/base.yaml new file mode 100644 index 000000000..9ede9fd08 --- /dev/null +++ b/conf/ppo/task/allegro_inhand/base.yaml @@ -0,0 +1,58 @@ +# @package _global_ +# Shared cross-backend config for allegro_inhand (mujoco/motrix). See AGENTS.md (Sim2Sim). + +algo: + num_envs: 16384 + num_steps_per_env: 8 + max_iterations: 201 + obs_groups: + actor: + - policy + critic: + - policy + actor: + class_name: rsl_rl.models.MLPModel + hidden_dims: + - 512 + - 256 + - 128 + activation: elu + obs_normalization: true + distribution_cfg: + class_name: rsl_rl.modules.distribution.GaussianDistribution + init_std: 1.0 + std_type: scalar + critic: + class_name: rsl_rl.models.MLPModel + hidden_dims: + - 512 + - 256 + - 128 + activation: elu + obs_normalization: true + algorithm: + value_loss_coef: 4.0 + desired_kl: 0.02 +reward: + scales: + rotate: 1.25 + obj_linvel: -0.3 + pose_diff: -0.3 + torque: -0.1 + work: -2.0 + drop: 0.0 + angvel_clip_min: -0.5 + angvel_clip_max: 0.5 + reset_z_threshold: 0.125 +env: + gen_grasp: false + max_episode_seconds: 20.0 + grasp_cache_path: caches/allegro_grasp_50k.npy + domain_rand: + randomize_base_mass: false + random_com: false + randomize_gravity: false + push_robots: false + joint_noise: 0.0 + ball_vel_noise: 0.0 + ball_z_offset: 0.0 diff --git a/conf/ppo/task/allegro_inhand/motrix.yaml b/conf/ppo/task/allegro_inhand/motrix.yaml index d4d68902b..7e9186d93 100644 --- a/conf/ppo/task/allegro_inhand/motrix.yaml +++ b/conf/ppo/task/allegro_inhand/motrix.yaml @@ -1,52 +1,8 @@ # @package _global_ +defaults: + - /task/allegro_inhand/base + - _self_ + training: task_name: AllegroInhandRotation sim_backend: motrix -algo: - num_envs: 16384 - num_steps_per_env: 8 - max_iterations: 201 - obs_groups: - actor: [policy] - critic: [policy] - actor: - class_name: rsl_rl.models.MLPModel - hidden_dims: [512, 256, 128] - activation: elu - obs_normalization: true - distribution_cfg: - class_name: rsl_rl.modules.distribution.GaussianDistribution - init_std: 1.0 - std_type: scalar - critic: - class_name: rsl_rl.models.MLPModel - hidden_dims: [512, 256, 128] - activation: elu - obs_normalization: true - algorithm: - value_loss_coef: 4.0 - desired_kl: 0.02 -reward: - scales: - rotate: 1.25 - obj_linvel: -0.3 - pose_diff: -0.3 - torque: -0.1 - work: -2.0 - drop: 0.0 - angvel_clip_min: -0.5 - angvel_clip_max: 0.5 - reset_z_threshold: 0.125 -env: - gen_grasp: false - max_episode_seconds: 20.0 - grasp_cache_path: caches/allegro_grasp_50k.npy - # Keep only grasp/pose reset variation. All online DR terms stay disabled. - domain_rand: - randomize_base_mass: false - random_com: false - randomize_gravity: false - push_robots: false - joint_noise: 0.0 - ball_vel_noise: 0.0 - ball_z_offset: 0.0 diff --git a/conf/ppo/task/allegro_inhand/motrix_play.yaml b/conf/ppo/task/allegro_inhand/motrix_play.yaml new file mode 100644 index 000000000..ff922b259 --- /dev/null +++ b/conf/ppo/task/allegro_inhand/motrix_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/allegro_inhand/motrix + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/allegro_inhand/mujoco.yaml b/conf/ppo/task/allegro_inhand/mujoco.yaml index 3bfed4175..031a618a1 100644 --- a/conf/ppo/task/allegro_inhand/mujoco.yaml +++ b/conf/ppo/task/allegro_inhand/mujoco.yaml @@ -1,56 +1,15 @@ # @package _global_ +defaults: + - /task/allegro_inhand/base + - _self_ + training: task_name: AllegroInhandRotation sim_backend: mujoco render_spacing: 0.5 cam_distance: 1.5 - cam_lookat: [0.75, 0.75, 0] + cam_lookat: + - 0.75 + - 0.75 + - 0 cam_elevation: -20.0 -algo: - num_envs: 16384 - num_steps_per_env: 8 - max_iterations: 201 - obs_groups: - actor: [policy] - critic: [policy] - actor: - class_name: rsl_rl.models.MLPModel - hidden_dims: [512, 256, 128] - activation: elu - obs_normalization: true - distribution_cfg: - class_name: rsl_rl.modules.distribution.GaussianDistribution - init_std: 1.0 - std_type: scalar - critic: - class_name: rsl_rl.models.MLPModel - hidden_dims: [512, 256, 128] - activation: elu - obs_normalization: true - algorithm: - value_loss_coef: 4.0 - desired_kl: 0.02 -reward: - scales: - rotate: 1.25 - obj_linvel: -0.3 - pose_diff: -0.3 - torque: -0.1 - work: -2.0 - drop: 0.0 - angvel_clip_min: -0.5 - angvel_clip_max: 0.5 - reset_z_threshold: 0.125 -env: - gen_grasp: false - max_episode_seconds: 20.0 - grasp_cache_path: caches/allegro_grasp_50k.npy - # Keep only grasp/pose reset variation. All online DR terms stay disabled. - domain_rand: - randomize_base_mass: false - random_com: false - randomize_gravity: false - push_robots: false - joint_noise: 0.0 - ball_vel_noise: 0.0 - ball_z_offset: 0.0 diff --git a/conf/ppo/task/allegro_inhand/mujoco_play.yaml b/conf/ppo/task/allegro_inhand/mujoco_play.yaml new file mode 100644 index 000000000..63d251085 --- /dev/null +++ b/conf/ppo/task/allegro_inhand/mujoco_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/allegro_inhand/mujoco + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/allegro_inhand_grasp/motrix_play.yaml b/conf/ppo/task/allegro_inhand_grasp/motrix_play.yaml new file mode 100644 index 000000000..db0fe057e --- /dev/null +++ b/conf/ppo/task/allegro_inhand_grasp/motrix_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/allegro_inhand_grasp/motrix + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/allegro_inhand_grasp/mujoco_play.yaml b/conf/ppo/task/allegro_inhand_grasp/mujoco_play.yaml new file mode 100644 index 000000000..916fabfa3 --- /dev/null +++ b/conf/ppo/task/allegro_inhand_grasp/mujoco_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/allegro_inhand_grasp/mujoco + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_box_tracking/base.yaml b/conf/ppo/task/g1_box_tracking/base.yaml new file mode 100644 index 000000000..706ce0bd2 --- /dev/null +++ b/conf/ppo/task/g1_box_tracking/base.yaml @@ -0,0 +1,39 @@ +# @package _global_ +# Shared cross-backend config for g1_box_tracking (mujoco/motrix). See AGENTS.md (Sim2Sim). + +training: + play_steps: 1000 +algo: + num_envs: 1024 + max_iterations: 30000 + save_interval: 500 + obs_groups: + actor: + - actor + algorithm: + entropy_coef: 0.005 +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 1.0 + motion_body_ori: 1.0 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_joint_pos: 0.0 + motion_joint_vel: 0.0 + action_rate_l2: -0.1 + joint_limit: -10.0 + undesired_contacts: -0.1 + object_global_ref_position_error_exp: 2.0 + object_global_ref_orientation_error_exp: 2.0 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 + std_object_pos: 0.2 + std_object_ori: 0.3 diff --git a/conf/ppo/task/g1_box_tracking/motrix.yaml b/conf/ppo/task/g1_box_tracking/motrix.yaml index 863ac1215..b8935f05b 100644 --- a/conf/ppo/task/g1_box_tracking/motrix.yaml +++ b/conf/ppo/task/g1_box_tracking/motrix.yaml @@ -1,19 +1,18 @@ # @package _global_ +defaults: + - /task/g1_box_tracking/base + - _self_ + training: task_name: G1BoxTracking sim_backend: motrix play_env_num: 16 - play_steps: 1000 algo: - num_envs: 1024 max_iterations: 40000 - save_interval: 500 empirical_normalization: true obs_groups: - actor: - - actor critic: - - critic + - critic algorithm: entropy_coef: 0.002 desired_kl: 0.01 @@ -30,31 +29,23 @@ play_profile: enabled: true source_model_file: src/unilab/assets/robots/g1/scene_flat_with_largebox.xml ground_texture_file: src/unilab/assets/robots/g1/textures/floor.png - skybox_rgb1: [0.90, 0.90, 0.91] - skybox_rgb2: [0.68, 0.68, 0.70] - ground_texrepeat: [0.25, 0.25] + skybox_rgb1: + - 0.9 + - 0.9 + - 0.91 + skybox_rgb2: + - 0.68 + - 0.68 + - 0.7 + ground_texrepeat: + - 0.25 + - 0.25 reward: scales: motion_global_root_pos: 1.0 - motion_global_root_ori: 0.5 - motion_body_pos: 1.0 motion_body_ori: 1.5 - motion_body_lin_vel: 1.0 motion_body_ang_vel: 1.5 - motion_joint_pos: 0.0 - motion_joint_vel: 0.0 - action_rate_l2: -0.1 - joint_limit: -10.0 - undesired_contacts: -0.1 object_global_ref_position_error_exp: 4.0 object_global_ref_orientation_error_exp: 3.0 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 std_object_pos: 0.12 std_object_ori: 0.2 diff --git a/conf/ppo/task/g1_box_tracking/motrix_play.yaml b/conf/ppo/task/g1_box_tracking/motrix_play.yaml new file mode 100644 index 000000000..287c4b354 --- /dev/null +++ b/conf/ppo/task/g1_box_tracking/motrix_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/g1_box_tracking/motrix + - _self_ + +training: + play_only: true diff --git a/conf/ppo/task/g1_box_tracking/mujoco.yaml b/conf/ppo/task/g1_box_tracking/mujoco.yaml index bb39b7ee4..b23f81672 100644 --- a/conf/ppo/task/g1_box_tracking/mujoco.yaml +++ b/conf/ppo/task/g1_box_tracking/mujoco.yaml @@ -1,44 +1,13 @@ # @package _global_ -training: - task_name: G1BoxTracking - sim_backend: mujoco - play_steps: 1000 -algo: - num_envs: 1024 - max_iterations: 30000 - save_interval: 500 - obs_groups: - actor: - - actor - algorithm: - entropy_coef: 0.005 +defaults: + - /task/g1_box_tracking/base + - _self_ + env: sim_dt: 0.005 sensor: gyro: pelvis_gyro upvector: pelvis_upvector -reward: - scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 1.0 - motion_body_ori: 1.0 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_joint_pos: 0.0 - motion_joint_vel: 0.0 - action_rate_l2: -0.1 - joint_limit: -10.0 - undesired_contacts: -0.1 - object_global_ref_position_error_exp: 2.0 - object_global_ref_orientation_error_exp: 2.0 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 - std_object_pos: 0.2 - std_object_ori: 0.3 +training: + task_name: G1BoxTracking + sim_backend: mujoco diff --git a/conf/ppo/task/g1_box_tracking/mujoco_play.yaml b/conf/ppo/task/g1_box_tracking/mujoco_play.yaml new file mode 100644 index 000000000..af69bba5f --- /dev/null +++ b/conf/ppo/task/g1_box_tracking/mujoco_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/g1_box_tracking/mujoco + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_climb_tracking/base.yaml b/conf/ppo/task/g1_climb_tracking/base.yaml new file mode 100644 index 000000000..c2206f39d --- /dev/null +++ b/conf/ppo/task/g1_climb_tracking/base.yaml @@ -0,0 +1,80 @@ +# @package _global_ +# Shared cross-backend config for g1_climb_tracking (mujoco/motrix). See AGENTS.md (Sim2Sim). + +training: + play_steps: 1000 +algo: + num_envs: 1024 + max_iterations: 20000 + save_interval: 500 + empirical_normalization: true + obs_groups: + actor: + - actor + critic: + - critic + algorithm: + entropy_coef: 0.005 + desired_kl: 0.01 +env: + sampling_mode: adaptive + truncate_on_clip_end: false + sim_dt: 0.005 + control_config: + action_scale: + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + anchor_pos_z_threshold: 0.3 + ee_body_pos_z_threshold: 0.3 + terminate_on_undesired_contacts: true + noise_config: + level: 0.0 +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 2.0 + motion_body_ori: 1.5 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_ee_body_pos_z: 2.0 + motion_joint_pos: 0.5 + motion_joint_vel: 0.25 + action_rate_l2: -0.005 + joint_limit: -10.0 + undesired_contacts: -0.1 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 diff --git a/conf/ppo/task/g1_climb_tracking/motrix.yaml b/conf/ppo/task/g1_climb_tracking/motrix.yaml index 7c6407e17..9077f325d 100644 --- a/conf/ppo/task/g1_climb_tracking/motrix.yaml +++ b/conf/ppo/task/g1_climb_tracking/motrix.yaml @@ -1,80 +1,8 @@ # @package _global_ +defaults: + - /task/g1_climb_tracking/base + - _self_ + training: task_name: G1ClimbTracking sim_backend: motrix - play_steps: 1000 -algo: - num_envs: 1024 - max_iterations: 20000 - save_interval: 500 - empirical_normalization: true - obs_groups: - actor: - - actor - critic: - - critic - algorithm: - entropy_coef: 0.005 - desired_kl: 0.01 -env: - sampling_mode: adaptive - truncate_on_clip_end: false - sim_dt: 0.005 - control_config: - action_scale: - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - anchor_pos_z_threshold: 0.3 - ee_body_pos_z_threshold: 0.3 - terminate_on_undesired_contacts: true - noise_config: - level: 0.0 -reward: - scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 2.0 - motion_body_ori: 1.5 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_ee_body_pos_z: 2.0 - motion_joint_pos: 0.5 - motion_joint_vel: 0.25 - action_rate_l2: -0.005 - joint_limit: -10.0 - undesired_contacts: -0.1 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 diff --git a/conf/ppo/task/g1_climb_tracking/motrix_play.yaml b/conf/ppo/task/g1_climb_tracking/motrix_play.yaml new file mode 100644 index 000000000..e73384e34 --- /dev/null +++ b/conf/ppo/task/g1_climb_tracking/motrix_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/g1_climb_tracking/motrix + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_climb_tracking/mujoco.yaml b/conf/ppo/task/g1_climb_tracking/mujoco.yaml index 9697dc3fc..1914cb9d2 100644 --- a/conf/ppo/task/g1_climb_tracking/mujoco.yaml +++ b/conf/ppo/task/g1_climb_tracking/mujoco.yaml @@ -1,80 +1,7 @@ # @package _global_ +defaults: + - /task/g1_climb_tracking/base + - _self_ training: task_name: G1ClimbTracking sim_backend: mujoco - play_steps: 1000 -algo: - num_envs: 1024 - max_iterations: 20000 - save_interval: 500 - empirical_normalization: true - obs_groups: - actor: - - actor - critic: - - critic - algorithm: - entropy_coef: 0.005 - desired_kl: 0.01 -env: - sampling_mode: adaptive - truncate_on_clip_end: false - sim_dt: 0.005 - control_config: - action_scale: - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - anchor_pos_z_threshold: 0.3 - ee_body_pos_z_threshold: 0.3 - terminate_on_undesired_contacts: true - noise_config: - level: 0.0 -reward: - scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 2.0 - motion_body_ori: 1.5 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_ee_body_pos_z: 2.0 - motion_joint_pos: 0.5 - motion_joint_vel: 0.25 - action_rate_l2: -0.005 - joint_limit: -10.0 - undesired_contacts: -0.1 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 diff --git a/conf/ppo/task/g1_climb_tracking/mujoco_play.yaml b/conf/ppo/task/g1_climb_tracking/mujoco_play.yaml new file mode 100644 index 000000000..0b4c96872 --- /dev/null +++ b/conf/ppo/task/g1_climb_tracking/mujoco_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/g1_climb_tracking/mujoco + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_flip_tracking/base.yaml b/conf/ppo/task/g1_flip_tracking/base.yaml new file mode 100644 index 000000000..a6b3841f1 --- /dev/null +++ b/conf/ppo/task/g1_flip_tracking/base.yaml @@ -0,0 +1,34 @@ +# @package _global_ +# Shared cross-backend config for g1_flip_tracking (mujoco/motrix). See AGENTS.md (Sim2Sim). + +training: + play_steps: 1000 +algo: + num_envs: 1024 + max_iterations: 20000 + save_interval: 500 + obs_groups: + actor: + - actor + algorithm: + entropy_coef: 0.005 +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 2.0 + motion_body_ori: 1.5 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_joint_pos: 0.0 + motion_joint_vel: 0.0 + action_rate_l2: -0.005 + joint_limit: -10.0 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 diff --git a/conf/ppo/task/g1_flip_tracking/motrix.yaml b/conf/ppo/task/g1_flip_tracking/motrix.yaml index 1d17d213c..0158b47c3 100644 --- a/conf/ppo/task/g1_flip_tracking/motrix.yaml +++ b/conf/ppo/task/g1_flip_tracking/motrix.yaml @@ -1,34 +1,16 @@ # @package _global_ +defaults: + - /task/g1_flip_tracking/base + - _self_ + training: task_name: G1FlipTracking sim_backend: motrix - play_steps: 1000 algo: - num_envs: 1024 max_iterations: 30000 - save_interval: 500 - obs_groups: - actor: - - actor - algorithm: - entropy_coef: 0.005 reward: scales: motion_global_root_pos: 1.0 - motion_global_root_ori: 0.5 motion_body_pos: 1.0 motion_body_ori: 1.0 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_joint_pos: 0.0 - motion_joint_vel: 0.0 action_rate_l2: -0.05 - joint_limit: -10.0 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 diff --git a/conf/ppo/task/g1_flip_tracking/motrix_play.yaml b/conf/ppo/task/g1_flip_tracking/motrix_play.yaml new file mode 100644 index 000000000..e2d1cc1c7 --- /dev/null +++ b/conf/ppo/task/g1_flip_tracking/motrix_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/g1_flip_tracking/motrix + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_flip_tracking/mujoco.yaml b/conf/ppo/task/g1_flip_tracking/mujoco.yaml index b9d736e2a..9998a7a61 100644 --- a/conf/ppo/task/g1_flip_tracking/mujoco.yaml +++ b/conf/ppo/task/g1_flip_tracking/mujoco.yaml @@ -1,20 +1,14 @@ # @package _global_ -training: - task_name: G1FlipTracking - sim_backend: mujoco - play_steps: 1000 +defaults: + - /task/g1_flip_tracking/base + - _self_ + algo: - num_envs: 1024 - max_iterations: 20000 - save_interval: 500 empirical_normalization: true obs_groups: - actor: - - actor critic: - - critic + - critic algorithm: - entropy_coef: 0.005 desired_kl: 0.01 env: sampling_mode: start @@ -22,35 +16,35 @@ env: sim_dt: 0.005 control_config: action_scale: - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 anchor_pos_z_threshold: 0.5 ee_body_pos_z_threshold: 0.5 terminate_on_undesired_contacts: true @@ -58,23 +52,8 @@ env: level: 0.0 reward: scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 2.0 - motion_body_ori: 1.5 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 motion_ee_body_pos_z: 2.0 - motion_joint_pos: 0.0 - motion_joint_vel: 0.0 - action_rate_l2: -0.005 - joint_limit: -10.0 undesired_contacts: -0.1 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 +training: + task_name: G1FlipTracking + sim_backend: mujoco diff --git a/conf/ppo/task/g1_flip_tracking/mujoco_play.yaml b/conf/ppo/task/g1_flip_tracking/mujoco_play.yaml new file mode 100644 index 000000000..76f600908 --- /dev/null +++ b/conf/ppo/task/g1_flip_tracking/mujoco_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/g1_flip_tracking/mujoco + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_motion_tracking/base.yaml b/conf/ppo/task/g1_motion_tracking/base.yaml new file mode 100644 index 000000000..46e54d369 --- /dev/null +++ b/conf/ppo/task/g1_motion_tracking/base.yaml @@ -0,0 +1,34 @@ +# @package _global_ +# Shared cross-backend config for g1_motion_tracking (mujoco/motrix). See AGENTS.md (Sim2Sim). + +training: + play_steps: 1000 +algo: + num_envs: 1024 + max_iterations: 15000 + save_interval: 500 + obs_groups: + actor: + - actor + algorithm: + entropy_coef: 0.005 +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 1.0 + motion_body_ori: 1.0 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_joint_pos: 0.0 + motion_joint_vel: 0.0 + action_rate_l2: -0.1 + joint_limit: -10.0 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 diff --git a/conf/ppo/task/g1_motion_tracking/motrix.yaml b/conf/ppo/task/g1_motion_tracking/motrix.yaml index 643939c54..dc0135dc8 100644 --- a/conf/ppo/task/g1_motion_tracking/motrix.yaml +++ b/conf/ppo/task/g1_motion_tracking/motrix.yaml @@ -1,18 +1,13 @@ # @package _global_ +defaults: + - /task/g1_motion_tracking/base + - _self_ + training: task_name: G1MotionTracking sim_backend: motrix play_env_num: 16 - play_steps: 1000 algo: - num_envs: 1024 - max_iterations: 15000 - save_interval: 500 - obs_groups: - actor: - - actor - algorithm: - entropy_coef: 0.005 noise_config: level: 1.0 scale_joint_angle: 0.01 @@ -26,27 +21,19 @@ play_profile: enabled: true source_model_file: src/unilab/assets/robots/g1/scene_flat.xml ground_texture_file: src/unilab/assets/robots/g1/textures/floor.png - skybox_rgb1: [0.90, 0.90, 0.91] - skybox_rgb2: [0.68, 0.68, 0.70] - ground_texrepeat: [0.25, 0.25] + skybox_rgb1: + - 0.9 + - 0.9 + - 0.91 + skybox_rgb2: + - 0.68 + - 0.68 + - 0.7 + ground_texrepeat: + - 0.25 + - 0.25 reward: scales: motion_global_root_pos: 1.0 - motion_global_root_ori: 0.5 - motion_body_pos: 1.0 - motion_body_ori: 1.0 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_joint_pos: 0.0 - motion_joint_vel: 0.0 action_rate_l2: -0.05 - joint_limit: -10.0 undesired_contacts: -0.1 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 diff --git a/conf/ppo/task/g1_motion_tracking/motrix_play.yaml b/conf/ppo/task/g1_motion_tracking/motrix_play.yaml new file mode 100644 index 000000000..9cebd0be8 --- /dev/null +++ b/conf/ppo/task/g1_motion_tracking/motrix_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/g1_motion_tracking/motrix + - _self_ + +training: + play_only: true diff --git a/conf/ppo/task/g1_motion_tracking/mujoco.yaml b/conf/ppo/task/g1_motion_tracking/mujoco.yaml index f257cd9e6..52b079dce 100644 --- a/conf/ppo/task/g1_motion_tracking/mujoco.yaml +++ b/conf/ppo/task/g1_motion_tracking/mujoco.yaml @@ -1,34 +1,7 @@ # @package _global_ +defaults: + - /task/g1_motion_tracking/base + - _self_ training: task_name: G1MotionTracking sim_backend: mujoco - play_steps: 1000 -algo: - num_envs: 1024 - max_iterations: 15000 - save_interval: 500 - obs_groups: - actor: - - actor - algorithm: - entropy_coef: 0.005 -reward: - scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 1.0 - motion_body_ori: 1.0 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_joint_pos: 0.0 - motion_joint_vel: 0.0 - action_rate_l2: -0.1 - joint_limit: -10.0 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 diff --git a/conf/ppo/task/g1_motion_tracking/mujoco_play.yaml b/conf/ppo/task/g1_motion_tracking/mujoco_play.yaml new file mode 100644 index 000000000..64a0b97ff --- /dev/null +++ b/conf/ppo/task/g1_motion_tracking/mujoco_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/g1_motion_tracking/mujoco + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_walk_flat/motrix_play.yaml b/conf/ppo/task/g1_walk_flat/motrix_play.yaml new file mode 100644 index 000000000..2ce9d3ac4 --- /dev/null +++ b/conf/ppo/task/g1_walk_flat/motrix_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/g1_walk_flat/motrix + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_walk_flat/mujoco_play.yaml b/conf/ppo/task/g1_walk_flat/mujoco_play.yaml new file mode 100644 index 000000000..37ae39c6a --- /dev/null +++ b/conf/ppo/task/g1_walk_flat/mujoco_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/g1_walk_flat/mujoco + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_wall_flip_tracking/base.yaml b/conf/ppo/task/g1_wall_flip_tracking/base.yaml new file mode 100644 index 000000000..a778abc01 --- /dev/null +++ b/conf/ppo/task/g1_wall_flip_tracking/base.yaml @@ -0,0 +1,80 @@ +# @package _global_ +# Shared cross-backend config for g1_wall_flip_tracking (mujoco/motrix). See AGENTS.md (Sim2Sim). + +training: + play_steps: 1000 +algo: + num_envs: 1024 + max_iterations: 20000 + save_interval: 500 + empirical_normalization: true + obs_groups: + actor: + - actor + critic: + - critic + algorithm: + entropy_coef: 0.005 + desired_kl: 0.01 +env: + sampling_mode: start + truncate_on_clip_end: false + sim_dt: 0.005 + control_config: + action_scale: + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + anchor_pos_z_threshold: 0.5 + ee_body_pos_z_threshold: 0.5 + terminate_on_undesired_contacts: true + noise_config: + level: 0.0 +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 2.0 + motion_body_ori: 1.5 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_ee_body_pos_z: 2.0 + motion_joint_pos: 0.5 + motion_joint_vel: 0.25 + action_rate_l2: -0.005 + joint_limit: -10.0 + undesired_contacts: -0.1 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 diff --git a/conf/ppo/task/g1_wall_flip_tracking/motrix.yaml b/conf/ppo/task/g1_wall_flip_tracking/motrix.yaml index c595203a6..bc1bc97c2 100644 --- a/conf/ppo/task/g1_wall_flip_tracking/motrix.yaml +++ b/conf/ppo/task/g1_wall_flip_tracking/motrix.yaml @@ -1,64 +1,17 @@ # @package _global_ +defaults: + - /task/g1_wall_flip_tracking/base + - _self_ + training: task_name: G1WallFlipTracking sim_backend: motrix play_env_num: 16 - play_steps: 1000 render_spacing: 3.0 algo: - num_envs: 1024 max_iterations: 12000 - save_interval: 500 - empirical_normalization: true - obs_groups: - actor: - - actor - critic: - - critic - algorithm: - entropy_coef: 0.005 - desired_kl: 0.01 env: motrix_max_iterations: 3 - sampling_mode: start - truncate_on_clip_end: false - sim_dt: 0.005 - control_config: - action_scale: - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - anchor_pos_z_threshold: 0.5 - ee_body_pos_z_threshold: 0.5 - terminate_on_undesired_contacts: true - noise_config: - level: 0.0 play_profile: enabled: true env: @@ -67,28 +20,14 @@ play_profile: enabled: true source_model_file: src/unilab/assets/robots/g1/scene_flat_with_wall.xml ground_texture_file: src/unilab/assets/robots/g1/textures/floor.png - skybox_rgb1: [0.90, 0.90, 0.91] - skybox_rgb2: [0.68, 0.68, 0.70] - ground_texrepeat: [0.25, 0.25] -reward: - scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 2.0 - motion_body_ori: 1.5 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_ee_body_pos_z: 2.0 - motion_joint_pos: 0.5 - motion_joint_vel: 0.25 - action_rate_l2: -0.005 - joint_limit: -10.0 - undesired_contacts: -0.1 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 + skybox_rgb1: + - 0.9 + - 0.9 + - 0.91 + skybox_rgb2: + - 0.68 + - 0.68 + - 0.7 + ground_texrepeat: + - 0.25 + - 0.25 diff --git a/conf/ppo/task/g1_wall_flip_tracking/motrix_play.yaml b/conf/ppo/task/g1_wall_flip_tracking/motrix_play.yaml new file mode 100644 index 000000000..78ea34f66 --- /dev/null +++ b/conf/ppo/task/g1_wall_flip_tracking/motrix_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/g1_wall_flip_tracking/motrix + - _self_ + +training: + play_only: true diff --git a/conf/ppo/task/g1_wall_flip_tracking/mujoco.yaml b/conf/ppo/task/g1_wall_flip_tracking/mujoco.yaml index 1dd234f91..b843297b7 100644 --- a/conf/ppo/task/g1_wall_flip_tracking/mujoco.yaml +++ b/conf/ppo/task/g1_wall_flip_tracking/mujoco.yaml @@ -1,80 +1,7 @@ # @package _global_ +defaults: + - /task/g1_wall_flip_tracking/base + - _self_ training: task_name: G1WallFlipTracking sim_backend: mujoco - play_steps: 1000 -algo: - num_envs: 1024 - max_iterations: 20000 - save_interval: 500 - empirical_normalization: true - obs_groups: - actor: - - actor - critic: - - critic - algorithm: - entropy_coef: 0.005 - desired_kl: 0.01 -env: - sampling_mode: start - truncate_on_clip_end: false - sim_dt: 0.005 - control_config: - action_scale: - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - anchor_pos_z_threshold: 0.5 - ee_body_pos_z_threshold: 0.5 - terminate_on_undesired_contacts: true - noise_config: - level: 0.0 -reward: - scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 2.0 - motion_body_ori: 1.5 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_ee_body_pos_z: 2.0 - motion_joint_pos: 0.5 - motion_joint_vel: 0.25 - action_rate_l2: -0.005 - joint_limit: -10.0 - undesired_contacts: -0.1 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 diff --git a/conf/ppo/task/g1_wall_flip_tracking/mujoco_play.yaml b/conf/ppo/task/g1_wall_flip_tracking/mujoco_play.yaml new file mode 100644 index 000000000..55856ab01 --- /dev/null +++ b/conf/ppo/task/g1_wall_flip_tracking/mujoco_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/g1_wall_flip_tracking/mujoco + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go1_joystick_flat/base.yaml b/conf/ppo/task/go1_joystick_flat/base.yaml new file mode 100644 index 000000000..ffef37a33 --- /dev/null +++ b/conf/ppo/task/go1_joystick_flat/base.yaml @@ -0,0 +1,32 @@ +# @package _global_ +# Shared cross-backend config for go1_joystick_flat (mujoco/motrix). See AGENTS.md (Sim2Sim). + +training: + play_steps: 500 + play_env_num: 16 + cam_tracking: true + cam_tracking_env_idx: 0 + cam_tracking_extra_envs: 9 +interactive: + action_mode: policy + policy_obs_mode: auto + camera_follow_body: true + use_env_visual_model: false +algo: + num_envs: 1024 + max_iterations: 151 + obs_groups: + actor: + - actor +reward: + scales: + tracking_lin_vel: 1.0 + tracking_ang_vel: 0.2 + lin_vel_z: -5.0 + ang_vel_xy: -0.1 + base_height: -100.0 + action_rate: -0.005 + similar_to_default: -0.1 + swing_feet_z: 4.0 + tracking_sigma: 0.25 + base_height_target: 0.3 diff --git a/conf/ppo/task/go1_joystick_flat/motrix.yaml b/conf/ppo/task/go1_joystick_flat/motrix.yaml index b699946ab..07938420b 100644 --- a/conf/ppo/task/go1_joystick_flat/motrix.yaml +++ b/conf/ppo/task/go1_joystick_flat/motrix.yaml @@ -1,45 +1,24 @@ # @package _global_ +defaults: + - /task/go1_joystick_flat/base + - _self_ + training: task_name: Go1JoystickFlat sim_backend: motrix - play_steps: 500 - play_env_num: 16 - cam_tracking: true - cam_tracking_env_idx: 0 - cam_tracking_extra_envs: 9 - -interactive: - action_mode: policy - policy_obs_mode: auto - camera_follow_body: true - use_env_visual_model: false - algo: - num_envs: 1024 - max_iterations: 151 - obs_groups: - actor: - - actor empirical_normalization: true policy: init_noise_std: 0.5 algorithm: - learning_rate: 3.0e-4 - entropy_coef: 1.0e-3 + learning_rate: 0.0003 + entropy_coef: 0.001 env: commands: vel_limit: - - [0.5, 0.0, 0.0] - - [0.5, 0.0, 0.0] -reward: - scales: - tracking_lin_vel: 1.0 - tracking_ang_vel: 0.2 - lin_vel_z: -5.0 - ang_vel_xy: -0.1 - base_height: -100.0 - action_rate: -0.005 - similar_to_default: -0.1 - swing_feet_z: 4.0 - tracking_sigma: 0.25 - base_height_target: 0.3 + - - 0.5 + - 0.0 + - 0.0 + - - 0.5 + - 0.0 + - 0.0 diff --git a/conf/ppo/task/go1_joystick_flat/motrix_play.yaml b/conf/ppo/task/go1_joystick_flat/motrix_play.yaml new file mode 100644 index 000000000..949a6c7e1 --- /dev/null +++ b/conf/ppo/task/go1_joystick_flat/motrix_play.yaml @@ -0,0 +1,12 @@ +# @package _global_ +defaults: + - /task/go1_joystick_flat/motrix + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go1_joystick_flat/mujoco.yaml b/conf/ppo/task/go1_joystick_flat/mujoco.yaml index e6edb90b6..afab50df6 100644 --- a/conf/ppo/task/go1_joystick_flat/mujoco.yaml +++ b/conf/ppo/task/go1_joystick_flat/mujoco.yaml @@ -1,36 +1,12 @@ # @package _global_ +defaults: + - /task/go1_joystick_flat/base + - _self_ + training: task_name: Go1JoystickFlat sim_backend: mujoco - play_steps: 500 - play_env_num: 16 render_spacing: 0.0 - cam_tracking: true - cam_tracking_env_idx: 0 - cam_tracking_extra_envs: 9 - -interactive: - action_mode: policy - policy_obs_mode: auto - camera_follow_body: true - use_env_visual_model: false - -algo: - num_envs: 1024 - max_iterations: 151 - obs_groups: - actor: - - actor reward: scales: - tracking_lin_vel: 1.0 - tracking_ang_vel: 0.2 - lin_vel_z: -5.0 - ang_vel_xy: -0.1 - base_height: -100.0 - action_rate: -0.005 - similar_to_default: -0.1 contact: 0.24 - swing_feet_z: 4.0 - tracking_sigma: 0.25 - base_height_target: 0.3 diff --git a/conf/ppo/task/go1_joystick_flat/mujoco_play.yaml b/conf/ppo/task/go1_joystick_flat/mujoco_play.yaml new file mode 100644 index 000000000..6f5fab2c4 --- /dev/null +++ b/conf/ppo/task/go1_joystick_flat/mujoco_play.yaml @@ -0,0 +1,12 @@ +# @package _global_ +defaults: + - /task/go1_joystick_flat/mujoco + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go1_joystick_rough/base.yaml b/conf/ppo/task/go1_joystick_rough/base.yaml new file mode 100644 index 000000000..779562693 --- /dev/null +++ b/conf/ppo/task/go1_joystick_rough/base.yaml @@ -0,0 +1,131 @@ +# @package _global_ +# Shared cross-backend config for go1_joystick_rough (mujoco/motrix). See AGENTS.md (Sim2Sim). + +training: + play_steps: 500 + play_env_num: 16 + cam_tracking: true + cam_tracking_env_idx: 0 + cam_tracking_extra_envs: 9 +interactive: + action_mode: policy + policy_obs_mode: auto + camera_follow_body: true + use_env_visual_model: false +algo: + num_envs: 2048 + num_steps_per_env: 24 + max_iterations: 1000 + empirical_normalization: false + obs_groups: + actor: + - actor + critic: + - critic + policy: + init_noise_std: 1.0 + algorithm: + learning_rate: 0.001 + entropy_coef: 0.01 +env: + control_config: + action_scale: 0.25 + hip_action_scale: 0.125 + non_hip_action_scale: 0.25 + clip_actions: 100.0 + commands: + vel_limit: + - - -1.0 + - -1.0 + - -1.0 + - - 1.0 + - 1.0 + - 1.0 + resampling_time: 10.0 + heading_command: true + heading_range: + - -3.141592653589793 + - 3.141592653589793 + rel_standing_envs: 0.1 + terrain_curriculum: + enabled: false + scene: + model_file: src/unilab/assets/robots/go1/go1_mujoco.xml + fragment_files: + - src/unilab/assets/robots/go1/locomotion_task.xml + terrain: + hfield_name: terrain_hfield + geom_name: floor + generator: + seed: 42 + curriculum: false + size: + - 8.0 + - 8.0 + num_rows: 6 + num_cols: 6 + border_width: 20.0 + terrain_scan: + enabled: true + geom_name: floor + termination_config: + terrain_out_of_bounds: true + terrain_distance_buffer: 3.0 + domain_rand: + randomize_base_mass: true + added_mass_range: + - -1.0 + - 3.0 + random_com: true + randomize_kp: true + kp_multiplier_range: + - 0.5 + - 2.0 + randomize_kd: true + kd_multiplier_range: + - 0.5 + - 2.0 + push_robots: true + push_interval: 625 + max_force: + - 1.0 + - 1.0 + - 0.5 +reward: + scales: + lin_vel_z: -2.0 + ang_vel_xy: -0.05 + joint_torques_l2: -2.5e-05 + joint_acc_l2: -2.5e-07 + joint_power: -2.0e-05 + stand_still: -2.0 + hip_pos: -0.5 + joint_pos_penalty: -1.0 + joint_mirror: -0.05 + action_rate: -0.01 + undesired_contacts: -1.0 + contact_forces: -0.00015 + tracking_lin_vel: 3.0 + tracking_ang_vel: 1.5 + feet_air_time: 0.5 + feet_air_time_variance: -1.0 + feet_contact_without_cmd: 0.1 + feet_slide: -0.1 + feet_height_body: -5.0 + feet_gait: 0.5 + upward: 1.0 + tracking_sigma: 0.25 + base_height_target: 0.33 + stand_still_command_threshold: 0.1 + joint_pos_penalty_stand_still_scale: 5.0 + joint_pos_penalty_velocity_threshold: 0.5 + joint_pos_penalty_command_threshold: 0.1 + contact_threshold: 1.0 + contact_forces_threshold: 100.0 + feet_air_time_threshold: 0.5 + feet_height_body_target: -0.2 + feet_height_body_tanh_mult: 2.0 + feet_gait_std: 0.7071067811865476 + feet_gait_max_err: 0.2 + feet_gait_velocity_threshold: 0.5 + feet_gait_command_threshold: 0.1 diff --git a/conf/ppo/task/go1_joystick_rough/motrix.yaml b/conf/ppo/task/go1_joystick_rough/motrix.yaml index b24e9ccd6..c382e2f5a 100644 --- a/conf/ppo/task/go1_joystick_rough/motrix.yaml +++ b/conf/ppo/task/go1_joystick_rough/motrix.yaml @@ -1,117 +1,12 @@ # @package _global_ +defaults: + - /task/go1_joystick_rough/base + - _self_ + training: task_name: Go1JoystickRough sim_backend: motrix - play_steps: 500 - play_env_num: 16 - cam_tracking: true - cam_tracking_env_idx: 0 - cam_tracking_extra_envs: 9 - -interactive: - action_mode: policy - policy_obs_mode: auto - camera_follow_body: true - use_env_visual_model: false - -algo: - num_envs: 2048 - num_steps_per_env: 24 - max_iterations: 1000 - empirical_normalization: false - obs_groups: - actor: - - actor - critic: - - critic - policy: - init_noise_std: 1.0 - algorithm: - learning_rate: 1.0e-3 - entropy_coef: 1.0e-2 - env: render_offset_mode: zero - control_config: - action_scale: 0.25 - hip_action_scale: 0.125 - non_hip_action_scale: 0.25 - clip_actions: 100.0 - commands: - vel_limit: [[-1.0, -1.0, -1.0], [1.0, 1.0, 1.0]] - resampling_time: 10.0 - heading_command: true - heading_range: [-3.141592653589793, 3.141592653589793] - rel_standing_envs: 0.1 - terrain_curriculum: - enabled: false scene: model_file: src/unilab/assets/robots/go1/go1.xml - fragment_files: - - src/unilab/assets/robots/go1/locomotion_task.xml - terrain: - hfield_name: terrain_hfield - geom_name: floor - generator: - seed: 42 - curriculum: false - size: [8.0, 8.0] - num_rows: 6 - num_cols: 6 - border_width: 20.0 - terrain_scan: - enabled: true - geom_name: floor - termination_config: - terrain_out_of_bounds: true - terrain_distance_buffer: 3.0 - domain_rand: - randomize_base_mass: true - added_mass_range: [-1.0, 3.0] - random_com: true - randomize_kp: true - kp_multiplier_range: [0.5, 2.0] - randomize_kd: true - kd_multiplier_range: [0.5, 2.0] - push_robots: true - push_interval: 625 - max_force: [1.0, 1.0, 0.5] - -reward: - scales: - lin_vel_z: -2.0 - ang_vel_xy: -0.05 - joint_torques_l2: -2.5e-5 - joint_acc_l2: -2.5e-7 - joint_power: -2.0e-5 - stand_still: -2.0 - hip_pos: -0.5 - joint_pos_penalty: -1.0 - joint_mirror: -0.05 - action_rate: -0.01 - undesired_contacts: -1.0 - contact_forces: -1.5e-4 - tracking_lin_vel: 3.0 - tracking_ang_vel: 1.5 - feet_air_time: 0.5 - feet_air_time_variance: -1.0 - feet_contact_without_cmd: 0.1 - feet_slide: -0.1 - feet_height_body: -5.0 - feet_gait: 0.5 - upward: 1.0 - tracking_sigma: 0.25 - base_height_target: 0.33 - stand_still_command_threshold: 0.1 - joint_pos_penalty_stand_still_scale: 5.0 - joint_pos_penalty_velocity_threshold: 0.5 - joint_pos_penalty_command_threshold: 0.1 - contact_threshold: 1.0 - contact_forces_threshold: 100.0 - feet_air_time_threshold: 0.5 - feet_height_body_target: -0.2 - feet_height_body_tanh_mult: 2.0 - feet_gait_std: 0.7071067811865476 - feet_gait_max_err: 0.2 - feet_gait_velocity_threshold: 0.5 - feet_gait_command_threshold: 0.1 diff --git a/conf/ppo/task/go1_joystick_rough/motrix_play.yaml b/conf/ppo/task/go1_joystick_rough/motrix_play.yaml new file mode 100644 index 000000000..68b32511c --- /dev/null +++ b/conf/ppo/task/go1_joystick_rough/motrix_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/go1_joystick_rough/motrix + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go1_joystick_rough/mujoco.yaml b/conf/ppo/task/go1_joystick_rough/mujoco.yaml index 05a43dd17..d45546650 100644 --- a/conf/ppo/task/go1_joystick_rough/mujoco.yaml +++ b/conf/ppo/task/go1_joystick_rough/mujoco.yaml @@ -1,118 +1,11 @@ # @package _global_ +defaults: + - /task/go1_joystick_rough/base + - _self_ + training: task_name: Go1JoystickRough sim_backend: mujoco - play_steps: 500 - play_env_num: 16 render_spacing: 0.0 - cam_tracking: true - cam_tracking_env_idx: 0 - cam_tracking_extra_envs: 9 - -interactive: - action_mode: policy - policy_obs_mode: auto - camera_follow_body: true - use_env_visual_model: false - -algo: - num_envs: 2048 - num_steps_per_env: 24 - max_iterations: 1000 - empirical_normalization: false - obs_groups: - actor: - - actor - critic: - - critic - policy: - init_noise_std: 1.0 - algorithm: - learning_rate: 1.0e-3 - entropy_coef: 1.0e-2 - env: sim_dt: 0.005 - control_config: - action_scale: 0.25 - hip_action_scale: 0.125 - non_hip_action_scale: 0.25 - clip_actions: 100.0 - commands: - vel_limit: [[-1.0, -1.0, -1.0], [1.0, 1.0, 1.0]] - resampling_time: 10.0 - heading_command: true - heading_range: [-3.141592653589793, 3.141592653589793] - rel_standing_envs: 0.1 - terrain_curriculum: - enabled: false - scene: - model_file: src/unilab/assets/robots/go1/go1_mujoco.xml - fragment_files: - - src/unilab/assets/robots/go1/locomotion_task.xml - terrain: - hfield_name: terrain_hfield - geom_name: floor - generator: - seed: 42 - curriculum: false - size: [8.0, 8.0] - num_rows: 6 - num_cols: 6 - border_width: 20.0 - terrain_scan: - enabled: true - geom_name: floor - termination_config: - terrain_out_of_bounds: true - terrain_distance_buffer: 3.0 - domain_rand: - randomize_base_mass: true - added_mass_range: [-1.0, 3.0] - random_com: true - randomize_kp: true - kp_multiplier_range: [0.5, 2.0] - randomize_kd: true - kd_multiplier_range: [0.5, 2.0] - push_robots: true - push_interval: 625 - max_force: [1.0, 1.0, 0.5] - -reward: - scales: - lin_vel_z: -2.0 - ang_vel_xy: -0.05 - joint_torques_l2: -2.5e-5 - joint_acc_l2: -2.5e-7 - joint_power: -2.0e-5 - stand_still: -2.0 - hip_pos: -0.5 - joint_pos_penalty: -1.0 - joint_mirror: -0.05 - action_rate: -0.01 - undesired_contacts: -1.0 - contact_forces: -1.5e-4 - tracking_lin_vel: 3.0 - tracking_ang_vel: 1.5 - feet_air_time: 0.5 - feet_air_time_variance: -1.0 - feet_contact_without_cmd: 0.1 - feet_slide: -0.1 - feet_height_body: -5.0 - feet_gait: 0.5 - upward: 1.0 - tracking_sigma: 0.25 - base_height_target: 0.33 - stand_still_command_threshold: 0.1 - joint_pos_penalty_stand_still_scale: 5.0 - joint_pos_penalty_velocity_threshold: 0.5 - joint_pos_penalty_command_threshold: 0.1 - contact_threshold: 1.0 - contact_forces_threshold: 100.0 - feet_air_time_threshold: 0.5 - feet_height_body_target: -0.2 - feet_height_body_tanh_mult: 2.0 - feet_gait_std: 0.7071067811865476 - feet_gait_max_err: 0.2 - feet_gait_velocity_threshold: 0.5 - feet_gait_command_threshold: 0.1 diff --git a/conf/ppo/task/go1_joystick_rough/mujoco_play.yaml b/conf/ppo/task/go1_joystick_rough/mujoco_play.yaml new file mode 100644 index 000000000..d180360fb --- /dev/null +++ b/conf/ppo/task/go1_joystick_rough/mujoco_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/go1_joystick_rough/mujoco + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2_arm_manip_loco/base.yaml b/conf/ppo/task/go2_arm_manip_loco/base.yaml new file mode 100644 index 000000000..e0e8465ec --- /dev/null +++ b/conf/ppo/task/go2_arm_manip_loco/base.yaml @@ -0,0 +1,149 @@ +# @package _global_ +# Shared cross-backend config for go2_arm_manip_loco (mujoco/motrix). See AGENTS.md (Sim2Sim). + +algo: + num_envs: 4096 + max_iterations: 151 + empirical_normalization: true + obs_groups: + actor: + - actor + policy: + init_noise_std: 0.5 + algorithm: + learning_rate: 0.0003 + entropy_coef: 0.001 + num_mini_batches: 4 +reward: + scales: + tracking_lin_vel: 2.0 + tracking_ang_vel: 0.5 + lin_vel_z: -5.0 + ang_vel_xy: -0.1 + roll: -5.0 + base_height: -100 + similar_to_default: -0.0 + leg_pose: -0.1 + dof_pos_limits: 0.0 + action_rate: -0.005 + torques: 0.0 + energy: 0.0 + dof_vel: 0.0 + dof_acc: 0.0 + stand_still: -0.5 + contact: 0.24 + swing_feet_z: 4.0 + foot_drag: -0.1 + object_distance: 2.0 + object_distance_l2: -0.5 + arm_collision: -1.0 + tracking_sigma: 0.25 + base_height_target: 0.3 + object_sigma: 0.1 +env: + goal_ee: + sphere_l_range: + - 0.3 + - 0.6 + sphere_phi_range: + - -1.2566 + - 1.0472 + sphere_theta_range: + - -2.3562 + - 2.3562 + traj_time_range: + - 1.0 + - 3.0 + hold_time_range: + - 0.5 + - 2.0 + collision_upper_limits: + - 0.3 + - 0.15 + - -0.115 + collision_lower_limits: + - -0.2 + - -0.15 + - -0.515 + underground_limit: -0.57 + num_collision_check_samples: 10 + num_resample_attempts: 10 + default_orn_roll: 0.0 + arm_induced_pitch: 0.78 + delta_orn_r: + - -0.5 + - 0.5 + delta_orn_p: + - -0.5 + - 0.5 + delta_orn_y: + - -0.5 + - 0.5 + init_ee_cart: + - 0.25 + - 0.0 + - 0.25 + control_config: + arm_action_scale: 0.0 + ik: + damping: 0.05 + gain: 1.0 + dq_clip: 0.2 + use_orientation: true + orientation_mode: target + commands: + vel_limit: + - - -0.6 + - -0.4 + - -0.8 + - - 1.0 + - 0.4 + - 0.8 + zero_command_prob: 0.15 + resample_time_s: 4.0 + curriculum: + enable: false + domain_rand: + randomize_base_mass: false + added_mass_range: + - -1.0 + - 1.0 + randomize_body_mass: true + body_mass_multiplier_range: + - 0.9 + - 1.1 + random_com: true + com_offset_x: + - -0.03 + - 0.03 + randomize_gravity: false + randomize_ground_friction: true + ground_friction_multiplier_range: + - 0.8 + - 1.2 + randomize_dof_armature: true + dof_armature_multiplier_range: + - 0.8 + - 1.2 + push_robots: true + push_interval: 500 + max_force: + - 1.2 + - 1.2 + - 0.6 + push_body_name: base + randomize_kp: true + kp_multiplier_range: + - 0.9 + - 1.1 + randomize_kd: true + kd_multiplier_range: + - 0.9 + - 1.1 + arm_stage: + freeze_arm_joints: false + disable_ee_goal_trajectory: false + fixed_ee_goal_cart: + - 0.15 + - 0.0 + - 0.25 diff --git a/conf/ppo/task/go2_arm_manip_loco/motrix.yaml b/conf/ppo/task/go2_arm_manip_loco/motrix.yaml index 1b7047bbd..a29528a7f 100644 --- a/conf/ppo/task/go2_arm_manip_loco/motrix.yaml +++ b/conf/ppo/task/go2_arm_manip_loco/motrix.yaml @@ -1,117 +1,32 @@ # @package _global_ +defaults: + - /task/go2_arm_manip_loco/base + - _self_ + training: task_name: Go2ArmManipLoco sim_backend: motrix algo: - num_envs: 4096 max_iterations: 3000 - empirical_normalization: true - obs_groups: - actor: - - actor - policy: - init_noise_std: 0.5 - algorithm: - learning_rate: 3.0e-4 - entropy_coef: 1.0e-3 - num_mini_batches: 4 play_profile: enabled: true scene: enabled: true source_model_file: src/unilab/assets/robots/go2_arm/scene_flat.xml ground_texture_file: src/unilab/assets/robots/g1/textures/floor.png - skybox_rgb1: [0.90, 0.90, 0.91] - skybox_rgb2: [0.68, 0.68, 0.70] - ground_texrepeat: [0.25, 0.25] -reward: - scales: - # locomotion: tracking - tracking_lin_vel: 2.0 #1.0 - tracking_ang_vel: 0.5 #0.2 - # locomotion: velocity / orientation penalties - lin_vel_z: -5.0 - ang_vel_xy: -0.1 - roll: -5.0 #-2.0 - # locomotion: height / pose - base_height: -100 #-20.0 # Go2+arm body oscillates more with arm mass; -100 conflicts with gait. - similar_to_default: -0.0 # Align with Go2 Joystick (L1). - leg_pose: -0.1 # Kept but unused; replaced by similar_to_default. - dof_pos_limits: 0.0 # Kept but unused; soft limits are not configured. - # locomotion: effort penalties - action_rate: -0.005 - torques: 0.0 # Kept but unused; backend does not expose torques. - energy: 0.0 # Kept but unused. - dof_vel: 0.0 # Kept but unused. - dof_acc: 0.0 # Kept but unused. - # locomotion: gait, aligned with Go2 Joystick - stand_still: -0.5 - contact: 0.24 - swing_feet_z: 4.0 - foot_drag: -0.1 - # manipulation rewards - object_distance: 2.0 - object_distance_l2: -0.5 - # arm collision penalty - arm_collision: -1.0 - tracking_sigma: 0.25 - base_height_target: 0.3 - object_sigma: 0.1 + skybox_rgb1: + - 0.9 + - 0.9 + - 0.91 + skybox_rgb2: + - 0.68 + - 0.68 + - 0.7 + ground_texrepeat: + - 0.25 + - 0.25 env: - goal_ee: - sphere_l_range: [0.3, 0.6] - sphere_phi_range: [-1.2566, 1.0472] - sphere_theta_range: [-2.3562, 2.3562] - traj_time_range: [1.0, 3.0] - hold_time_range: [0.5, 2.0] - collision_upper_limits: [0.3, 0.15, -0.115] - collision_lower_limits: [-0.2, -0.15, -0.515] - underground_limit: -0.57 - num_collision_check_samples: 10 - num_resample_attempts: 10 - default_orn_roll: 0.0 - arm_induced_pitch: 0.78 - delta_orn_r: [-0.5, 0.5] - delta_orn_p: [-0.5, 0.5] - delta_orn_y: [-0.5, 0.5] - init_ee_cart: [0.25, 0.0, 0.25] - control_config: - arm_action_scale: 0.0 - ik: - damping: 0.05 - gain: 1.0 - dq_clip: 0.2 - use_orientation: true - orientation_mode: target # target: track sampled orientation; zero_error: constrain orientation change through Jrot without tracking orientation. - commands: - vel_limit: - - [-0.6, -0.4, -0.8] # Align with the default Go2 Joystick range. - - [1.0, 0.4, 0.8] - zero_command_prob: 0.15 # Explicitly sample command=0 for stable standing. - resample_time_s: 4.0 # Omit for null, which disables mid-episode resampling and matches Joystick. - curriculum: - enable: false # Align with Joystick; no curriculum. domain_rand: - randomize_base_mass: false - added_mass_range: [-1.0, 1.0] - randomize_body_mass: true - body_mass_multiplier_range: [0.9, 1.1] - random_com: true - com_offset_x: [-0.03, 0.03] - randomize_gravity: false - randomize_ground_friction: true - ground_friction_multiplier_range: [0.8, 1.2] randomize_dof_armature: false - dof_armature_multiplier_range: [0.8, 1.2] - push_robots: true - push_interval: 500 - max_force: [1.2, 1.2, 0.6] - push_body_name: base randomize_kp: false - kp_multiplier_range: [0.9, 1.1] randomize_kd: false - kd_multiplier_range: [0.9, 1.1] - arm_stage: - freeze_arm_joints: false - disable_ee_goal_trajectory: false - fixed_ee_goal_cart: [0.15, 0.0, 0.25] diff --git a/conf/ppo/task/go2_arm_manip_loco/motrix_play.yaml b/conf/ppo/task/go2_arm_manip_loco/motrix_play.yaml new file mode 100644 index 000000000..be532cc63 --- /dev/null +++ b/conf/ppo/task/go2_arm_manip_loco/motrix_play.yaml @@ -0,0 +1,8 @@ +# @package _global_ + +defaults: + - /task/go2_arm_manip_loco/motrix + - _self_ + +training: + play_only: true diff --git a/conf/ppo/task/go2_arm_manip_loco/mujoco.yaml b/conf/ppo/task/go2_arm_manip_loco/mujoco.yaml index a5b998089..3573222eb 100644 --- a/conf/ppo/task/go2_arm_manip_loco/mujoco.yaml +++ b/conf/ppo/task/go2_arm_manip_loco/mujoco.yaml @@ -1,108 +1,7 @@ # @package _global_ +defaults: + - /task/go2_arm_manip_loco/base + - _self_ training: task_name: Go2ArmManipLoco sim_backend: mujoco -algo: - num_envs: 4096 - max_iterations: 151 - empirical_normalization: true - obs_groups: - actor: - - actor - policy: - init_noise_std: 0.5 - algorithm: - learning_rate: 3.0e-4 - entropy_coef: 1.0e-3 - num_mini_batches: 4 -reward: - scales: - # locomotion: tracking - tracking_lin_vel: 2.0 #1.0 - tracking_ang_vel: 0.5 #0.2 - # locomotion: velocity / orientation penalties - lin_vel_z: -5.0 - ang_vel_xy: -0.1 - roll: -5.0 #-2.0 - # locomotion: height / pose - base_height: -100 #-20.0 # Go2+arm body oscillates more with arm mass; -100 conflicts with gait. - similar_to_default: -0.0 # Align with Go2 Joystick (L1). - leg_pose: -0.1 # Kept but unused; replaced by similar_to_default. - dof_pos_limits: 0.0 # Kept but unused; soft limits are not configured. - # locomotion: effort penalties - action_rate: -0.005 - torques: 0.0 # Kept but unused; backend does not expose torques. - energy: 0.0 # Kept but unused. - dof_vel: 0.0 # Kept but unused. - dof_acc: 0.0 # Kept but unused. - # locomotion: gait, aligned with Go2 Joystick - stand_still: -0.5 - contact: 0.24 - swing_feet_z: 4.0 - foot_drag: -0.1 - # manipulation rewards - object_distance: 2.0 - object_distance_l2: -0.5 - # arm collision penalty - arm_collision: -1.0 - tracking_sigma: 0.25 - base_height_target: 0.3 - object_sigma: 0.1 -env: - goal_ee: - sphere_l_range: [0.3, 0.6] - sphere_phi_range: [-1.2566, 1.0472] - sphere_theta_range: [-2.3562, 2.3562] - traj_time_range: [1.0, 3.0] - hold_time_range: [0.5, 2.0] - collision_upper_limits: [0.3, 0.15, -0.115] - collision_lower_limits: [-0.2, -0.15, -0.515] - underground_limit: -0.57 - num_collision_check_samples: 10 - num_resample_attempts: 10 - default_orn_roll: 0.0 - arm_induced_pitch: 0.78 - delta_orn_r: [-0.5, 0.5] - delta_orn_p: [-0.5, 0.5] - delta_orn_y: [-0.5, 0.5] - init_ee_cart: [0.25, 0.0, 0.25] - control_config: - arm_action_scale: 0.0 - ik: - damping: 0.05 - gain: 1.0 - dq_clip: 0.2 - use_orientation: true - orientation_mode: target # target: track sampled orientation; zero_error: constrain orientation change through Jrot without tracking orientation. - commands: - vel_limit: - - [-0.6, -0.4, -0.8] # Align with the default Go2 Joystick range. - - [1.0, 0.4, 0.8] - zero_command_prob: 0.15 # Explicitly sample command=0 for stable standing. - resample_time_s: 4.0 # Omit for null, which disables mid-episode resampling and matches Joystick. - curriculum: - enable: false # Align with Joystick; no curriculum. - domain_rand: - randomize_base_mass: false - added_mass_range: [-1.0, 1.0] - randomize_body_mass: true - body_mass_multiplier_range: [0.9, 1.1] - random_com: true - com_offset_x: [-0.03, 0.03] - randomize_gravity: false - randomize_ground_friction: true - ground_friction_multiplier_range: [0.8, 1.2] - randomize_dof_armature: true - dof_armature_multiplier_range: [0.8, 1.2] - push_robots: true - push_interval: 500 - max_force: [1.2, 1.2, 0.6] - push_body_name: base - randomize_kp: true - kp_multiplier_range: [0.9, 1.1] - randomize_kd: true - kd_multiplier_range: [0.9, 1.1] - arm_stage: - freeze_arm_joints: false - disable_ee_goal_trajectory: false - fixed_ee_goal_cart: [0.15, 0.0, 0.25] diff --git a/conf/ppo/task/go2_arm_manip_loco/mujoco_play.yaml b/conf/ppo/task/go2_arm_manip_loco/mujoco_play.yaml new file mode 100644 index 000000000..1c33cc956 --- /dev/null +++ b/conf/ppo/task/go2_arm_manip_loco/mujoco_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/go2_arm_manip_loco/mujoco + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2_joystick_flat/base.yaml b/conf/ppo/task/go2_joystick_flat/base.yaml new file mode 100644 index 000000000..8a69a270f --- /dev/null +++ b/conf/ppo/task/go2_joystick_flat/base.yaml @@ -0,0 +1,28 @@ +# @package _global_ +# Shared cross-backend config for go2_joystick_flat (mujoco/motrix). See AGENTS.md (Sim2Sim). + +algo: + num_envs: 1024 + max_iterations: 151 + empirical_normalization: true + obs_groups: + actor: + - actor + policy: + init_noise_std: 0.5 + algorithm: + learning_rate: 0.0003 + entropy_coef: 0.001 +reward: + scales: + tracking_lin_vel: 1.0 + tracking_ang_vel: 0.2 + lin_vel_z: -5.0 + ang_vel_xy: -0.1 + base_height: -100.0 + action_rate: -0.005 + similar_to_default: -0.1 + contact: 0.24 + swing_feet_z: 4.0 + tracking_sigma: 0.25 + base_height_target: 0.3 diff --git a/conf/ppo/task/go2_joystick_flat/motrix.yaml b/conf/ppo/task/go2_joystick_flat/motrix.yaml index 682935bc5..204b72e45 100644 --- a/conf/ppo/task/go2_joystick_flat/motrix.yaml +++ b/conf/ppo/task/go2_joystick_flat/motrix.yaml @@ -1,37 +1,20 @@ # @package _global_ +defaults: + - /task/go2_joystick_flat/base + - _self_ + training: task_name: Go2JoystickFlat sim_backend: motrix -algo: - num_envs: 1024 - max_iterations: 151 - empirical_normalization: true - obs_groups: - actor: - - actor - policy: - init_noise_std: 0.5 - algorithm: - learning_rate: 3.0e-4 - entropy_coef: 1.0e-3 env: commands: vel_limit: - - [0.5, 0.0, 0.0] - - [0.5, 0.0, 0.0] + - - 0.5 + - 0.0 + - 0.0 + - - 0.5 + - 0.0 + - 0.0 domain_rand: randomize_kp: false randomize_kd: false -reward: - scales: - tracking_lin_vel: 1.0 - tracking_ang_vel: 0.2 - lin_vel_z: -5.0 - ang_vel_xy: -0.1 - base_height: -100.0 - action_rate: -0.005 - similar_to_default: -0.1 - contact: 0.24 - swing_feet_z: 4.0 - tracking_sigma: 0.25 - base_height_target: 0.3 diff --git a/conf/ppo/task/go2_joystick_flat/motrix_play.yaml b/conf/ppo/task/go2_joystick_flat/motrix_play.yaml new file mode 100644 index 000000000..87deb99da --- /dev/null +++ b/conf/ppo/task/go2_joystick_flat/motrix_play.yaml @@ -0,0 +1,12 @@ +# @package _global_ +defaults: + - /task/go2_joystick_flat/motrix + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2_joystick_flat/mujoco.yaml b/conf/ppo/task/go2_joystick_flat/mujoco.yaml index d27260e44..9217af855 100644 --- a/conf/ppo/task/go2_joystick_flat/mujoco.yaml +++ b/conf/ppo/task/go2_joystick_flat/mujoco.yaml @@ -1,29 +1,7 @@ # @package _global_ +defaults: + - /task/go2_joystick_flat/base + - _self_ training: task_name: Go2JoystickFlat sim_backend: mujoco -algo: - num_envs: 1024 - max_iterations: 151 - empirical_normalization: true - obs_groups: - actor: - - actor - policy: - init_noise_std: 0.5 - algorithm: - learning_rate: 3.0e-4 - entropy_coef: 1.0e-3 -reward: - scales: - tracking_lin_vel: 1.0 - tracking_ang_vel: 0.2 - lin_vel_z: -5.0 - ang_vel_xy: -0.1 - base_height: -100.0 - action_rate: -0.005 - similar_to_default: -0.1 - contact: 0.24 - swing_feet_z: 4.0 - tracking_sigma: 0.25 - base_height_target: 0.3 diff --git a/conf/ppo/task/go2_joystick_flat/mujoco_play.yaml b/conf/ppo/task/go2_joystick_flat/mujoco_play.yaml new file mode 100644 index 000000000..a53110c28 --- /dev/null +++ b/conf/ppo/task/go2_joystick_flat/mujoco_play.yaml @@ -0,0 +1,12 @@ +# @package _global_ +defaults: + - /task/go2_joystick_flat/mujoco + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2_joystick_rough/base.yaml b/conf/ppo/task/go2_joystick_rough/base.yaml new file mode 100644 index 000000000..ca0f520d1 --- /dev/null +++ b/conf/ppo/task/go2_joystick_rough/base.yaml @@ -0,0 +1,131 @@ +# @package _global_ +# Shared cross-backend config for go2_joystick_rough (mujoco/motrix). See AGENTS.md (Sim2Sim). + +training: + play_steps: 500 + play_env_num: 16 + cam_tracking: true + cam_tracking_env_idx: 0 + cam_tracking_extra_envs: 9 +interactive: + action_mode: policy + policy_obs_mode: auto + camera_follow_body: true + use_env_visual_model: false +algo: + num_envs: 2048 + num_steps_per_env: 24 + max_iterations: 1500 + empirical_normalization: false + obs_groups: + actor: + - actor + critic: + - critic + policy: + init_noise_std: 1.0 + algorithm: + learning_rate: 0.001 + entropy_coef: 0.01 +env: + control_config: + action_scale: 0.25 + hip_action_scale: 0.125 + non_hip_action_scale: 0.25 + clip_actions: 100.0 + commands: + vel_limit: + - - -1.0 + - -1.0 + - -1.0 + - - 1.0 + - 1.0 + - 1.0 + resampling_time: 10.0 + heading_command: true + heading_range: + - -3.141592653589793 + - 3.141592653589793 + rel_standing_envs: 0.1 + terrain_curriculum: + enabled: false + scene: + model_file: src/unilab/assets/robots/go2/go2_mujoco.xml + fragment_files: + - src/unilab/assets/robots/go2/locomotion_task.xml + terrain: + hfield_name: terrain_hfield + geom_name: floor + generator: + seed: 42 + curriculum: false + size: + - 8.0 + - 8.0 + num_rows: 6 + num_cols: 6 + border_width: 20.0 + terrain_scan: + enabled: true + geom_name: floor + termination_config: + terrain_out_of_bounds: true + terrain_distance_buffer: 3.0 + domain_rand: + randomize_base_mass: true + added_mass_range: + - -1.0 + - 3.0 + random_com: true + randomize_kp: true + kp_multiplier_range: + - 0.5 + - 2.0 + randomize_kd: true + kd_multiplier_range: + - 0.5 + - 2.0 + push_robots: true + push_interval: 625 + max_force: + - 1.0 + - 1.0 + - 0.5 +reward: + scales: + lin_vel_z: -2.0 + ang_vel_xy: -0.05 + joint_torques_l2: -2.5e-05 + joint_acc_l2: -2.5e-07 + joint_power: -2.0e-05 + stand_still: -2.0 + hip_pos: -0.5 + joint_pos_penalty: -1.0 + joint_mirror: -0.05 + action_rate: -0.01 + undesired_contacts: -1.0 + contact_forces: -0.00015 + tracking_lin_vel: 3.0 + tracking_ang_vel: 1.5 + feet_air_time: 0.5 + feet_air_time_variance: -1.0 + feet_contact_without_cmd: 0.1 + feet_slide: -0.1 + feet_height_body: -5.0 + feet_gait: 0.5 + upward: 1.0 + tracking_sigma: 0.25 + base_height_target: 0.3 + stand_still_command_threshold: 0.1 + joint_pos_penalty_stand_still_scale: 5.0 + joint_pos_penalty_velocity_threshold: 0.5 + joint_pos_penalty_command_threshold: 0.1 + contact_threshold: 1.0 + contact_forces_threshold: 100.0 + feet_air_time_threshold: 0.5 + feet_height_body_target: -0.2 + feet_height_body_tanh_mult: 2.0 + feet_gait_std: 0.7071067811865476 + feet_gait_max_err: 0.2 + feet_gait_velocity_threshold: 0.5 + feet_gait_command_threshold: 0.1 diff --git a/conf/ppo/task/go2_joystick_rough/motrix.yaml b/conf/ppo/task/go2_joystick_rough/motrix.yaml index 93db22561..b5cf9e676 100644 --- a/conf/ppo/task/go2_joystick_rough/motrix.yaml +++ b/conf/ppo/task/go2_joystick_rough/motrix.yaml @@ -1,116 +1,14 @@ # @package _global_ +defaults: + - /task/go2_joystick_rough/base + - _self_ + training: task_name: Go2JoystickRough sim_backend: motrix - play_steps: 500 - play_env_num: 16 - cam_tracking: true - cam_tracking_env_idx: 0 - cam_tracking_extra_envs: 9 - -interactive: - action_mode: policy - policy_obs_mode: auto - camera_follow_body: true - use_env_visual_model: false - algo: num_envs: 4096 - num_steps_per_env: 24 - max_iterations: 1500 - empirical_normalization: false - obs_groups: - actor: - - actor - critic: - - critic - policy: - init_noise_std: 1.0 - algorithm: - learning_rate: 1.0e-3 - entropy_coef: 1.0e-2 - env: render_offset_mode: zero - control_config: - action_scale: 0.25 - hip_action_scale: 0.125 - non_hip_action_scale: 0.25 - clip_actions: 100.0 - commands: - vel_limit: [[-1.0, -1.0, -1.0], [1.0, 1.0, 1.0]] - resampling_time: 10.0 - heading_command: true - heading_range: [-3.141592653589793, 3.141592653589793] - rel_standing_envs: 0.1 - terrain_curriculum: - enabled: false scene: model_file: src/unilab/assets/robots/go2/go2.xml - fragment_files: - - src/unilab/assets/robots/go2/locomotion_task.xml - terrain: - hfield_name: terrain_hfield - geom_name: floor - generator: - seed: 42 - curriculum: false - size: [8.0, 8.0] - num_rows: 6 - num_cols: 6 - border_width: 20.0 - terrain_scan: - enabled: true - geom_name: floor - termination_config: - terrain_out_of_bounds: true - terrain_distance_buffer: 3.0 - domain_rand: - randomize_base_mass: true - added_mass_range: [-1.0, 3.0] - random_com: true - randomize_kp: true - kp_multiplier_range: [0.5, 2.0] - randomize_kd: true - kd_multiplier_range: [0.5, 2.0] - push_robots: true - push_interval: 625 - max_force: [1.0, 1.0, 0.5] -reward: - scales: - lin_vel_z: -2.0 - ang_vel_xy: -0.05 - joint_torques_l2: -2.5e-5 - joint_acc_l2: -2.5e-7 - joint_power: -2.0e-5 - stand_still: -2.0 - hip_pos: -0.5 - joint_pos_penalty: -1.0 - joint_mirror: -0.05 - action_rate: -0.01 - undesired_contacts: -1.0 - contact_forces: -1.5e-4 - tracking_lin_vel: 3.0 - tracking_ang_vel: 1.5 - feet_air_time: 0.5 - feet_air_time_variance: -1.0 - feet_contact_without_cmd: 0.1 - feet_slide: -0.1 - feet_height_body: -5.0 - feet_gait: 0.5 - upward: 1.0 - tracking_sigma: 0.25 - base_height_target: 0.3 - stand_still_command_threshold: 0.1 - joint_pos_penalty_stand_still_scale: 5.0 - joint_pos_penalty_velocity_threshold: 0.5 - joint_pos_penalty_command_threshold: 0.1 - contact_threshold: 1.0 - contact_forces_threshold: 100.0 - feet_air_time_threshold: 0.5 - feet_height_body_target: -0.2 - feet_height_body_tanh_mult: 2.0 - feet_gait_std: 0.7071067811865476 - feet_gait_max_err: 0.2 - feet_gait_velocity_threshold: 0.5 - feet_gait_command_threshold: 0.1 diff --git a/conf/ppo/task/go2_joystick_rough/motrix_play.yaml b/conf/ppo/task/go2_joystick_rough/motrix_play.yaml new file mode 100644 index 000000000..d556ac266 --- /dev/null +++ b/conf/ppo/task/go2_joystick_rough/motrix_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/go2_joystick_rough/motrix + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2_joystick_rough/mujoco.yaml b/conf/ppo/task/go2_joystick_rough/mujoco.yaml index 00d8cf446..a5f0411f6 100644 --- a/conf/ppo/task/go2_joystick_rough/mujoco.yaml +++ b/conf/ppo/task/go2_joystick_rough/mujoco.yaml @@ -1,118 +1,11 @@ # @package _global_ +defaults: + - /task/go2_joystick_rough/base + - _self_ + training: task_name: Go2JoystickRough sim_backend: mujoco - play_steps: 500 - play_env_num: 16 render_spacing: 0.0 - cam_tracking: true - cam_tracking_env_idx: 0 - cam_tracking_extra_envs: 9 - -interactive: - action_mode: policy - policy_obs_mode: auto - camera_follow_body: true - use_env_visual_model: false - -algo: - num_envs: 2048 - num_steps_per_env: 24 - max_iterations: 1500 - empirical_normalization: false - obs_groups: - actor: - - actor - critic: - - critic - policy: - init_noise_std: 1.0 - algorithm: - learning_rate: 1.0e-3 - entropy_coef: 1.0e-2 - env: sim_dt: 0.002 - control_config: - action_scale: 0.25 - hip_action_scale: 0.125 - non_hip_action_scale: 0.25 - clip_actions: 100.0 - commands: - vel_limit: [[-1.0, -1.0, -1.0], [1.0, 1.0, 1.0]] - resampling_time: 10.0 - heading_command: true - heading_range: [-3.141592653589793, 3.141592653589793] - rel_standing_envs: 0.1 - terrain_curriculum: - enabled: false - scene: - model_file: src/unilab/assets/robots/go2/go2_mujoco.xml - fragment_files: - - src/unilab/assets/robots/go2/locomotion_task.xml - terrain: - hfield_name: terrain_hfield - geom_name: floor - generator: - seed: 42 - curriculum: false - size: [8.0, 8.0] - num_rows: 6 - num_cols: 6 - border_width: 20.0 - terrain_scan: - enabled: true - geom_name: floor - termination_config: - terrain_out_of_bounds: true - terrain_distance_buffer: 3.0 - domain_rand: - randomize_base_mass: true - added_mass_range: [-1.0, 3.0] - random_com: true - randomize_kp: true - kp_multiplier_range: [0.5, 2.0] - randomize_kd: true - kd_multiplier_range: [0.5, 2.0] - push_robots: true - push_interval: 625 - max_force: [1.0, 1.0, 0.5] - -reward: - scales: - lin_vel_z: -2.0 - ang_vel_xy: -0.05 - joint_torques_l2: -2.5e-5 - joint_acc_l2: -2.5e-7 - joint_power: -2.0e-5 - stand_still: -2.0 - hip_pos: -0.5 - joint_pos_penalty: -1.0 - joint_mirror: -0.05 - action_rate: -0.01 - undesired_contacts: -1.0 - contact_forces: -1.5e-4 - tracking_lin_vel: 3.0 - tracking_ang_vel: 1.5 - feet_air_time: 0.5 - feet_air_time_variance: -1.0 - feet_contact_without_cmd: 0.1 - feet_slide: -0.1 - feet_height_body: -5.0 - feet_gait: 0.5 - upward: 1.0 - tracking_sigma: 0.25 - base_height_target: 0.3 - stand_still_command_threshold: 0.1 - joint_pos_penalty_stand_still_scale: 5.0 - joint_pos_penalty_velocity_threshold: 0.5 - joint_pos_penalty_command_threshold: 0.1 - contact_threshold: 1.0 - contact_forces_threshold: 100.0 - feet_air_time_threshold: 0.5 - feet_height_body_target: -0.2 - feet_height_body_tanh_mult: 2.0 - feet_gait_std: 0.7071067811865476 - feet_gait_max_err: 0.2 - feet_gait_velocity_threshold: 0.5 - feet_gait_command_threshold: 0.1 diff --git a/conf/ppo/task/go2_joystick_rough/mujoco_play.yaml b/conf/ppo/task/go2_joystick_rough/mujoco_play.yaml new file mode 100644 index 000000000..5e0d4b515 --- /dev/null +++ b/conf/ppo/task/go2_joystick_rough/mujoco_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/go2_joystick_rough/mujoco + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2w_joystick_flat/base.yaml b/conf/ppo/task/go2w_joystick_flat/base.yaml new file mode 100644 index 000000000..3b8034c90 --- /dev/null +++ b/conf/ppo/task/go2w_joystick_flat/base.yaml @@ -0,0 +1,49 @@ +# @package _global_ +# Shared cross-backend config for go2w_joystick_flat (mujoco/motrix). See AGENTS.md (Sim2Sim). + +algo: + num_envs: 1024 + max_iterations: 151 + empirical_normalization: true + obs_groups: + actor: + - actor + policy: + init_noise_std: 0.5 + algorithm: + learning_rate: 0.0003 + entropy_coef: 0.001 +env: + commands: + vel_limit: + - - 0.0 + - 0.0 + - -1.0 + - - 1.0 + - 0.0 + - 1.0 + control_config: + action_scale: 0.5 + wheel_action_scale: 10.0 + Kp: 50.0 + Kd: 1.5 + wheel_Kd: 0.5 + domain_rand: + randomize_kp: false + randomize_kd: false +reward: + scales: + tracking_lin_vel: 1.0 + tracking_ang_vel: 0.75 + lin_vel_z: -5.0 + ang_vel_xy: -0.1 + base_height: -100.0 + orientation: -2.0 + action_rate: -0.005 + similar_to_default: -0.5 + torques: -0.0002 + wheel_vel: 0.0 + alive: 0.5 + upward: 1.0 + tracking_sigma: 0.25 + base_height_target: 0.4 diff --git a/conf/ppo/task/go2w_joystick_flat/motrix.yaml b/conf/ppo/task/go2w_joystick_flat/motrix.yaml index a2b4256f7..2a98d54d9 100644 --- a/conf/ppo/task/go2w_joystick_flat/motrix.yaml +++ b/conf/ppo/task/go2w_joystick_flat/motrix.yaml @@ -1,47 +1,10 @@ # @package _global_ +defaults: + - /task/go2w_joystick_flat/base + - _self_ + training: task_name: Go2WJoystickFlat sim_backend: motrix -algo: - num_envs: 1024 - max_iterations: 151 - empirical_normalization: true - obs_groups: - actor: - - actor - policy: - init_noise_std: 0.5 - algorithm: - learning_rate: 3.0e-4 - entropy_coef: 1.0e-3 env: render_offset_mode: zero - commands: - vel_limit: - - [0.0, 0.0, -1.0] - - [1.0, 0.0, 1.0] - control_config: - action_scale: 0.5 - wheel_action_scale: 10.0 - Kp: 50.0 - Kd: 1.5 - wheel_Kd: 0.5 - domain_rand: - randomize_kp: false - randomize_kd: false -reward: - scales: - tracking_lin_vel: 1.0 - tracking_ang_vel: 0.75 - lin_vel_z: -5.0 - ang_vel_xy: -0.1 - base_height: -100.0 - orientation: -2.0 - action_rate: -0.005 - similar_to_default: -0.5 - torques: -0.0002 - wheel_vel: 0.0 - alive: 0.5 - upward: 1.0 - tracking_sigma: 0.25 - base_height_target: 0.4 diff --git a/conf/ppo/task/go2w_joystick_flat/motrix_play.yaml b/conf/ppo/task/go2w_joystick_flat/motrix_play.yaml new file mode 100644 index 000000000..e3d3376fa --- /dev/null +++ b/conf/ppo/task/go2w_joystick_flat/motrix_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/go2w_joystick_flat/motrix + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2w_joystick_flat/mujoco.yaml b/conf/ppo/task/go2w_joystick_flat/mujoco.yaml index 311b6017a..58cd7b523 100644 --- a/conf/ppo/task/go2w_joystick_flat/mujoco.yaml +++ b/conf/ppo/task/go2w_joystick_flat/mujoco.yaml @@ -1,46 +1,7 @@ # @package _global_ +defaults: + - /task/go2w_joystick_flat/base + - _self_ training: task_name: Go2WJoystickFlat sim_backend: mujoco -algo: - num_envs: 1024 - max_iterations: 151 - empirical_normalization: true - obs_groups: - actor: - - actor - policy: - init_noise_std: 0.5 - algorithm: - learning_rate: 3.0e-4 - entropy_coef: 1.0e-3 -env: - commands: - vel_limit: - - [0.0, 0.0, -1.0] - - [1.0, 0.0, 1.0] - control_config: - action_scale: 0.5 - wheel_action_scale: 10.0 - Kp: 50.0 - Kd: 1.5 - wheel_Kd: 0.5 - domain_rand: - randomize_kp: false - randomize_kd: false -reward: - scales: - tracking_lin_vel: 1.0 - tracking_ang_vel: 0.75 - lin_vel_z: -5.0 - ang_vel_xy: -0.1 - base_height: -100.0 - orientation: -2.0 - action_rate: -0.005 - similar_to_default: -0.5 - torques: -0.0002 - wheel_vel: 0.0 - alive: 0.5 - upward: 1.0 - tracking_sigma: 0.25 - base_height_target: 0.4 diff --git a/conf/ppo/task/go2w_joystick_flat/mujoco_play.yaml b/conf/ppo/task/go2w_joystick_flat/mujoco_play.yaml new file mode 100644 index 000000000..e906eb02d --- /dev/null +++ b/conf/ppo/task/go2w_joystick_flat/mujoco_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/go2w_joystick_flat/mujoco + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2w_joystick_rough/base.yaml b/conf/ppo/task/go2w_joystick_rough/base.yaml new file mode 100644 index 000000000..ccd82b628 --- /dev/null +++ b/conf/ppo/task/go2w_joystick_rough/base.yaml @@ -0,0 +1,113 @@ +# @package _global_ +# Shared cross-backend config for go2w_joystick_rough (mujoco/motrix). See AGENTS.md (Sim2Sim). + +training: + play_steps: 500 + play_env_num: 16 + cam_tracking: true + cam_tracking_env_idx: 0 + cam_tracking_extra_envs: 9 +interactive: + action_mode: policy + policy_obs_mode: auto + camera_follow_body: true + use_env_visual_model: false +algo: + num_envs: 2048 + num_steps_per_env: 24 + max_iterations: 1200 + empirical_normalization: false + obs_groups: + actor: + - actor + critic: + - critic +env: + scene: + model_file: src/unilab/assets/robots/go2w/go2w_mujoco.xml + fragment_files: + - src/unilab/assets/robots/go2w/locomotion_task.xml + terrain: + hfield_name: terrain_hfield + geom_name: floor + generator: + seed: 42 + curriculum: false + size: + - 8.0 + - 8.0 + num_rows: 6 + num_cols: 6 + border_width: 20.0 + commands: + vel_limit: + - - -1.0 + - -1.0 + - -1.0 + - - 1.0 + - 1.0 + - 1.0 + resampling_time: 10.0 + heading_command: true + heading_range: + - -3.141592653589793 + - 3.141592653589793 + rel_standing_envs: 0.1 + control_config: + action_scale: 0.25 + hip_action_scale: 0.125 + wheel_action_scale: 5.0 + wheel_Kd: 0.5 + clip_actions: 100.0 + simulate_action_latency: false + terrain_scan: + enabled: true + hfield_name: terrain_hfield + geom_name: floor + termination_config: + terrain_out_of_bounds: true + terrain_distance_buffer: 3.0 + domain_rand: + randomize_base_mass: true + added_mass_range: + - -1.0 + - 3.0 + random_com: true + com_offset_x: + - -0.05 + - 0.05 + randomize_kp: true + kp_multiplier_range: + - 0.5 + - 1.0 + randomize_kd: true + kd_multiplier_range: + - 0.5 + - 1.0 + push_robots: true + push_interval: 625 + max_force: + - 1.0 + - 1.0 + - 0.5 + push_body_name: base_link +reward: + scales: + tracking_lin_vel: 3.0 + tracking_ang_vel: 1.5 + lin_vel_z: -2.0 + ang_vel_xy: -0.05 + orientation: -2.0 + joint_torques_l2: -2.5e-05 + joint_acc_l2: -2.5e-07 + joint_acc_wheel_l2: -2.5e-09 + joint_power: -2.0e-05 + action_rate: -0.01 + stand_still: -2.0 + hip_pos: -2.0 + joint_pos_penalty: -1.0 + joint_mirror: -0.05 + upward: 1.0 + tracking_sigma: 0.25 + base_height_target: 0.4 + only_positive_rewards: false diff --git a/conf/ppo/task/go2w_joystick_rough/motrix.yaml b/conf/ppo/task/go2w_joystick_rough/motrix.yaml index 77fc3dd72..209bb8c9c 100644 --- a/conf/ppo/task/go2w_joystick_rough/motrix.yaml +++ b/conf/ppo/task/go2w_joystick_rough/motrix.yaml @@ -1,95 +1,15 @@ # @package _global_ +defaults: + - /task/go2w_joystick_rough/base + - _self_ + training: task_name: Go2WJoystickRough sim_backend: motrix - play_steps: 500 - play_env_num: 16 - cam_tracking: true - cam_tracking_env_idx: 0 - cam_tracking_extra_envs: 9 - -interactive: - action_mode: policy - policy_obs_mode: auto - camera_follow_body: true - use_env_visual_model: false - -algo: - num_envs: 2048 - num_steps_per_env: 24 - max_iterations: 1200 - empirical_normalization: false - obs_groups: - actor: - - actor - critic: - - critic env: render_offset_mode: zero scene: model_file: src/unilab/assets/robots/go2w/go2w.xml - fragment_files: - - src/unilab/assets/robots/go2w/locomotion_task.xml - terrain: - hfield_name: terrain_hfield - geom_name: floor - generator: - seed: 42 - curriculum: false - size: [8.0, 8.0] - num_rows: 6 - num_cols: 6 - border_width: 20.0 - commands: - vel_limit: [[-1.0, -1.0, -1.0], [1.0, 1.0, 1.0]] - resampling_time: 10.0 - heading_command: true - heading_range: [-3.141592653589793, 3.141592653589793] - rel_standing_envs: 0.1 - control_config: - action_scale: 0.25 - hip_action_scale: 0.125 - wheel_action_scale: 5.0 - wheel_Kd: 0.5 - clip_actions: 100.0 - simulate_action_latency: false - terrain_scan: - enabled: true - hfield_name: terrain_hfield - geom_name: floor - termination_config: - terrain_out_of_bounds: true - terrain_distance_buffer: 3.0 - domain_rand: - randomize_base_mass: true - added_mass_range: [-1.0, 3.0] - random_com: true - com_offset_x: [-0.05, 0.05] - randomize_kp: true - kp_multiplier_range: [0.5, 1.0] - randomize_kd: true - kd_multiplier_range: [0.5, 1.0] - push_robots: true - push_interval: 625 - max_force: [1.0, 1.0, 0.5] - push_body_name: base_link reward: scales: - tracking_lin_vel: 3.0 - tracking_ang_vel: 1.5 - lin_vel_z: -2.0 - ang_vel_xy: -0.05 - orientation: -2.0 - joint_torques_l2: -2.5e-5 - joint_acc_l2: -2.5e-7 - joint_acc_wheel_l2: -2.5e-9 - joint_power: -2.0e-5 - action_rate: -0.01 - stand_still: -2.0 hip_pos: -0.5 - joint_pos_penalty: -1.0 - joint_mirror: -0.05 - upward: 1.0 - tracking_sigma: 0.25 - base_height_target: 0.4 - only_positive_rewards: false diff --git a/conf/ppo/task/go2w_joystick_rough/motrix_play.yaml b/conf/ppo/task/go2w_joystick_rough/motrix_play.yaml new file mode 100644 index 000000000..1df1d2aac --- /dev/null +++ b/conf/ppo/task/go2w_joystick_rough/motrix_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/go2w_joystick_rough/motrix + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2w_joystick_rough/mujoco.yaml b/conf/ppo/task/go2w_joystick_rough/mujoco.yaml index 6f29fc7fe..a90a37c62 100644 --- a/conf/ppo/task/go2w_joystick_rough/mujoco.yaml +++ b/conf/ppo/task/go2w_joystick_rough/mujoco.yaml @@ -1,95 +1,9 @@ # @package _global_ +defaults: + - /task/go2w_joystick_rough/base + - _self_ + training: task_name: Go2WJoystickRough sim_backend: mujoco - play_steps: 500 - play_env_num: 16 render_spacing: 0.0 - cam_tracking: true - cam_tracking_env_idx: 0 - cam_tracking_extra_envs: 9 - -interactive: - action_mode: policy - policy_obs_mode: auto - camera_follow_body: true - use_env_visual_model: false - -algo: - num_envs: 2048 - num_steps_per_env: 24 - max_iterations: 1200 - empirical_normalization: false - obs_groups: - actor: - - actor - critic: - - critic -env: - scene: - model_file: src/unilab/assets/robots/go2w/go2w_mujoco.xml - fragment_files: - - src/unilab/assets/robots/go2w/locomotion_task.xml - terrain: - hfield_name: terrain_hfield - geom_name: floor - generator: - seed: 42 - curriculum: false - size: [8.0, 8.0] - num_rows: 6 - num_cols: 6 - border_width: 20.0 - commands: - vel_limit: [[-1.0, -1.0, -1.0], [1.0, 1.0, 1.0]] - resampling_time: 10.0 - heading_command: true - heading_range: [-3.141592653589793, 3.141592653589793] - rel_standing_envs: 0.1 - control_config: - action_scale: 0.25 - hip_action_scale: 0.125 - wheel_action_scale: 5.0 - wheel_Kd: 0.5 - clip_actions: 100.0 - simulate_action_latency: false - terrain_scan: - enabled: true - hfield_name: terrain_hfield - geom_name: floor - termination_config: - terrain_out_of_bounds: true - terrain_distance_buffer: 3.0 - domain_rand: - randomize_base_mass: true - added_mass_range: [-1.0, 3.0] - random_com: true - com_offset_x: [-0.05, 0.05] - randomize_kp: true - kp_multiplier_range: [0.5, 1.0] - randomize_kd: true - kd_multiplier_range: [0.5, 1.0] - push_robots: true - push_interval: 625 - max_force: [1.0, 1.0, 0.5] - push_body_name: base_link -reward: - scales: - tracking_lin_vel: 3.0 - tracking_ang_vel: 1.5 - lin_vel_z: -2.0 - ang_vel_xy: -0.05 - orientation: -2.0 - joint_torques_l2: -2.5e-5 - joint_acc_l2: -2.5e-7 - joint_acc_wheel_l2: -2.5e-9 - joint_power: -2.0e-5 - action_rate: -0.01 - stand_still: -2.0 - hip_pos: -2.0 - joint_pos_penalty: -1.0 - joint_mirror: -0.05 - upward: 1.0 - tracking_sigma: 0.25 - base_height_target: 0.4 - only_positive_rewards: false diff --git a/conf/ppo/task/go2w_joystick_rough/mujoco_play.yaml b/conf/ppo/task/go2w_joystick_rough/mujoco_play.yaml new file mode 100644 index 000000000..473eb4b36 --- /dev/null +++ b/conf/ppo/task/go2w_joystick_rough/mujoco_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/go2w_joystick_rough/mujoco + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/sharpa_inhand/motrix_play.yaml b/conf/ppo/task/sharpa_inhand/motrix_play.yaml new file mode 100644 index 000000000..e3def6b0c --- /dev/null +++ b/conf/ppo/task/sharpa_inhand/motrix_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/sharpa_inhand/motrix + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/sharpa_inhand/mujoco_play.yaml b/conf/ppo/task/sharpa_inhand/mujoco_play.yaml new file mode 100644 index 000000000..6712b025b --- /dev/null +++ b/conf/ppo/task/sharpa_inhand/mujoco_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/sharpa_inhand/mujoco + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/sharpa_inhand_grasp/motrix_play.yaml b/conf/ppo/task/sharpa_inhand_grasp/motrix_play.yaml new file mode 100644 index 000000000..2fd21b8d9 --- /dev/null +++ b/conf/ppo/task/sharpa_inhand_grasp/motrix_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/sharpa_inhand_grasp/motrix + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/sharpa_inhand_grasp/mujoco_play.yaml b/conf/ppo/task/sharpa_inhand_grasp/mujoco_play.yaml new file mode 100644 index 000000000..08e7019e6 --- /dev/null +++ b/conf/ppo/task/sharpa_inhand_grasp/mujoco_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/sharpa_inhand_grasp/mujoco + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/stewart_balance/motrix_play.yaml b/conf/ppo/task/stewart_balance/motrix_play.yaml new file mode 100644 index 000000000..8025b60c3 --- /dev/null +++ b/conf/ppo/task/stewart_balance/motrix_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/stewart_balance/motrix + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/stewart_balance/mujoco_play.yaml b/conf/ppo/task/stewart_balance/mujoco_play.yaml new file mode 100644 index 000000000..040964c0d --- /dev/null +++ b/conf/ppo/task/stewart_balance/mujoco_play.yaml @@ -0,0 +1,13 @@ +# @package _global_ + +defaults: + - /task/stewart_balance/mujoco + - _self_ + +training: + play_only: true + +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/src/unilab/cli.py b/src/unilab/cli.py index e3b03897e..c5c60de77 100644 --- a/src/unilab/cli.py +++ b/src/unilab/cli.py @@ -233,6 +233,15 @@ def build_command( if not script.is_file(): raise SystemExit(f"Entrypoint script not found: {script}") + # In eval mode, prefer the decoupled render/play config (`_play.yaml`) + # when one exists for this task, so eval automatically uses the play profile + # while train never does. Fall back to the bare backend owner if no `_play` + # variant is present. An explicit --profile always takes precedence. + if mode == "eval" and profile is None: + play_route = build_route(algo, task, sim, "play") + if _owner_yaml_path(play_route, selected_root).is_file(): + route = play_route + owner_yaml = _owner_yaml_path(route, selected_root) if not owner_yaml.is_file(): raise SystemExit( diff --git a/tests/envs/test_stewart.py b/tests/envs/test_stewart.py index 24f3ba640..a6a5e2b78 100644 --- a/tests/envs/test_stewart.py +++ b/tests/envs/test_stewart.py @@ -19,6 +19,7 @@ def _make_env(num_envs: int = 2): + pytest.importorskip("motrixsim", reason="motrixsim not installed") ensure_registries() return registry.make("StewartBalance", sim_backend="motrix", num_envs=num_envs) From e92893c154c549d1618823f1fc03636c24f816b7 Mon Sep 17 00:00:00 2001 From: LeeLeno <1846301824@qq.com> Date: Wed, 1 Jul 2026 16:42:39 +0800 Subject: [PATCH 2/3] =?UTF-8?q?fix=EF=BC=9Av2?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 2 +- README_zh.md | 2 +- conf/appo/task/allegro_inhand/base.yaml | 73 -------- conf/appo/task/allegro_inhand/motrix.yaml | 65 +++++++- .../appo/task/allegro_inhand/motrix_play.yaml | 8 - conf/appo/task/allegro_inhand/mujoco.yaml | 73 +++++++- .../appo/task/allegro_inhand/mujoco_play.yaml | 8 - conf/appo/task/g1_climb_tracking/base.yaml | 71 -------- conf/appo/task/g1_climb_tracking/motrix.yaml | 71 +++++++- .../task/g1_climb_tracking/motrix_play.yaml | 8 - conf/appo/task/g1_climb_tracking/mujoco.yaml | 70 +++++++- .../task/g1_climb_tracking/mujoco_play.yaml | 8 - conf/appo/task/g1_flip_tracking/base.yaml | 92 ---------- conf/appo/task/g1_flip_tracking/motrix.yaml | 92 +++++++++- .../task/g1_flip_tracking/motrix_play.yaml | 8 - conf/appo/task/g1_flip_tracking/mujoco.yaml | 91 +++++++++- .../task/g1_flip_tracking/mujoco_play.yaml | 8 - conf/appo/task/g1_motion_tracking/base.yaml | 29 ---- conf/appo/task/g1_motion_tracking/motrix.yaml | 29 +++- .../task/g1_motion_tracking/motrix_play.yaml | 8 - conf/appo/task/g1_motion_tracking/mujoco.yaml | 34 +++- .../task/g1_motion_tracking/mujoco_play.yaml | 8 - .../appo/task/g1_wall_flip_tracking/base.yaml | 50 ------ .../task/g1_wall_flip_tracking/motrix.yaml | 38 ++++- .../g1_wall_flip_tracking/motrix_play.yaml | 8 - .../task/g1_wall_flip_tracking/mujoco.yaml | 53 +++++- .../g1_wall_flip_tracking/mujoco_play.yaml | 8 - conf/appo/task/go1_joystick_flat/base.yaml | 16 -- conf/appo/task/go1_joystick_flat/motrix.yaml | 31 ++-- .../task/go1_joystick_flat/motrix_play.yaml | 8 - conf/appo/task/go1_joystick_flat/mujoco.yaml | 15 +- .../task/go1_joystick_flat/mujoco_play.yaml | 8 - conf/appo/task/go2_joystick_flat/base.yaml | 19 --- conf/appo/task/go2_joystick_flat/motrix.yaml | 26 ++- .../task/go2_joystick_flat/motrix_play.yaml | 8 - conf/appo/task/go2_joystick_flat/mujoco.yaml | 19 ++- .../task/go2_joystick_flat/mujoco_play.yaml | 8 - conf/appo/task/sharpa_inhand/motrix_play.yaml | 8 - conf/appo/task/sharpa_inhand/mujoco_play.yaml | 8 - .../sac/g1_motion_tracking/motrix_play.yaml | 8 - .../sac/g1_motion_tracking/mujoco_play.yaml | 8 - .../offpolicy/task/sac/g1_walk_flat/base.yaml | 75 --------- .../task/sac/g1_walk_flat/motrix.yaml | 66 +++++++- .../task/sac/g1_walk_flat/motrix_play.yaml | 8 - .../task/sac/g1_walk_flat/mujoco.yaml | 75 ++++++++- .../task/sac/g1_walk_flat/mujoco_play.yaml | 8 - .../task/sac/g1_walk_rough/base.yaml | 75 --------- .../task/sac/g1_walk_rough/motrix.yaml | 66 +++++++- .../task/sac/g1_walk_rough/motrix_play.yaml | 8 - .../task/sac/g1_walk_rough/mujoco.yaml | 75 ++++++++- .../task/sac/g1_walk_rough/mujoco_play.yaml | 8 - conf/ppo/task/allegro_inhand/base.yaml | 58 ------- conf/ppo/task/allegro_inhand/motrix.yaml | 63 ++++++- conf/ppo/task/allegro_inhand/motrix_play.yaml | 13 -- conf/ppo/task/allegro_inhand/mujoco.yaml | 63 ++++++- conf/ppo/task/allegro_inhand/mujoco_play.yaml | 13 -- .../ppo/task/allegro_inhand_grasp/motrix.yaml | 4 + .../allegro_inhand_grasp/motrix_play.yaml | 13 -- .../ppo/task/allegro_inhand_grasp/mujoco.yaml | 4 + .../allegro_inhand_grasp/mujoco_play.yaml | 13 -- conf/ppo/task/g1_box_tracking/base.yaml | 39 ----- conf/ppo/task/g1_box_tracking/motrix.yaml | 45 +++-- .../ppo/task/g1_box_tracking/motrix_play.yaml | 8 - conf/ppo/task/g1_box_tracking/mujoco.yaml | 49 +++++- .../ppo/task/g1_box_tracking/mujoco_play.yaml | 13 -- conf/ppo/task/g1_climb_tracking/base.yaml | 80 --------- conf/ppo/task/g1_climb_tracking/motrix.yaml | 84 +++++++++- .../task/g1_climb_tracking/motrix_play.yaml | 13 -- conf/ppo/task/g1_climb_tracking/mujoco.yaml | 83 ++++++++- .../task/g1_climb_tracking/mujoco_play.yaml | 13 -- conf/ppo/task/g1_flip_tracking/base.yaml | 34 ---- conf/ppo/task/g1_flip_tracking/motrix.yaml | 30 +++- .../task/g1_flip_tracking/motrix_play.yaml | 13 -- conf/ppo/task/g1_flip_tracking/mujoco.yaml | 49 ++++-- .../task/g1_flip_tracking/mujoco_play.yaml | 13 -- conf/ppo/task/g1_motion_tracking/base.yaml | 34 ---- conf/ppo/task/g1_motion_tracking/motrix.yaml | 39 ++++- .../task/g1_motion_tracking/motrix_play.yaml | 8 - conf/ppo/task/g1_motion_tracking/mujoco.yaml | 37 ++++- .../task/g1_motion_tracking/mujoco_play.yaml | 13 -- conf/ppo/task/g1_walk_flat/base.yaml | 13 -- conf/ppo/task/g1_walk_flat/motrix.yaml | 48 +++--- conf/ppo/task/g1_walk_flat/motrix_play.yaml | 13 -- conf/ppo/task/g1_walk_flat/mujoco.yaml | 61 ++++++- conf/ppo/task/g1_walk_flat/mujoco_play.yaml | 13 -- conf/ppo/task/g1_wall_flip_tracking/base.yaml | 80 --------- .../task/g1_wall_flip_tracking/motrix.yaml | 77 ++++++++- .../g1_wall_flip_tracking/motrix_play.yaml | 8 - .../task/g1_wall_flip_tracking/mujoco.yaml | 83 ++++++++- .../g1_wall_flip_tracking/mujoco_play.yaml | 13 -- conf/ppo/task/go1_joystick_flat/base.yaml | 32 ---- conf/ppo/task/go1_joystick_flat/motrix.yaml | 35 +++- .../task/go1_joystick_flat/motrix_play.yaml | 12 -- conf/ppo/task/go1_joystick_flat/mujoco.yaml | 34 +++- .../task/go1_joystick_flat/mujoco_play.yaml | 12 -- conf/ppo/task/go1_joystick_rough/base.yaml | 131 --------------- conf/ppo/task/go1_joystick_rough/motrix.yaml | 134 ++++++++++++++- .../task/go1_joystick_rough/motrix_play.yaml | 13 -- conf/ppo/task/go1_joystick_rough/mujoco.yaml | 134 ++++++++++++++- .../task/go1_joystick_rough/mujoco_play.yaml | 13 -- conf/ppo/task/go2_arm_manip_loco/base.yaml | 149 ----------------- conf/ppo/task/go2_arm_manip_loco/motrix.yaml | 157 ++++++++++++++++-- .../task/go2_arm_manip_loco/motrix_play.yaml | 8 - conf/ppo/task/go2_arm_manip_loco/mujoco.yaml | 153 ++++++++++++++++- .../task/go2_arm_manip_loco/mujoco_play.yaml | 13 -- conf/ppo/task/go2_joystick_flat/base.yaml | 28 ---- conf/ppo/task/go2_joystick_flat/motrix.yaml | 33 +++- .../task/go2_joystick_flat/motrix_play.yaml | 12 -- conf/ppo/task/go2_joystick_flat/mujoco.yaml | 32 +++- .../task/go2_joystick_flat/mujoco_play.yaml | 12 -- conf/ppo/task/go2_joystick_rough/base.yaml | 131 --------------- conf/ppo/task/go2_joystick_rough/motrix.yaml | 132 ++++++++++++++- .../task/go2_joystick_rough/motrix_play.yaml | 13 -- conf/ppo/task/go2_joystick_rough/mujoco.yaml | 134 ++++++++++++++- .../task/go2_joystick_rough/mujoco_play.yaml | 13 -- conf/ppo/task/go2w_joystick_flat/base.yaml | 49 ------ conf/ppo/task/go2w_joystick_flat/motrix.yaml | 57 ++++++- .../task/go2w_joystick_flat/motrix_play.yaml | 13 -- conf/ppo/task/go2w_joystick_flat/mujoco.yaml | 53 +++++- .../task/go2w_joystick_flat/mujoco_play.yaml | 13 -- conf/ppo/task/go2w_joystick_rough/base.yaml | 113 ------------- conf/ppo/task/go2w_joystick_rough/motrix.yaml | 113 ++++++++++++- .../task/go2w_joystick_rough/motrix_play.yaml | 13 -- conf/ppo/task/go2w_joystick_rough/mujoco.yaml | 117 ++++++++++++- .../task/go2w_joystick_rough/mujoco_play.yaml | 13 -- conf/ppo/task/sharpa_inhand/motrix.yaml | 4 + conf/ppo/task/sharpa_inhand/motrix_play.yaml | 13 -- conf/ppo/task/sharpa_inhand/mujoco.yaml | 4 + conf/ppo/task/sharpa_inhand/mujoco_play.yaml | 13 -- conf/ppo/task/sharpa_inhand_grasp/motrix.yaml | 4 + .../task/sharpa_inhand_grasp/motrix_play.yaml | 13 -- conf/ppo/task/sharpa_inhand_grasp/mujoco.yaml | 4 + .../task/sharpa_inhand_grasp/mujoco_play.yaml | 13 -- conf/ppo/task/stewart_balance/motrix.yaml | 4 + .../ppo/task/stewart_balance/motrix_play.yaml | 13 -- conf/ppo/task/stewart_balance/mujoco.yaml | 4 + .../ppo/task/stewart_balance/mujoco_play.yaml | 13 -- src/unilab/cli.py | 9 - tests/training/test_sim2sim_resolver.py | 2 +- uv | 0 140 files changed, 2892 insertions(+), 2364 deletions(-) delete mode 100644 conf/appo/task/allegro_inhand/base.yaml delete mode 100644 conf/appo/task/allegro_inhand/motrix_play.yaml delete mode 100644 conf/appo/task/allegro_inhand/mujoco_play.yaml delete mode 100644 conf/appo/task/g1_climb_tracking/base.yaml delete mode 100644 conf/appo/task/g1_climb_tracking/motrix_play.yaml delete mode 100644 conf/appo/task/g1_climb_tracking/mujoco_play.yaml delete mode 100644 conf/appo/task/g1_flip_tracking/base.yaml delete mode 100644 conf/appo/task/g1_flip_tracking/motrix_play.yaml delete mode 100644 conf/appo/task/g1_flip_tracking/mujoco_play.yaml delete mode 100644 conf/appo/task/g1_motion_tracking/base.yaml delete mode 100644 conf/appo/task/g1_motion_tracking/motrix_play.yaml delete mode 100644 conf/appo/task/g1_motion_tracking/mujoco_play.yaml delete mode 100644 conf/appo/task/g1_wall_flip_tracking/base.yaml delete mode 100644 conf/appo/task/g1_wall_flip_tracking/motrix_play.yaml delete mode 100644 conf/appo/task/g1_wall_flip_tracking/mujoco_play.yaml delete mode 100644 conf/appo/task/go1_joystick_flat/base.yaml delete mode 100644 conf/appo/task/go1_joystick_flat/motrix_play.yaml delete mode 100644 conf/appo/task/go1_joystick_flat/mujoco_play.yaml delete mode 100644 conf/appo/task/go2_joystick_flat/base.yaml delete mode 100644 conf/appo/task/go2_joystick_flat/motrix_play.yaml delete mode 100644 conf/appo/task/go2_joystick_flat/mujoco_play.yaml delete mode 100644 conf/appo/task/sharpa_inhand/motrix_play.yaml delete mode 100644 conf/appo/task/sharpa_inhand/mujoco_play.yaml delete mode 100644 conf/offpolicy/task/sac/g1_motion_tracking/motrix_play.yaml delete mode 100644 conf/offpolicy/task/sac/g1_motion_tracking/mujoco_play.yaml delete mode 100644 conf/offpolicy/task/sac/g1_walk_flat/base.yaml delete mode 100644 conf/offpolicy/task/sac/g1_walk_flat/motrix_play.yaml delete mode 100644 conf/offpolicy/task/sac/g1_walk_flat/mujoco_play.yaml delete mode 100644 conf/offpolicy/task/sac/g1_walk_rough/base.yaml delete mode 100644 conf/offpolicy/task/sac/g1_walk_rough/motrix_play.yaml delete mode 100644 conf/offpolicy/task/sac/g1_walk_rough/mujoco_play.yaml delete mode 100644 conf/ppo/task/allegro_inhand/base.yaml delete mode 100644 conf/ppo/task/allegro_inhand/motrix_play.yaml delete mode 100644 conf/ppo/task/allegro_inhand/mujoco_play.yaml delete mode 100644 conf/ppo/task/allegro_inhand_grasp/motrix_play.yaml delete mode 100644 conf/ppo/task/allegro_inhand_grasp/mujoco_play.yaml delete mode 100644 conf/ppo/task/g1_box_tracking/base.yaml delete mode 100644 conf/ppo/task/g1_box_tracking/motrix_play.yaml delete mode 100644 conf/ppo/task/g1_box_tracking/mujoco_play.yaml delete mode 100644 conf/ppo/task/g1_climb_tracking/base.yaml delete mode 100644 conf/ppo/task/g1_climb_tracking/motrix_play.yaml delete mode 100644 conf/ppo/task/g1_climb_tracking/mujoco_play.yaml delete mode 100644 conf/ppo/task/g1_flip_tracking/base.yaml delete mode 100644 conf/ppo/task/g1_flip_tracking/motrix_play.yaml delete mode 100644 conf/ppo/task/g1_flip_tracking/mujoco_play.yaml delete mode 100644 conf/ppo/task/g1_motion_tracking/base.yaml delete mode 100644 conf/ppo/task/g1_motion_tracking/motrix_play.yaml delete mode 100644 conf/ppo/task/g1_motion_tracking/mujoco_play.yaml delete mode 100644 conf/ppo/task/g1_walk_flat/base.yaml delete mode 100644 conf/ppo/task/g1_walk_flat/motrix_play.yaml delete mode 100644 conf/ppo/task/g1_walk_flat/mujoco_play.yaml delete mode 100644 conf/ppo/task/g1_wall_flip_tracking/base.yaml delete mode 100644 conf/ppo/task/g1_wall_flip_tracking/motrix_play.yaml delete mode 100644 conf/ppo/task/g1_wall_flip_tracking/mujoco_play.yaml delete mode 100644 conf/ppo/task/go1_joystick_flat/base.yaml delete mode 100644 conf/ppo/task/go1_joystick_flat/motrix_play.yaml delete mode 100644 conf/ppo/task/go1_joystick_flat/mujoco_play.yaml delete mode 100644 conf/ppo/task/go1_joystick_rough/base.yaml delete mode 100644 conf/ppo/task/go1_joystick_rough/motrix_play.yaml delete mode 100644 conf/ppo/task/go1_joystick_rough/mujoco_play.yaml delete mode 100644 conf/ppo/task/go2_arm_manip_loco/base.yaml delete mode 100644 conf/ppo/task/go2_arm_manip_loco/motrix_play.yaml delete mode 100644 conf/ppo/task/go2_arm_manip_loco/mujoco_play.yaml delete mode 100644 conf/ppo/task/go2_joystick_flat/base.yaml delete mode 100644 conf/ppo/task/go2_joystick_flat/motrix_play.yaml delete mode 100644 conf/ppo/task/go2_joystick_flat/mujoco_play.yaml delete mode 100644 conf/ppo/task/go2_joystick_rough/base.yaml delete mode 100644 conf/ppo/task/go2_joystick_rough/motrix_play.yaml delete mode 100644 conf/ppo/task/go2_joystick_rough/mujoco_play.yaml delete mode 100644 conf/ppo/task/go2w_joystick_flat/base.yaml delete mode 100644 conf/ppo/task/go2w_joystick_flat/motrix_play.yaml delete mode 100644 conf/ppo/task/go2w_joystick_flat/mujoco_play.yaml delete mode 100644 conf/ppo/task/go2w_joystick_rough/base.yaml delete mode 100644 conf/ppo/task/go2w_joystick_rough/motrix_play.yaml delete mode 100644 conf/ppo/task/go2w_joystick_rough/mujoco_play.yaml delete mode 100644 conf/ppo/task/sharpa_inhand/motrix_play.yaml delete mode 100644 conf/ppo/task/sharpa_inhand/mujoco_play.yaml delete mode 100644 conf/ppo/task/sharpa_inhand_grasp/motrix_play.yaml delete mode 100644 conf/ppo/task/sharpa_inhand_grasp/mujoco_play.yaml delete mode 100644 conf/ppo/task/stewart_balance/motrix_play.yaml delete mode 100644 conf/ppo/task/stewart_balance/mujoco_play.yaml create mode 100644 uv diff --git a/README.md b/README.md index b3455cef8..5058bd966 100644 --- a/README.md +++ b/README.md @@ -132,7 +132,7 @@ uv run eval --algo appo --task go2_joystick_flat --sim motrix --load-run -1 uv run eval --algo appo --task go2_joystick_flat --sim motrix --load-run -1 --render-mode record ``` -This routes through the `go2_joystick_flat/motrix` task owner config and keeps backend selection explicit. In eval mode, the CLI automatically prefers the decoupled play config `/_play` when it exists (train never does); pass `--profile play` to force it. +This routes through the `go2_joystick_flat/motrix` task owner config and keeps backend selection explicit. Each backend owner carries an optional `play_profile` block that layers render-only overrides at eval time (`training.play_only=true`) without affecting training. On macOS / MacBook, the UniLab CLI routes Motrix interactive playback through `mxpython` when needed. Motrix defaults to interactive playback; use `--render-mode record` for headless video export or `--render-mode none` to skip playback. Detailed script-level commands are in the [Training Guide](https://unilabsim.github.io/UniLab-doc/en/2-user_guide/1-training/0-index.html). diff --git a/README_zh.md b/README_zh.md index e60b76001..39a50dc1a 100644 --- a/README_zh.md +++ b/README_zh.md @@ -134,7 +134,7 @@ uv run eval --algo appo --task go2_joystick_flat --sim motrix --load-run -1 uv run eval --algo appo --task go2_joystick_flat --sim motrix --load-run -1 --render-mode record ``` -这会路由到 `go2_joystick_flat/motrix` 任务 owner 配置，并保持后端选择显式化。eval 模式下，CLI 会在解耦的 play 配置 `/_play` 存在时自动优先使用它（train 不会），可用 `--profile play` 强制指定。 +这会路由到 `go2_joystick_flat/motrix` 任务 owner 配置，并保持后端选择显式化。每个后端 owner 带一个可选的 `play_profile` 块，在 eval 时（`training.play_only=true`）叠加仅渲染相关的覆盖，不影响训练。 在 macOS / MacBook 上，UniLab CLI 在需要时会通过 `mxpython` 路由 Motrix 交互式回放。Motrix 默认使用交互式回放；要导出无头视频请使用 `--render-mode record`，要跳过回放请使用 `--render-mode none`。更细的脚本级命令请参阅 [训练指南](https://unilabsim.github.io/UniLab-doc/zh_CN/2-user_guide/1-training/0-index.html)。 diff --git a/conf/appo/task/allegro_inhand/base.yaml b/conf/appo/task/allegro_inhand/base.yaml deleted file mode 100644 index 10a27347d..000000000 --- a/conf/appo/task/allegro_inhand/base.yaml +++ /dev/null @@ -1,73 +0,0 @@ -# @package _global_ -# Shared cross-backend config for allegro_inhand (mujoco/motrix). See AGENTS.md (Sim2Sim). - -training: - play_steps: 200 - render_spacing: 0.5 - cam_distance: 0.75 - cam_lookat: - - 0.0 - - 0.0 - - 0.15 - cam_elevation: -25.0 -algo: - num_envs: 1024 - steps_per_env: 8 - max_iterations: 3000 - save_interval: 500 - algorithm: - value_loss_coef: 4.0 - entropy_coef: 0.01 - learning_rate: 0.001 - desired_kl: 0.025 - adaptive_kl_factor: 2.0 - adaptive_lr_factor: 1.5 - num_learning_epochs: 5 - num_mini_batches: 4 - clip_param: 0.2 - gamma: 0.99 - lam: 0.95 - max_grad_norm: 1.0 - use_clipped_value_loss: true - schedule: adaptive - actor: - hidden_dims: - - 512 - - 256 - - 128 - activation: elu - obs_normalization: true - distribution_cfg: - class_name: rsl_rl.modules.distribution.GaussianDistribution - init_std: 1.0 - std_type: scalar - critic: - hidden_dims: - - 512 - - 256 - - 128 - activation: elu - obs_normalization: true -reward: - scales: - rotate: 1.25 - obj_linvel: -0.3 - pose_diff: -0.3 - torque: -0.1 - work: -2.0 - drop: 0.0 - angvel_clip_min: -0.5 - angvel_clip_max: 0.5 - reset_z_threshold: 0.125 -env: - gen_grasp: false - max_episode_seconds: 20.0 - grasp_cache_path: caches/allegro_grasp_50k.npy - domain_rand: - randomize_base_mass: false - random_com: false - randomize_gravity: false - push_robots: false - joint_noise: 0.0 - ball_vel_noise: 0.0 - ball_z_offset: 0.0 diff --git a/conf/appo/task/allegro_inhand/motrix.yaml b/conf/appo/task/allegro_inhand/motrix.yaml index b2f67f1a7..a582b495d 100644 --- a/conf/appo/task/allegro_inhand/motrix.yaml +++ b/conf/appo/task/allegro_inhand/motrix.yaml @@ -1,20 +1,73 @@ # @package _global_ -defaults: - - /task/allegro_inhand/base - - _self_ - training: - task_name: AllegroInhandRotation - sim_backend: motrix + play_steps: 200 + render_spacing: 0.5 cam_distance: 1.5 cam_lookat: - 0.75 - 0.75 - 0 cam_elevation: -20.0 + task_name: AllegroInhandRotation + sim_backend: motrix algo: num_envs: 16384 + steps_per_env: 8 max_iterations: 201 save_interval: 100 algorithm: + value_loss_coef: 4.0 + entropy_coef: 0.01 + learning_rate: 0.001 desired_kl: 0.02 + adaptive_kl_factor: 2.0 + adaptive_lr_factor: 1.5 + num_learning_epochs: 5 + num_mini_batches: 4 + clip_param: 0.2 + gamma: 0.99 + lam: 0.95 + max_grad_norm: 1.0 + use_clipped_value_loss: true + schedule: adaptive + actor: + hidden_dims: + - 512 + - 256 + - 128 + activation: elu + obs_normalization: true + distribution_cfg: + class_name: rsl_rl.modules.distribution.GaussianDistribution + init_std: 1.0 + std_type: scalar + critic: + hidden_dims: + - 512 + - 256 + - 128 + activation: elu + obs_normalization: true +reward: + scales: + rotate: 1.25 + obj_linvel: -0.3 + pose_diff: -0.3 + torque: -0.1 + work: -2.0 + drop: 0.0 + angvel_clip_min: -0.5 + angvel_clip_max: 0.5 + reset_z_threshold: 0.125 +env: + gen_grasp: false + max_episode_seconds: 20.0 + grasp_cache_path: caches/allegro_grasp_50k.npy + domain_rand: + randomize_base_mass: false + random_com: false + randomize_gravity: false + push_robots: false + joint_noise: 0.0 + ball_vel_noise: 0.0 + ball_z_offset: 0.0 diff --git a/conf/appo/task/allegro_inhand/motrix_play.yaml b/conf/appo/task/allegro_inhand/motrix_play.yaml deleted file mode 100644 index a64bff78b..000000000 --- a/conf/appo/task/allegro_inhand/motrix_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/allegro_inhand/motrix - - _self_ - -training: - play_only: true diff --git a/conf/appo/task/allegro_inhand/mujoco.yaml b/conf/appo/task/allegro_inhand/mujoco.yaml index 222a16aec..f884f40ff 100644 --- a/conf/appo/task/allegro_inhand/mujoco.yaml +++ b/conf/appo/task/allegro_inhand/mujoco.yaml @@ -1,10 +1,75 @@ # @package _global_ -defaults: - - /task/allegro_inhand/base - - _self_ - training: + play_steps: 200 + render_spacing: 0.5 + cam_distance: 0.75 + cam_lookat: + - 0.0 + - 0.0 + - 0.15 + cam_elevation: -25.0 task_name: AllegroInhandRotation sim_backend: mujoco cam_azimuth: 45.0 replay_queue_size: 4 +algo: + num_envs: 1024 + steps_per_env: 8 + max_iterations: 3000 + save_interval: 500 + algorithm: + value_loss_coef: 4.0 + entropy_coef: 0.01 + learning_rate: 0.001 + desired_kl: 0.025 + adaptive_kl_factor: 2.0 + adaptive_lr_factor: 1.5 + num_learning_epochs: 5 + num_mini_batches: 4 + clip_param: 0.2 + gamma: 0.99 + lam: 0.95 + max_grad_norm: 1.0 + use_clipped_value_loss: true + schedule: adaptive + actor: + hidden_dims: + - 512 + - 256 + - 128 + activation: elu + obs_normalization: true + distribution_cfg: + class_name: rsl_rl.modules.distribution.GaussianDistribution + init_std: 1.0 + std_type: scalar + critic: + hidden_dims: + - 512 + - 256 + - 128 + activation: elu + obs_normalization: true +reward: + scales: + rotate: 1.25 + obj_linvel: -0.3 + pose_diff: -0.3 + torque: -0.1 + work: -2.0 + drop: 0.0 + angvel_clip_min: -0.5 + angvel_clip_max: 0.5 + reset_z_threshold: 0.125 +env: + gen_grasp: false + max_episode_seconds: 20.0 + grasp_cache_path: caches/allegro_grasp_50k.npy + domain_rand: + randomize_base_mass: false + random_com: false + randomize_gravity: false + push_robots: false + joint_noise: 0.0 + ball_vel_noise: 0.0 + ball_z_offset: 0.0 diff --git a/conf/appo/task/allegro_inhand/mujoco_play.yaml b/conf/appo/task/allegro_inhand/mujoco_play.yaml deleted file mode 100644 index 99969b4c6..000000000 --- a/conf/appo/task/allegro_inhand/mujoco_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/allegro_inhand/mujoco - - _self_ - -training: - play_only: true diff --git a/conf/appo/task/g1_climb_tracking/base.yaml b/conf/appo/task/g1_climb_tracking/base.yaml deleted file mode 100644 index e071bef77..000000000 --- a/conf/appo/task/g1_climb_tracking/base.yaml +++ /dev/null @@ -1,71 +0,0 @@ -# @package _global_ -# Shared cross-backend config for g1_climb_tracking (mujoco/motrix). See AGENTS.md (Sim2Sim). - -training: - play_steps: 1000 -algo: - num_envs: 1024 - max_iterations: 20000 - save_interval: 500 -env: - sampling_mode: adaptive - truncate_on_clip_end: false - sim_dt: 0.005 - control_config: - action_scale: - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - anchor_pos_z_threshold: 0.3 - ee_body_pos_z_threshold: 0.3 - terminate_on_undesired_contacts: true - noise_config: - level: 0.0 -reward: - scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 2.0 - motion_body_ori: 1.5 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_ee_body_pos_z: 2.0 - motion_joint_pos: 0.5 - motion_joint_vel: 0.25 - action_rate_l2: -0.005 - joint_limit: -10.0 - undesired_contacts: -0.1 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 diff --git a/conf/appo/task/g1_climb_tracking/motrix.yaml b/conf/appo/task/g1_climb_tracking/motrix.yaml index 9077f325d..f90b7aa91 100644 --- a/conf/appo/task/g1_climb_tracking/motrix.yaml +++ b/conf/appo/task/g1_climb_tracking/motrix.yaml @@ -1,8 +1,71 @@ # @package _global_ -defaults: - - /task/g1_climb_tracking/base - - _self_ - training: + play_steps: 1000 task_name: G1ClimbTracking sim_backend: motrix +algo: + num_envs: 1024 + max_iterations: 20000 + save_interval: 500 +env: + sampling_mode: adaptive + truncate_on_clip_end: false + sim_dt: 0.005 + control_config: + action_scale: + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + anchor_pos_z_threshold: 0.3 + ee_body_pos_z_threshold: 0.3 + terminate_on_undesired_contacts: true + noise_config: + level: 0.0 +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 2.0 + motion_body_ori: 1.5 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_ee_body_pos_z: 2.0 + motion_joint_pos: 0.5 + motion_joint_vel: 0.25 + action_rate_l2: -0.005 + joint_limit: -10.0 + undesired_contacts: -0.1 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 diff --git a/conf/appo/task/g1_climb_tracking/motrix_play.yaml b/conf/appo/task/g1_climb_tracking/motrix_play.yaml deleted file mode 100644 index 5ec021233..000000000 --- a/conf/appo/task/g1_climb_tracking/motrix_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/g1_climb_tracking/motrix - - _self_ - -training: - play_only: true diff --git a/conf/appo/task/g1_climb_tracking/mujoco.yaml b/conf/appo/task/g1_climb_tracking/mujoco.yaml index 1914cb9d2..d36baf4ff 100644 --- a/conf/appo/task/g1_climb_tracking/mujoco.yaml +++ b/conf/appo/task/g1_climb_tracking/mujoco.yaml @@ -1,7 +1,71 @@ # @package _global_ -defaults: - - /task/g1_climb_tracking/base - - _self_ training: + play_steps: 1000 task_name: G1ClimbTracking sim_backend: mujoco +algo: + num_envs: 1024 + max_iterations: 20000 + save_interval: 500 +env: + sampling_mode: adaptive + truncate_on_clip_end: false + sim_dt: 0.005 + control_config: + action_scale: + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + anchor_pos_z_threshold: 0.3 + ee_body_pos_z_threshold: 0.3 + terminate_on_undesired_contacts: true + noise_config: + level: 0.0 +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 2.0 + motion_body_ori: 1.5 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_ee_body_pos_z: 2.0 + motion_joint_pos: 0.5 + motion_joint_vel: 0.25 + action_rate_l2: -0.005 + joint_limit: -10.0 + undesired_contacts: -0.1 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 diff --git a/conf/appo/task/g1_climb_tracking/mujoco_play.yaml b/conf/appo/task/g1_climb_tracking/mujoco_play.yaml deleted file mode 100644 index 92c075a73..000000000 --- a/conf/appo/task/g1_climb_tracking/mujoco_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/g1_climb_tracking/mujoco - - _self_ - -training: - play_only: true diff --git a/conf/appo/task/g1_flip_tracking/base.yaml b/conf/appo/task/g1_flip_tracking/base.yaml deleted file mode 100644 index ffe0b96ba..000000000 --- a/conf/appo/task/g1_flip_tracking/base.yaml +++ /dev/null @@ -1,92 +0,0 @@ -# @package _global_ -# Shared cross-backend config for g1_flip_tracking (mujoco/motrix). See AGENTS.md (Sim2Sim). - -training: - play_steps: 1000 -algo: - num_envs: 1024 - steps_per_env: 24 - max_iterations: 3500 - save_interval: 500 - algorithm: - num_learning_epochs: 10 - num_mini_batches: 8 - clip_param: 0.2 - gamma: 0.99 - lam: 0.95 - value_loss_coef: 1.0 - entropy_coef: 0.005 - learning_rate: 0.001 - max_grad_norm: 1.0 - use_clipped_value_loss: true - schedule: adaptive - desired_kl: 0.01 - adaptive_kl_factor: 2.0 - adaptive_lr_factor: 1.5 - optimizer: adam - tau: 1.0 - target_update_freq: 1 - vtrace_clip_rho: 1.0 - vtrace_clip_c: 1.0 - enable_compile: true -env: - sampling_mode: start - truncate_on_clip_end: true - sim_dt: 0.005 - control_config: - action_scale: - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - anchor_pos_z_threshold: 0.5 - ee_body_pos_z_threshold: 0.5 - terminate_on_undesired_contacts: true - noise_config: - level: 0.0 -reward: - scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 2.0 - motion_body_ori: 1.5 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_ee_body_pos_z: 2.0 - motion_joint_pos: 0.0 - motion_joint_vel: 0.0 - action_rate_l2: -0.005 - joint_limit: -10.0 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 diff --git a/conf/appo/task/g1_flip_tracking/motrix.yaml b/conf/appo/task/g1_flip_tracking/motrix.yaml index e3c99fbb5..67ea8e630 100644 --- a/conf/appo/task/g1_flip_tracking/motrix.yaml +++ b/conf/appo/task/g1_flip_tracking/motrix.yaml @@ -1,8 +1,92 @@ # @package _global_ -defaults: - - /task/g1_flip_tracking/base - - _self_ - training: + play_steps: 1000 task_name: G1FlipTracking sim_backend: motrix +algo: + num_envs: 1024 + steps_per_env: 24 + max_iterations: 3500 + save_interval: 500 + algorithm: + num_learning_epochs: 10 + num_mini_batches: 8 + clip_param: 0.2 + gamma: 0.99 + lam: 0.95 + value_loss_coef: 1.0 + entropy_coef: 0.005 + learning_rate: 0.001 + max_grad_norm: 1.0 + use_clipped_value_loss: true + schedule: adaptive + desired_kl: 0.01 + adaptive_kl_factor: 2.0 + adaptive_lr_factor: 1.5 + optimizer: adam + tau: 1.0 + target_update_freq: 1 + vtrace_clip_rho: 1.0 + vtrace_clip_c: 1.0 + enable_compile: true +env: + sampling_mode: start + truncate_on_clip_end: true + sim_dt: 0.005 + control_config: + action_scale: + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + anchor_pos_z_threshold: 0.5 + ee_body_pos_z_threshold: 0.5 + terminate_on_undesired_contacts: true + noise_config: + level: 0.0 +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 2.0 + motion_body_ori: 1.5 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_ee_body_pos_z: 2.0 + motion_joint_pos: 0.0 + motion_joint_vel: 0.0 + action_rate_l2: -0.005 + joint_limit: -10.0 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 diff --git a/conf/appo/task/g1_flip_tracking/motrix_play.yaml b/conf/appo/task/g1_flip_tracking/motrix_play.yaml deleted file mode 100644 index 00fc916f9..000000000 --- a/conf/appo/task/g1_flip_tracking/motrix_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/g1_flip_tracking/motrix - - _self_ - -training: - play_only: true diff --git a/conf/appo/task/g1_flip_tracking/mujoco.yaml b/conf/appo/task/g1_flip_tracking/mujoco.yaml index ebcfe80ee..1389affde 100644 --- a/conf/appo/task/g1_flip_tracking/mujoco.yaml +++ b/conf/appo/task/g1_flip_tracking/mujoco.yaml @@ -1,7 +1,92 @@ # @package _global_ -defaults: - - /task/g1_flip_tracking/base - - _self_ training: + play_steps: 1000 task_name: G1FlipTracking sim_backend: mujoco +algo: + num_envs: 1024 + steps_per_env: 24 + max_iterations: 3500 + save_interval: 500 + algorithm: + num_learning_epochs: 10 + num_mini_batches: 8 + clip_param: 0.2 + gamma: 0.99 + lam: 0.95 + value_loss_coef: 1.0 + entropy_coef: 0.005 + learning_rate: 0.001 + max_grad_norm: 1.0 + use_clipped_value_loss: true + schedule: adaptive + desired_kl: 0.01 + adaptive_kl_factor: 2.0 + adaptive_lr_factor: 1.5 + optimizer: adam + tau: 1.0 + target_update_freq: 1 + vtrace_clip_rho: 1.0 + vtrace_clip_c: 1.0 + enable_compile: true +env: + sampling_mode: start + truncate_on_clip_end: true + sim_dt: 0.005 + control_config: + action_scale: + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + anchor_pos_z_threshold: 0.5 + ee_body_pos_z_threshold: 0.5 + terminate_on_undesired_contacts: true + noise_config: + level: 0.0 +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 2.0 + motion_body_ori: 1.5 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_ee_body_pos_z: 2.0 + motion_joint_pos: 0.0 + motion_joint_vel: 0.0 + action_rate_l2: -0.005 + joint_limit: -10.0 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 diff --git a/conf/appo/task/g1_flip_tracking/mujoco_play.yaml b/conf/appo/task/g1_flip_tracking/mujoco_play.yaml deleted file mode 100644 index fb1db6ec2..000000000 --- a/conf/appo/task/g1_flip_tracking/mujoco_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/g1_flip_tracking/mujoco - - _self_ - -training: - play_only: true diff --git a/conf/appo/task/g1_motion_tracking/base.yaml b/conf/appo/task/g1_motion_tracking/base.yaml deleted file mode 100644 index e9d8ab265..000000000 --- a/conf/appo/task/g1_motion_tracking/base.yaml +++ /dev/null @@ -1,29 +0,0 @@ -# @package _global_ -# Shared cross-backend config for g1_motion_tracking (mujoco/motrix). See AGENTS.md (Sim2Sim). - -training: - play_steps: 1000 -algo: - num_envs: 1024 - max_iterations: 5000 - save_interval: 500 -reward: - scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 1.0 - motion_body_ori: 1.0 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_joint_pos: 0.0 - motion_joint_vel: 0.0 - action_rate_l2: -0.1 - joint_limit: -10.0 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 diff --git a/conf/appo/task/g1_motion_tracking/motrix.yaml b/conf/appo/task/g1_motion_tracking/motrix.yaml index 01043ec56..5765dbc3d 100644 --- a/conf/appo/task/g1_motion_tracking/motrix.yaml +++ b/conf/appo/task/g1_motion_tracking/motrix.yaml @@ -1,8 +1,29 @@ # @package _global_ -defaults: - - /task/g1_motion_tracking/base - - _self_ - training: + play_steps: 1000 task_name: G1MotionTracking sim_backend: motrix +algo: + num_envs: 1024 + max_iterations: 5000 + save_interval: 500 +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 1.0 + motion_body_ori: 1.0 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_joint_pos: 0.0 + motion_joint_vel: 0.0 + action_rate_l2: -0.1 + joint_limit: -10.0 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 diff --git a/conf/appo/task/g1_motion_tracking/motrix_play.yaml b/conf/appo/task/g1_motion_tracking/motrix_play.yaml deleted file mode 100644 index 9cebd0be8..000000000 --- a/conf/appo/task/g1_motion_tracking/motrix_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/g1_motion_tracking/motrix - - _self_ - -training: - play_only: true diff --git a/conf/appo/task/g1_motion_tracking/mujoco.yaml b/conf/appo/task/g1_motion_tracking/mujoco.yaml index 2718d83ea..fa5c20b02 100644 --- a/conf/appo/task/g1_motion_tracking/mujoco.yaml +++ b/conf/appo/task/g1_motion_tracking/mujoco.yaml @@ -1,12 +1,32 @@ # @package _global_ -defaults: - - /task/g1_motion_tracking/base - - _self_ - +training: + play_steps: 1000 + task_name: G1MotionTracking + sim_backend: mujoco algo: + num_envs: 1024 + max_iterations: 5000 + save_interval: 500 algorithm: adaptive_kl_factor: 2.0 adaptive_lr_factor: 1.5 -training: - task_name: G1MotionTracking - sim_backend: mujoco +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 1.0 + motion_body_ori: 1.0 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_joint_pos: 0.0 + motion_joint_vel: 0.0 + action_rate_l2: -0.1 + joint_limit: -10.0 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 diff --git a/conf/appo/task/g1_motion_tracking/mujoco_play.yaml b/conf/appo/task/g1_motion_tracking/mujoco_play.yaml deleted file mode 100644 index a6212d376..000000000 --- a/conf/appo/task/g1_motion_tracking/mujoco_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/g1_motion_tracking/mujoco - - _self_ - -training: - play_only: true diff --git a/conf/appo/task/g1_wall_flip_tracking/base.yaml b/conf/appo/task/g1_wall_flip_tracking/base.yaml deleted file mode 100644 index c0bb67912..000000000 --- a/conf/appo/task/g1_wall_flip_tracking/base.yaml +++ /dev/null @@ -1,50 +0,0 @@ -# @package _global_ -# Shared cross-backend config for g1_wall_flip_tracking (mujoco/motrix). See AGENTS.md (Sim2Sim). - -training: - play_steps: 1000 -algo: - num_envs: 1024 - max_iterations: 7000 - save_interval: 500 - algorithm: - num_learning_epochs: 6 - num_mini_batches: 8 - clip_param: 0.2 - gamma: 0.99 - lam: 0.95 - value_loss_coef: 1.0 - entropy_coef: 0.005 - learning_rate: 0.001 - max_grad_norm: 1.0 - use_clipped_value_loss: true - schedule: adaptive - desired_kl: 0.008 - adaptive_kl_factor: 2.0 - adaptive_lr_factor: 1.5 - optimizer: adam - tau: 1.0 - target_update_freq: 1 - vtrace_clip_rho: 1.0 - vtrace_clip_c: 1.0 - enable_compile: true -reward: - scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 2.0 - motion_body_ori: 1.5 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_joint_pos: 0.0 - motion_joint_vel: 0.0 - action_rate_l2: -0.005 - joint_limit: -10.0 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 diff --git a/conf/appo/task/g1_wall_flip_tracking/motrix.yaml b/conf/appo/task/g1_wall_flip_tracking/motrix.yaml index 495c7b876..06d5e8721 100644 --- a/conf/appo/task/g1_wall_flip_tracking/motrix.yaml +++ b/conf/appo/task/g1_wall_flip_tracking/motrix.yaml @@ -1,20 +1,50 @@ # @package _global_ -defaults: - - /task/g1_wall_flip_tracking/base - - _self_ - training: + play_steps: 1000 task_name: G1WallFlipTracking sim_backend: motrix algo: + num_envs: 1024 max_iterations: 5000 + save_interval: 500 algorithm: num_learning_epochs: 5 num_mini_batches: 4 + clip_param: 0.2 + gamma: 0.99 + lam: 0.95 + value_loss_coef: 1.0 entropy_coef: 0.01 + learning_rate: 0.001 + max_grad_norm: 1.0 + use_clipped_value_loss: true + schedule: adaptive desired_kl: 0.01 + adaptive_kl_factor: 2.0 + adaptive_lr_factor: 1.5 + optimizer: adam + tau: 1.0 + target_update_freq: 1 + vtrace_clip_rho: 1.0 + vtrace_clip_c: 1.0 + enable_compile: true reward: scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 motion_body_pos: 1.0 motion_body_ori: 1.0 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_joint_pos: 0.0 + motion_joint_vel: 0.0 action_rate_l2: -0.1 + joint_limit: -10.0 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 diff --git a/conf/appo/task/g1_wall_flip_tracking/motrix_play.yaml b/conf/appo/task/g1_wall_flip_tracking/motrix_play.yaml deleted file mode 100644 index 78ea34f66..000000000 --- a/conf/appo/task/g1_wall_flip_tracking/motrix_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/g1_wall_flip_tracking/motrix - - _self_ - -training: - play_only: true diff --git a/conf/appo/task/g1_wall_flip_tracking/mujoco.yaml b/conf/appo/task/g1_wall_flip_tracking/mujoco.yaml index 712156a02..62101d6b2 100644 --- a/conf/appo/task/g1_wall_flip_tracking/mujoco.yaml +++ b/conf/appo/task/g1_wall_flip_tracking/mujoco.yaml @@ -1,14 +1,56 @@ # @package _global_ -defaults: - - /task/g1_wall_flip_tracking/base - - _self_ - training: + play_steps: 1000 task_name: G1WallFlipTracking sim_backend: mujoco replay_queue_size: 5 algo: + num_envs: 1024 + max_iterations: 7000 + save_interval: 500 + algorithm: + num_learning_epochs: 6 + num_mini_batches: 8 + clip_param: 0.2 + gamma: 0.99 + lam: 0.95 + value_loss_coef: 1.0 + entropy_coef: 0.005 + learning_rate: 0.001 + max_grad_norm: 1.0 + use_clipped_value_loss: true + schedule: adaptive + desired_kl: 0.008 + adaptive_kl_factor: 2.0 + adaptive_lr_factor: 1.5 + optimizer: adam + tau: 1.0 + target_update_freq: 1 + vtrace_clip_rho: 1.0 + vtrace_clip_c: 1.0 + enable_compile: true steps_per_env: 20 +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 2.0 + motion_body_ori: 1.5 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_joint_pos: 0.0 + motion_joint_vel: 0.0 + action_rate_l2: -0.005 + joint_limit: -10.0 + motion_ee_body_pos_z: 2.0 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 env: sampling_mode: start truncate_on_clip_end: true @@ -49,6 +91,3 @@ env: terminate_on_undesired_contacts: true noise_config: level: 0.0 -reward: - scales: - motion_ee_body_pos_z: 2.0 diff --git a/conf/appo/task/g1_wall_flip_tracking/mujoco_play.yaml b/conf/appo/task/g1_wall_flip_tracking/mujoco_play.yaml deleted file mode 100644 index b2ccd7aac..000000000 --- a/conf/appo/task/g1_wall_flip_tracking/mujoco_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/g1_wall_flip_tracking/mujoco - - _self_ - -training: - play_only: true diff --git a/conf/appo/task/go1_joystick_flat/base.yaml b/conf/appo/task/go1_joystick_flat/base.yaml deleted file mode 100644 index 2f15f0884..000000000 --- a/conf/appo/task/go1_joystick_flat/base.yaml +++ /dev/null @@ -1,16 +0,0 @@ -# @package _global_ -# Shared cross-backend config for go1_joystick_flat (mujoco/motrix). See AGENTS.md (Sim2Sim). - -algo: - max_iterations: 150 -reward: - scales: - tracking_lin_vel: 1.0 - tracking_ang_vel: 0.2 - lin_vel_z: -5.0 - ang_vel_xy: -0.1 - base_height: -100.0 - action_rate: -0.005 - similar_to_default: -0.1 - tracking_sigma: 0.25 - base_height_target: 0.3 diff --git a/conf/appo/task/go1_joystick_flat/motrix.yaml b/conf/appo/task/go1_joystick_flat/motrix.yaml index c3dcebb93..f16211730 100644 --- a/conf/appo/task/go1_joystick_flat/motrix.yaml +++ b/conf/appo/task/go1_joystick_flat/motrix.yaml @@ -1,15 +1,8 @@ # @package _global_ -defaults: - - /task/go1_joystick_flat/base - - _self_ - -training: - task_name: Go1JoystickFlat - sim_backend: motrix algo: + max_iterations: 300 num_envs: 1024 steps_per_env: 24 - max_iterations: 300 actor: distribution_cfg: init_std: 0.5 @@ -17,6 +10,22 @@ algo: learning_rate: 0.0005 entropy_coef: 0.001 desired_kl: 0.008 +reward: + scales: + tracking_lin_vel: 1.0 + tracking_ang_vel: 0.2 + lin_vel_z: -5.0 + ang_vel_xy: -0.1 + base_height: -100.0 + action_rate: -0.015 + similar_to_default: -0.15 + action_smooth: -0.01 + swing_feet_z: 2.0 + tracking_sigma: 0.25 + base_height_target: 0.3 +training: + task_name: Go1JoystickFlat + sim_backend: motrix env: sim_dt: 0.01 commands: @@ -27,9 +36,3 @@ env: - - 0.5 - 0.0 - 0.0 -reward: - scales: - action_rate: -0.015 - action_smooth: -0.01 - similar_to_default: -0.15 - swing_feet_z: 2.0 diff --git a/conf/appo/task/go1_joystick_flat/motrix_play.yaml b/conf/appo/task/go1_joystick_flat/motrix_play.yaml deleted file mode 100644 index ecb36fb63..000000000 --- a/conf/appo/task/go1_joystick_flat/motrix_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/go1_joystick_flat/motrix - - _self_ - -training: - play_only: true diff --git a/conf/appo/task/go1_joystick_flat/mujoco.yaml b/conf/appo/task/go1_joystick_flat/mujoco.yaml index 2e57b1246..5124e32cb 100644 --- a/conf/appo/task/go1_joystick_flat/mujoco.yaml +++ b/conf/appo/task/go1_joystick_flat/mujoco.yaml @@ -1,11 +1,18 @@ # @package _global_ -defaults: - - /task/go1_joystick_flat/base - - _self_ - +algo: + max_iterations: 150 reward: scales: + tracking_lin_vel: 1.0 + tracking_ang_vel: 0.2 + lin_vel_z: -5.0 + ang_vel_xy: -0.1 + base_height: -100.0 + action_rate: -0.005 + similar_to_default: -0.1 contact: 0.24 + tracking_sigma: 0.25 + base_height_target: 0.3 training: task_name: Go1JoystickFlat sim_backend: mujoco diff --git a/conf/appo/task/go1_joystick_flat/mujoco_play.yaml b/conf/appo/task/go1_joystick_flat/mujoco_play.yaml deleted file mode 100644 index a2ead3142..000000000 --- a/conf/appo/task/go1_joystick_flat/mujoco_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/go1_joystick_flat/mujoco - - _self_ - -training: - play_only: true diff --git a/conf/appo/task/go2_joystick_flat/base.yaml b/conf/appo/task/go2_joystick_flat/base.yaml deleted file mode 100644 index ce06048fb..000000000 --- a/conf/appo/task/go2_joystick_flat/base.yaml +++ /dev/null @@ -1,19 +0,0 @@ -# @package _global_ -# Shared cross-backend config for go2_joystick_flat (mujoco/motrix). See AGENTS.md (Sim2Sim). - -algo: - max_iterations: 150 -reward: - scales: - tracking_lin_vel: 1.0 - tracking_ang_vel: 0.2 - lin_vel_z: -5.0 - ang_vel_xy: -0.1 - base_height: -100.0 - action_rate: -0.005 - similar_to_default: -0.1 - alive: 0.0 - contact: 0.24 - swing_feet_z: 4.0 - tracking_sigma: 0.25 - base_height_target: 0.3 diff --git a/conf/appo/task/go2_joystick_flat/motrix.yaml b/conf/appo/task/go2_joystick_flat/motrix.yaml index a5e12f27f..b76cdecc2 100644 --- a/conf/appo/task/go2_joystick_flat/motrix.yaml +++ b/conf/appo/task/go2_joystick_flat/motrix.yaml @@ -1,14 +1,24 @@ # @package _global_ -defaults: - - /task/go2_joystick_flat/base - - _self_ - -training: - task_name: Go2JoystickFlat - sim_backend: motrix algo: + max_iterations: 180 num_envs: 512 steps_per_env: 24 - max_iterations: 180 +reward: + scales: + tracking_lin_vel: 1.0 + tracking_ang_vel: 0.2 + lin_vel_z: -5.0 + ang_vel_xy: -0.1 + base_height: -100.0 + action_rate: -0.005 + similar_to_default: -0.1 + alive: 0.0 + contact: 0.24 + swing_feet_z: 4.0 + tracking_sigma: 0.25 + base_height_target: 0.3 +training: + task_name: Go2JoystickFlat + sim_backend: motrix env: sim_dt: 0.015 diff --git a/conf/appo/task/go2_joystick_flat/motrix_play.yaml b/conf/appo/task/go2_joystick_flat/motrix_play.yaml deleted file mode 100644 index 3d2f98124..000000000 --- a/conf/appo/task/go2_joystick_flat/motrix_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/go2_joystick_flat/motrix - - _self_ - -training: - play_only: true diff --git a/conf/appo/task/go2_joystick_flat/mujoco.yaml b/conf/appo/task/go2_joystick_flat/mujoco.yaml index 9217af855..ee7ee5340 100644 --- a/conf/appo/task/go2_joystick_flat/mujoco.yaml +++ b/conf/appo/task/go2_joystick_flat/mujoco.yaml @@ -1,7 +1,20 @@ # @package _global_ -defaults: - - /task/go2_joystick_flat/base - - _self_ +algo: + max_iterations: 150 +reward: + scales: + tracking_lin_vel: 1.0 + tracking_ang_vel: 0.2 + lin_vel_z: -5.0 + ang_vel_xy: -0.1 + base_height: -100.0 + action_rate: -0.005 + similar_to_default: -0.1 + alive: 0.0 + contact: 0.24 + swing_feet_z: 4.0 + tracking_sigma: 0.25 + base_height_target: 0.3 training: task_name: Go2JoystickFlat sim_backend: mujoco diff --git a/conf/appo/task/go2_joystick_flat/mujoco_play.yaml b/conf/appo/task/go2_joystick_flat/mujoco_play.yaml deleted file mode 100644 index 4708d5243..000000000 --- a/conf/appo/task/go2_joystick_flat/mujoco_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/go2_joystick_flat/mujoco - - _self_ - -training: - play_only: true diff --git a/conf/appo/task/sharpa_inhand/motrix_play.yaml b/conf/appo/task/sharpa_inhand/motrix_play.yaml deleted file mode 100644 index 98779fc8e..000000000 --- a/conf/appo/task/sharpa_inhand/motrix_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/sharpa_inhand/motrix - - _self_ - -training: - play_only: true diff --git a/conf/appo/task/sharpa_inhand/mujoco_play.yaml b/conf/appo/task/sharpa_inhand/mujoco_play.yaml deleted file mode 100644 index 3b23bf337..000000000 --- a/conf/appo/task/sharpa_inhand/mujoco_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/sharpa_inhand/mujoco - - _self_ - -training: - play_only: true diff --git a/conf/offpolicy/task/sac/g1_motion_tracking/motrix_play.yaml b/conf/offpolicy/task/sac/g1_motion_tracking/motrix_play.yaml deleted file mode 100644 index 8fa9f0499..000000000 --- a/conf/offpolicy/task/sac/g1_motion_tracking/motrix_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/sac/g1_motion_tracking/motrix - - _self_ - -training: - play_only: true diff --git a/conf/offpolicy/task/sac/g1_motion_tracking/mujoco_play.yaml b/conf/offpolicy/task/sac/g1_motion_tracking/mujoco_play.yaml deleted file mode 100644 index 10b404ff6..000000000 --- a/conf/offpolicy/task/sac/g1_motion_tracking/mujoco_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/sac/g1_motion_tracking/mujoco - - _self_ - -training: - play_only: true diff --git a/conf/offpolicy/task/sac/g1_walk_flat/base.yaml b/conf/offpolicy/task/sac/g1_walk_flat/base.yaml deleted file mode 100644 index c2dcc00f6..000000000 --- a/conf/offpolicy/task/sac/g1_walk_flat/base.yaml +++ /dev/null @@ -1,75 +0,0 @@ -# @package _global_ -# Shared cross-backend config for g1_walk_flat (mujoco/motrix). See AGENTS.md (Sim2Sim). - -algo: - num_envs: 2048 - learning_starts: 10 - max_iterations: 5000 - save_interval: 1000 - updates_per_step: 8 - use_symmetry: true - algo_params: - alpha_init: 0.001 - target_entropy_ratio: 0.0 -env: - control_config: - action_scale: 1.0 - gait_phase_init_mode: offset_phase - reset_base_qvel_limit: 0.5 - curriculum: - enabled: true - initial_scale: 0.5 - min_scale: 0.5 - max_scale: 1.0 - level_down_threshold: 150.0 - level_up_threshold: 750.0 - degree: 0.001 -reward: - scales: - tracking_lin_vel: 2.0 - tracking_ang_vel: 1.5 - penalty_ang_vel_xy: -1.0 - penalty_orientation: -10.0 - penalty_action_rate: -4.0 - pose: -0.5 - penalty_feet_ori: -20.0 - feet_phase: 5.0 - alive: 10.0 - tracking_sigma: 0.25 - base_height_target: 0.754 - min_base_height: 0.3 - max_tilt_deg: 65.0 - gait_frequency: 1.5 - feet_phase_swing_height: 0.09 - feet_phase_tracking_sigma: 0.04 - close_feet_threshold: 0.15 - pose_weights: - - 0.01 - - 1.0 - - 5.0 - - 0.01 - - 5.0 - - 5.0 - - 0.01 - - 1.0 - - 5.0 - - 0.01 - - 5.0 - - 5.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 diff --git a/conf/offpolicy/task/sac/g1_walk_flat/motrix.yaml b/conf/offpolicy/task/sac/g1_walk_flat/motrix.yaml index a17dc4556..e51ae851d 100644 --- a/conf/offpolicy/task/sac/g1_walk_flat/motrix.yaml +++ b/conf/offpolicy/task/sac/g1_walk_flat/motrix.yaml @@ -1,15 +1,27 @@ # @package _global_ -defaults: - - /task/sac/g1_walk_flat/base - - _self_ - -training: - task_name: G1WalkFlat - sim_backend: motrix algo: + num_envs: 2048 learning_starts: 1 + max_iterations: 5000 + save_interval: 1000 + updates_per_step: 8 use_symmetry: false + algo_params: + alpha_init: 0.001 + target_entropy_ratio: 0.0 env: + control_config: + action_scale: 1.0 + gait_phase_init_mode: offset_phase + reset_base_qvel_limit: 0.5 + curriculum: + enabled: true + initial_scale: 0.5 + min_scale: 0.5 + max_scale: 1.0 + level_down_threshold: 150.0 + level_up_threshold: 750.0 + degree: 0.001 domain_rand: randomize_kp: false randomize_kd: false @@ -24,4 +36,44 @@ reward: penalty_feet_ori: -5.0 feet_phase: 6.0 alive: 12.0 + tracking_sigma: 0.25 + base_height_target: 0.754 + min_base_height: 0.3 + max_tilt_deg: 65.0 + gait_frequency: 1.5 + feet_phase_swing_height: 0.09 feet_phase_tracking_sigma: 0.008 + close_feet_threshold: 0.15 + pose_weights: + - 0.01 + - 1.0 + - 5.0 + - 0.01 + - 5.0 + - 5.0 + - 0.01 + - 1.0 + - 5.0 + - 0.01 + - 5.0 + - 5.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 +training: + task_name: G1WalkFlat + sim_backend: motrix diff --git a/conf/offpolicy/task/sac/g1_walk_flat/motrix_play.yaml b/conf/offpolicy/task/sac/g1_walk_flat/motrix_play.yaml deleted file mode 100644 index a857ecae4..000000000 --- a/conf/offpolicy/task/sac/g1_walk_flat/motrix_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/sac/g1_walk_flat/motrix - - _self_ - -training: - play_only: true diff --git a/conf/offpolicy/task/sac/g1_walk_flat/mujoco.yaml b/conf/offpolicy/task/sac/g1_walk_flat/mujoco.yaml index e6f385199..dd47efba0 100644 --- a/conf/offpolicy/task/sac/g1_walk_flat/mujoco.yaml +++ b/conf/offpolicy/task/sac/g1_walk_flat/mujoco.yaml @@ -1,9 +1,27 @@ # @package _global_ -defaults: - - /task/sac/g1_walk_flat/base - - _self_ - +algo: + num_envs: 2048 + learning_starts: 10 + max_iterations: 5000 + save_interval: 1000 + updates_per_step: 8 + use_symmetry: true + algo_params: + alpha_init: 0.001 + target_entropy_ratio: 0.0 env: + control_config: + action_scale: 1.0 + gait_phase_init_mode: offset_phase + reset_base_qvel_limit: 0.5 + curriculum: + enabled: true + initial_scale: 0.5 + min_scale: 0.5 + max_scale: 1.0 + level_down_threshold: 150.0 + level_up_threshold: 750.0 + degree: 0.001 noise_config: level: 1.0 scale_gyro: 0.0 @@ -11,6 +29,55 @@ env: scale_joint_angle: 0.01 scale_joint_vel: 0.1 scale_linvel: 0.0 +reward: + scales: + tracking_lin_vel: 2.0 + tracking_ang_vel: 1.5 + penalty_ang_vel_xy: -1.0 + penalty_orientation: -10.0 + penalty_action_rate: -4.0 + pose: -0.5 + penalty_feet_ori: -20.0 + feet_phase: 5.0 + alive: 10.0 + tracking_sigma: 0.25 + base_height_target: 0.754 + min_base_height: 0.3 + max_tilt_deg: 65.0 + gait_frequency: 1.5 + feet_phase_swing_height: 0.09 + feet_phase_tracking_sigma: 0.04 + close_feet_threshold: 0.15 + pose_weights: + - 0.01 + - 1.0 + - 5.0 + - 0.01 + - 5.0 + - 5.0 + - 0.01 + - 1.0 + - 5.0 + - 0.01 + - 5.0 + - 5.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 training: task_name: G1WalkFlat sim_backend: mujoco diff --git a/conf/offpolicy/task/sac/g1_walk_flat/mujoco_play.yaml b/conf/offpolicy/task/sac/g1_walk_flat/mujoco_play.yaml deleted file mode 100644 index 5b65b2b12..000000000 --- a/conf/offpolicy/task/sac/g1_walk_flat/mujoco_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/sac/g1_walk_flat/mujoco - - _self_ - -training: - play_only: true diff --git a/conf/offpolicy/task/sac/g1_walk_rough/base.yaml b/conf/offpolicy/task/sac/g1_walk_rough/base.yaml deleted file mode 100644 index a987eb7a9..000000000 --- a/conf/offpolicy/task/sac/g1_walk_rough/base.yaml +++ /dev/null @@ -1,75 +0,0 @@ -# @package _global_ -# Shared cross-backend config for g1_walk_rough (mujoco/motrix). See AGENTS.md (Sim2Sim). - -algo: - num_envs: 2048 - learning_starts: 10 - max_iterations: 5000 - save_interval: 1000 - updates_per_step: 8 - use_symmetry: true - algo_params: - alpha_init: 0.001 - target_entropy_ratio: 0.0 -env: - control_config: - action_scale: 1.0 - gait_phase_init_mode: offset_phase - reset_base_qvel_limit: 0.5 - curriculum: - enabled: true - initial_scale: 0.5 - min_scale: 0.5 - max_scale: 1.0 - level_down_threshold: 150.0 - level_up_threshold: 750.0 - degree: 0.001 -reward: - scales: - tracking_lin_vel: 2.0 - tracking_ang_vel: 1.5 - penalty_ang_vel_xy: -1.0 - penalty_orientation: -10.0 - penalty_action_rate: -4.0 - pose: -0.5 - penalty_feet_ori: -20.0 - feet_phase: 5.0 - alive: 10.0 - tracking_sigma: 0.25 - base_height_target: 0.754 - min_base_height: 0.3 - max_tilt_deg: 65.0 - gait_frequency: 1.5 - feet_phase_swing_height: 0.09 - feet_phase_tracking_sigma: 0.04 - close_feet_threshold: 0.15 - pose_weights: - - 0.01 - - 1.0 - - 5.0 - - 0.01 - - 5.0 - - 5.0 - - 0.01 - - 1.0 - - 5.0 - - 0.01 - - 5.0 - - 5.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 diff --git a/conf/offpolicy/task/sac/g1_walk_rough/motrix.yaml b/conf/offpolicy/task/sac/g1_walk_rough/motrix.yaml index a37acc1a4..d2d7731b6 100644 --- a/conf/offpolicy/task/sac/g1_walk_rough/motrix.yaml +++ b/conf/offpolicy/task/sac/g1_walk_rough/motrix.yaml @@ -1,15 +1,27 @@ # @package _global_ -defaults: - - /task/sac/g1_walk_rough/base - - _self_ - -training: - task_name: G1WalkRough - sim_backend: motrix algo: + num_envs: 2048 learning_starts: 1 + max_iterations: 5000 + save_interval: 1000 + updates_per_step: 8 use_symmetry: false + algo_params: + alpha_init: 0.001 + target_entropy_ratio: 0.0 env: + control_config: + action_scale: 1.0 + gait_phase_init_mode: offset_phase + reset_base_qvel_limit: 0.5 + curriculum: + enabled: true + initial_scale: 0.5 + min_scale: 0.5 + max_scale: 1.0 + level_down_threshold: 150.0 + level_up_threshold: 750.0 + degree: 0.001 sim_dt: 0.01 domain_rand: randomize_kp: false @@ -25,4 +37,44 @@ reward: penalty_feet_ori: -5.0 feet_phase: 6.0 alive: 12.0 + tracking_sigma: 0.25 + base_height_target: 0.754 + min_base_height: 0.3 + max_tilt_deg: 65.0 + gait_frequency: 1.5 + feet_phase_swing_height: 0.09 feet_phase_tracking_sigma: 0.008 + close_feet_threshold: 0.15 + pose_weights: + - 0.01 + - 1.0 + - 5.0 + - 0.01 + - 5.0 + - 5.0 + - 0.01 + - 1.0 + - 5.0 + - 0.01 + - 5.0 + - 5.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 +training: + task_name: G1WalkRough + sim_backend: motrix diff --git a/conf/offpolicy/task/sac/g1_walk_rough/motrix_play.yaml b/conf/offpolicy/task/sac/g1_walk_rough/motrix_play.yaml deleted file mode 100644 index 06338b01e..000000000 --- a/conf/offpolicy/task/sac/g1_walk_rough/motrix_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/sac/g1_walk_rough/motrix - - _self_ - -training: - play_only: true diff --git a/conf/offpolicy/task/sac/g1_walk_rough/mujoco.yaml b/conf/offpolicy/task/sac/g1_walk_rough/mujoco.yaml index 8dc372210..daaa8c7fd 100644 --- a/conf/offpolicy/task/sac/g1_walk_rough/mujoco.yaml +++ b/conf/offpolicy/task/sac/g1_walk_rough/mujoco.yaml @@ -1,9 +1,27 @@ # @package _global_ -defaults: - - /task/sac/g1_walk_rough/base - - _self_ - +algo: + num_envs: 2048 + learning_starts: 10 + max_iterations: 5000 + save_interval: 1000 + updates_per_step: 8 + use_symmetry: true + algo_params: + alpha_init: 0.001 + target_entropy_ratio: 0.0 env: + control_config: + action_scale: 1.0 + gait_phase_init_mode: offset_phase + reset_base_qvel_limit: 0.5 + curriculum: + enabled: true + initial_scale: 0.5 + min_scale: 0.5 + max_scale: 1.0 + level_down_threshold: 150.0 + level_up_threshold: 750.0 + degree: 0.001 noise_config: level: 1.0 scale_gyro: 0.0 @@ -11,6 +29,55 @@ env: scale_joint_angle: 0.01 scale_joint_vel: 0.1 scale_linvel: 0.0 +reward: + scales: + tracking_lin_vel: 2.0 + tracking_ang_vel: 1.5 + penalty_ang_vel_xy: -1.0 + penalty_orientation: -10.0 + penalty_action_rate: -4.0 + pose: -0.5 + penalty_feet_ori: -20.0 + feet_phase: 5.0 + alive: 10.0 + tracking_sigma: 0.25 + base_height_target: 0.754 + min_base_height: 0.3 + max_tilt_deg: 65.0 + gait_frequency: 1.5 + feet_phase_swing_height: 0.09 + feet_phase_tracking_sigma: 0.04 + close_feet_threshold: 0.15 + pose_weights: + - 0.01 + - 1.0 + - 5.0 + - 0.01 + - 5.0 + - 5.0 + - 0.01 + - 1.0 + - 5.0 + - 0.01 + - 5.0 + - 5.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 training: task_name: G1WalkRough sim_backend: mujoco diff --git a/conf/offpolicy/task/sac/g1_walk_rough/mujoco_play.yaml b/conf/offpolicy/task/sac/g1_walk_rough/mujoco_play.yaml deleted file mode 100644 index 08bdabf81..000000000 --- a/conf/offpolicy/task/sac/g1_walk_rough/mujoco_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/sac/g1_walk_rough/mujoco - - _self_ - -training: - play_only: true diff --git a/conf/ppo/task/allegro_inhand/base.yaml b/conf/ppo/task/allegro_inhand/base.yaml deleted file mode 100644 index 9ede9fd08..000000000 --- a/conf/ppo/task/allegro_inhand/base.yaml +++ /dev/null @@ -1,58 +0,0 @@ -# @package _global_ -# Shared cross-backend config for allegro_inhand (mujoco/motrix). See AGENTS.md (Sim2Sim). - -algo: - num_envs: 16384 - num_steps_per_env: 8 - max_iterations: 201 - obs_groups: - actor: - - policy - critic: - - policy - actor: - class_name: rsl_rl.models.MLPModel - hidden_dims: - - 512 - - 256 - - 128 - activation: elu - obs_normalization: true - distribution_cfg: - class_name: rsl_rl.modules.distribution.GaussianDistribution - init_std: 1.0 - std_type: scalar - critic: - class_name: rsl_rl.models.MLPModel - hidden_dims: - - 512 - - 256 - - 128 - activation: elu - obs_normalization: true - algorithm: - value_loss_coef: 4.0 - desired_kl: 0.02 -reward: - scales: - rotate: 1.25 - obj_linvel: -0.3 - pose_diff: -0.3 - torque: -0.1 - work: -2.0 - drop: 0.0 - angvel_clip_min: -0.5 - angvel_clip_max: 0.5 - reset_z_threshold: 0.125 -env: - gen_grasp: false - max_episode_seconds: 20.0 - grasp_cache_path: caches/allegro_grasp_50k.npy - domain_rand: - randomize_base_mass: false - random_com: false - randomize_gravity: false - push_robots: false - joint_noise: 0.0 - ball_vel_noise: 0.0 - ball_z_offset: 0.0 diff --git a/conf/ppo/task/allegro_inhand/motrix.yaml b/conf/ppo/task/allegro_inhand/motrix.yaml index 7e9186d93..e7b631431 100644 --- a/conf/ppo/task/allegro_inhand/motrix.yaml +++ b/conf/ppo/task/allegro_inhand/motrix.yaml @@ -1,8 +1,63 @@ # @package _global_ -defaults: - - /task/allegro_inhand/base - - _self_ - +algo: + num_envs: 16384 + num_steps_per_env: 8 + max_iterations: 201 + obs_groups: + actor: + - policy + critic: + - policy + actor: + class_name: rsl_rl.models.MLPModel + hidden_dims: + - 512 + - 256 + - 128 + activation: elu + obs_normalization: true + distribution_cfg: + class_name: rsl_rl.modules.distribution.GaussianDistribution + init_std: 1.0 + std_type: scalar + critic: + class_name: rsl_rl.models.MLPModel + hidden_dims: + - 512 + - 256 + - 128 + activation: elu + obs_normalization: true + algorithm: + value_loss_coef: 4.0 + desired_kl: 0.02 +reward: + scales: + rotate: 1.25 + obj_linvel: -0.3 + pose_diff: -0.3 + torque: -0.1 + work: -2.0 + drop: 0.0 + angvel_clip_min: -0.5 + angvel_clip_max: 0.5 + reset_z_threshold: 0.125 +env: + gen_grasp: false + max_episode_seconds: 20.0 + grasp_cache_path: caches/allegro_grasp_50k.npy + domain_rand: + randomize_base_mass: false + random_com: false + randomize_gravity: false + push_robots: false + joint_noise: 0.0 + ball_vel_noise: 0.0 + ball_z_offset: 0.0 training: task_name: AllegroInhandRotation sim_backend: motrix +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/allegro_inhand/motrix_play.yaml b/conf/ppo/task/allegro_inhand/motrix_play.yaml deleted file mode 100644 index ff922b259..000000000 --- a/conf/ppo/task/allegro_inhand/motrix_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/allegro_inhand/motrix - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/allegro_inhand/mujoco.yaml b/conf/ppo/task/allegro_inhand/mujoco.yaml index 031a618a1..f770bdd01 100644 --- a/conf/ppo/task/allegro_inhand/mujoco.yaml +++ b/conf/ppo/task/allegro_inhand/mujoco.yaml @@ -1,8 +1,59 @@ # @package _global_ -defaults: - - /task/allegro_inhand/base - - _self_ - +algo: + num_envs: 16384 + num_steps_per_env: 8 + max_iterations: 201 + obs_groups: + actor: + - policy + critic: + - policy + actor: + class_name: rsl_rl.models.MLPModel + hidden_dims: + - 512 + - 256 + - 128 + activation: elu + obs_normalization: true + distribution_cfg: + class_name: rsl_rl.modules.distribution.GaussianDistribution + init_std: 1.0 + std_type: scalar + critic: + class_name: rsl_rl.models.MLPModel + hidden_dims: + - 512 + - 256 + - 128 + activation: elu + obs_normalization: true + algorithm: + value_loss_coef: 4.0 + desired_kl: 0.02 +reward: + scales: + rotate: 1.25 + obj_linvel: -0.3 + pose_diff: -0.3 + torque: -0.1 + work: -2.0 + drop: 0.0 + angvel_clip_min: -0.5 + angvel_clip_max: 0.5 + reset_z_threshold: 0.125 +env: + gen_grasp: false + max_episode_seconds: 20.0 + grasp_cache_path: caches/allegro_grasp_50k.npy + domain_rand: + randomize_base_mass: false + random_com: false + randomize_gravity: false + push_robots: false + joint_noise: 0.0 + ball_vel_noise: 0.0 + ball_z_offset: 0.0 training: task_name: AllegroInhandRotation sim_backend: mujoco @@ -13,3 +64,7 @@ training: - 0.75 - 0 cam_elevation: -20.0 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/allegro_inhand/mujoco_play.yaml b/conf/ppo/task/allegro_inhand/mujoco_play.yaml deleted file mode 100644 index 63d251085..000000000 --- a/conf/ppo/task/allegro_inhand/mujoco_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/allegro_inhand/mujoco - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/allegro_inhand_grasp/motrix.yaml b/conf/ppo/task/allegro_inhand_grasp/motrix.yaml index 8dfe5c1d4..6b5cdca31 100644 --- a/conf/ppo/task/allegro_inhand_grasp/motrix.yaml +++ b/conf/ppo/task/allegro_inhand_grasp/motrix.yaml @@ -31,3 +31,7 @@ env: push_robots: false ball_vel_noise: 0.0 joint_noise: 0.25 # random sampling of the grasp poses +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/allegro_inhand_grasp/motrix_play.yaml b/conf/ppo/task/allegro_inhand_grasp/motrix_play.yaml deleted file mode 100644 index db0fe057e..000000000 --- a/conf/ppo/task/allegro_inhand_grasp/motrix_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/allegro_inhand_grasp/motrix - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/allegro_inhand_grasp/mujoco.yaml b/conf/ppo/task/allegro_inhand_grasp/mujoco.yaml index 20d565f51..e6e62f2f6 100644 --- a/conf/ppo/task/allegro_inhand_grasp/mujoco.yaml +++ b/conf/ppo/task/allegro_inhand_grasp/mujoco.yaml @@ -31,3 +31,7 @@ env: push_robots: false ball_vel_noise: 0.0 joint_noise: 0.25 # random sampling of the grasp poses +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/allegro_inhand_grasp/mujoco_play.yaml b/conf/ppo/task/allegro_inhand_grasp/mujoco_play.yaml deleted file mode 100644 index 916fabfa3..000000000 --- a/conf/ppo/task/allegro_inhand_grasp/mujoco_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/allegro_inhand_grasp/mujoco - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/g1_box_tracking/base.yaml b/conf/ppo/task/g1_box_tracking/base.yaml deleted file mode 100644 index 706ce0bd2..000000000 --- a/conf/ppo/task/g1_box_tracking/base.yaml +++ /dev/null @@ -1,39 +0,0 @@ -# @package _global_ -# Shared cross-backend config for g1_box_tracking (mujoco/motrix). See AGENTS.md (Sim2Sim). - -training: - play_steps: 1000 -algo: - num_envs: 1024 - max_iterations: 30000 - save_interval: 500 - obs_groups: - actor: - - actor - algorithm: - entropy_coef: 0.005 -reward: - scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 1.0 - motion_body_ori: 1.0 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_joint_pos: 0.0 - motion_joint_vel: 0.0 - action_rate_l2: -0.1 - joint_limit: -10.0 - undesired_contacts: -0.1 - object_global_ref_position_error_exp: 2.0 - object_global_ref_orientation_error_exp: 2.0 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 - std_object_pos: 0.2 - std_object_ori: 0.3 diff --git a/conf/ppo/task/g1_box_tracking/motrix.yaml b/conf/ppo/task/g1_box_tracking/motrix.yaml index b8935f05b..302b734a0 100644 --- a/conf/ppo/task/g1_box_tracking/motrix.yaml +++ b/conf/ppo/task/g1_box_tracking/motrix.yaml @@ -1,26 +1,52 @@ # @package _global_ -defaults: - - /task/g1_box_tracking/base - - _self_ - training: + play_steps: 1000 task_name: G1BoxTracking sim_backend: motrix play_env_num: 16 algo: + num_envs: 1024 max_iterations: 40000 - empirical_normalization: true + save_interval: 500 obs_groups: + actor: + - actor critic: - critic algorithm: entropy_coef: 0.002 desired_kl: 0.01 + empirical_normalization: true noise_config: level: 1.0 scale_joint_angle: 0.01 scale_joint_vel: 1.5 scale_gyro: 0.2 +reward: + scales: + motion_global_root_pos: 1.0 + motion_global_root_ori: 0.5 + motion_body_pos: 1.0 + motion_body_ori: 1.5 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.5 + motion_joint_pos: 0.0 + motion_joint_vel: 0.0 + action_rate_l2: -0.1 + joint_limit: -10.0 + undesired_contacts: -0.1 + object_global_ref_position_error_exp: 4.0 + object_global_ref_orientation_error_exp: 3.0 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 + std_object_pos: 0.12 + std_object_ori: 0.2 play_profile: enabled: true env: @@ -40,12 +66,3 @@ play_profile: ground_texrepeat: - 0.25 - 0.25 -reward: - scales: - motion_global_root_pos: 1.0 - motion_body_ori: 1.5 - motion_body_ang_vel: 1.5 - object_global_ref_position_error_exp: 4.0 - object_global_ref_orientation_error_exp: 3.0 - std_object_pos: 0.12 - std_object_ori: 0.2 diff --git a/conf/ppo/task/g1_box_tracking/motrix_play.yaml b/conf/ppo/task/g1_box_tracking/motrix_play.yaml deleted file mode 100644 index 287c4b354..000000000 --- a/conf/ppo/task/g1_box_tracking/motrix_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/g1_box_tracking/motrix - - _self_ - -training: - play_only: true diff --git a/conf/ppo/task/g1_box_tracking/mujoco.yaml b/conf/ppo/task/g1_box_tracking/mujoco.yaml index b23f81672..aade813dd 100644 --- a/conf/ppo/task/g1_box_tracking/mujoco.yaml +++ b/conf/ppo/task/g1_box_tracking/mujoco.yaml @@ -1,13 +1,48 @@ # @package _global_ -defaults: - - /task/g1_box_tracking/base - - _self_ - +training: + play_steps: 1000 + task_name: G1BoxTracking + sim_backend: mujoco +algo: + num_envs: 1024 + max_iterations: 30000 + save_interval: 500 + obs_groups: + actor: + - actor + algorithm: + entropy_coef: 0.005 +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 1.0 + motion_body_ori: 1.0 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_joint_pos: 0.0 + motion_joint_vel: 0.0 + action_rate_l2: -0.1 + joint_limit: -10.0 + undesired_contacts: -0.1 + object_global_ref_position_error_exp: 2.0 + object_global_ref_orientation_error_exp: 2.0 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 + std_object_pos: 0.2 + std_object_ori: 0.3 env: sim_dt: 0.005 sensor: gyro: pelvis_gyro upvector: pelvis_upvector -training: - task_name: G1BoxTracking - sim_backend: mujoco +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_box_tracking/mujoco_play.yaml b/conf/ppo/task/g1_box_tracking/mujoco_play.yaml deleted file mode 100644 index af69bba5f..000000000 --- a/conf/ppo/task/g1_box_tracking/mujoco_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/g1_box_tracking/mujoco - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/g1_climb_tracking/base.yaml b/conf/ppo/task/g1_climb_tracking/base.yaml deleted file mode 100644 index c2206f39d..000000000 --- a/conf/ppo/task/g1_climb_tracking/base.yaml +++ /dev/null @@ -1,80 +0,0 @@ -# @package _global_ -# Shared cross-backend config for g1_climb_tracking (mujoco/motrix). See AGENTS.md (Sim2Sim). - -training: - play_steps: 1000 -algo: - num_envs: 1024 - max_iterations: 20000 - save_interval: 500 - empirical_normalization: true - obs_groups: - actor: - - actor - critic: - - critic - algorithm: - entropy_coef: 0.005 - desired_kl: 0.01 -env: - sampling_mode: adaptive - truncate_on_clip_end: false - sim_dt: 0.005 - control_config: - action_scale: - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - anchor_pos_z_threshold: 0.3 - ee_body_pos_z_threshold: 0.3 - terminate_on_undesired_contacts: true - noise_config: - level: 0.0 -reward: - scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 2.0 - motion_body_ori: 1.5 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_ee_body_pos_z: 2.0 - motion_joint_pos: 0.5 - motion_joint_vel: 0.25 - action_rate_l2: -0.005 - joint_limit: -10.0 - undesired_contacts: -0.1 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 diff --git a/conf/ppo/task/g1_climb_tracking/motrix.yaml b/conf/ppo/task/g1_climb_tracking/motrix.yaml index 9077f325d..63a07e012 100644 --- a/conf/ppo/task/g1_climb_tracking/motrix.yaml +++ b/conf/ppo/task/g1_climb_tracking/motrix.yaml @@ -1,8 +1,84 @@ # @package _global_ -defaults: - - /task/g1_climb_tracking/base - - _self_ - training: + play_steps: 1000 task_name: G1ClimbTracking sim_backend: motrix +algo: + num_envs: 1024 + max_iterations: 20000 + save_interval: 500 + empirical_normalization: true + obs_groups: + actor: + - actor + critic: + - critic + algorithm: + entropy_coef: 0.005 + desired_kl: 0.01 +env: + sampling_mode: adaptive + truncate_on_clip_end: false + sim_dt: 0.005 + control_config: + action_scale: + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + anchor_pos_z_threshold: 0.3 + ee_body_pos_z_threshold: 0.3 + terminate_on_undesired_contacts: true + noise_config: + level: 0.0 +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 2.0 + motion_body_ori: 1.5 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_ee_body_pos_z: 2.0 + motion_joint_pos: 0.5 + motion_joint_vel: 0.25 + action_rate_l2: -0.005 + joint_limit: -10.0 + undesired_contacts: -0.1 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_climb_tracking/motrix_play.yaml b/conf/ppo/task/g1_climb_tracking/motrix_play.yaml deleted file mode 100644 index e73384e34..000000000 --- a/conf/ppo/task/g1_climb_tracking/motrix_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/g1_climb_tracking/motrix - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/g1_climb_tracking/mujoco.yaml b/conf/ppo/task/g1_climb_tracking/mujoco.yaml index 1914cb9d2..bddd42ef0 100644 --- a/conf/ppo/task/g1_climb_tracking/mujoco.yaml +++ b/conf/ppo/task/g1_climb_tracking/mujoco.yaml @@ -1,7 +1,84 @@ # @package _global_ -defaults: - - /task/g1_climb_tracking/base - - _self_ training: + play_steps: 1000 task_name: G1ClimbTracking sim_backend: mujoco +algo: + num_envs: 1024 + max_iterations: 20000 + save_interval: 500 + empirical_normalization: true + obs_groups: + actor: + - actor + critic: + - critic + algorithm: + entropy_coef: 0.005 + desired_kl: 0.01 +env: + sampling_mode: adaptive + truncate_on_clip_end: false + sim_dt: 0.005 + control_config: + action_scale: + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + anchor_pos_z_threshold: 0.3 + ee_body_pos_z_threshold: 0.3 + terminate_on_undesired_contacts: true + noise_config: + level: 0.0 +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 2.0 + motion_body_ori: 1.5 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_ee_body_pos_z: 2.0 + motion_joint_pos: 0.5 + motion_joint_vel: 0.25 + action_rate_l2: -0.005 + joint_limit: -10.0 + undesired_contacts: -0.1 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_climb_tracking/mujoco_play.yaml b/conf/ppo/task/g1_climb_tracking/mujoco_play.yaml deleted file mode 100644 index 0b4c96872..000000000 --- a/conf/ppo/task/g1_climb_tracking/mujoco_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/g1_climb_tracking/mujoco - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/g1_flip_tracking/base.yaml b/conf/ppo/task/g1_flip_tracking/base.yaml deleted file mode 100644 index a6b3841f1..000000000 --- a/conf/ppo/task/g1_flip_tracking/base.yaml +++ /dev/null @@ -1,34 +0,0 @@ -# @package _global_ -# Shared cross-backend config for g1_flip_tracking (mujoco/motrix). See AGENTS.md (Sim2Sim). - -training: - play_steps: 1000 -algo: - num_envs: 1024 - max_iterations: 20000 - save_interval: 500 - obs_groups: - actor: - - actor - algorithm: - entropy_coef: 0.005 -reward: - scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 2.0 - motion_body_ori: 1.5 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_joint_pos: 0.0 - motion_joint_vel: 0.0 - action_rate_l2: -0.005 - joint_limit: -10.0 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 diff --git a/conf/ppo/task/g1_flip_tracking/motrix.yaml b/conf/ppo/task/g1_flip_tracking/motrix.yaml index 0158b47c3..1978da4ef 100644 --- a/conf/ppo/task/g1_flip_tracking/motrix.yaml +++ b/conf/ppo/task/g1_flip_tracking/motrix.yaml @@ -1,16 +1,38 @@ # @package _global_ -defaults: - - /task/g1_flip_tracking/base - - _self_ - training: + play_steps: 1000 task_name: G1FlipTracking sim_backend: motrix algo: + num_envs: 1024 max_iterations: 30000 + save_interval: 500 + obs_groups: + actor: + - actor + algorithm: + entropy_coef: 0.005 reward: scales: motion_global_root_pos: 1.0 + motion_global_root_ori: 0.5 motion_body_pos: 1.0 motion_body_ori: 1.0 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_joint_pos: 0.0 + motion_joint_vel: 0.0 action_rate_l2: -0.05 + joint_limit: -10.0 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_flip_tracking/motrix_play.yaml b/conf/ppo/task/g1_flip_tracking/motrix_play.yaml deleted file mode 100644 index e2d1cc1c7..000000000 --- a/conf/ppo/task/g1_flip_tracking/motrix_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/g1_flip_tracking/motrix - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/g1_flip_tracking/mujoco.yaml b/conf/ppo/task/g1_flip_tracking/mujoco.yaml index 9998a7a61..bb4d5188f 100644 --- a/conf/ppo/task/g1_flip_tracking/mujoco.yaml +++ b/conf/ppo/task/g1_flip_tracking/mujoco.yaml @@ -1,15 +1,43 @@ # @package _global_ -defaults: - - /task/g1_flip_tracking/base - - _self_ - +training: + play_steps: 1000 + task_name: G1FlipTracking + sim_backend: mujoco algo: - empirical_normalization: true + num_envs: 1024 + max_iterations: 20000 + save_interval: 500 obs_groups: + actor: + - actor critic: - critic algorithm: + entropy_coef: 0.005 desired_kl: 0.01 + empirical_normalization: true +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 2.0 + motion_body_ori: 1.5 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_joint_pos: 0.0 + motion_joint_vel: 0.0 + action_rate_l2: -0.005 + joint_limit: -10.0 + motion_ee_body_pos_z: 2.0 + undesired_contacts: -0.1 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 env: sampling_mode: start truncate_on_clip_end: false @@ -50,10 +78,7 @@ env: terminate_on_undesired_contacts: true noise_config: level: 0.0 -reward: - scales: - motion_ee_body_pos_z: 2.0 - undesired_contacts: -0.1 -training: - task_name: G1FlipTracking - sim_backend: mujoco +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_flip_tracking/mujoco_play.yaml b/conf/ppo/task/g1_flip_tracking/mujoco_play.yaml deleted file mode 100644 index 76f600908..000000000 --- a/conf/ppo/task/g1_flip_tracking/mujoco_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/g1_flip_tracking/mujoco - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/g1_motion_tracking/base.yaml b/conf/ppo/task/g1_motion_tracking/base.yaml deleted file mode 100644 index 46e54d369..000000000 --- a/conf/ppo/task/g1_motion_tracking/base.yaml +++ /dev/null @@ -1,34 +0,0 @@ -# @package _global_ -# Shared cross-backend config for g1_motion_tracking (mujoco/motrix). See AGENTS.md (Sim2Sim). - -training: - play_steps: 1000 -algo: - num_envs: 1024 - max_iterations: 15000 - save_interval: 500 - obs_groups: - actor: - - actor - algorithm: - entropy_coef: 0.005 -reward: - scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 1.0 - motion_body_ori: 1.0 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_joint_pos: 0.0 - motion_joint_vel: 0.0 - action_rate_l2: -0.1 - joint_limit: -10.0 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 diff --git a/conf/ppo/task/g1_motion_tracking/motrix.yaml b/conf/ppo/task/g1_motion_tracking/motrix.yaml index dc0135dc8..88deb4eef 100644 --- a/conf/ppo/task/g1_motion_tracking/motrix.yaml +++ b/conf/ppo/task/g1_motion_tracking/motrix.yaml @@ -1,18 +1,44 @@ # @package _global_ -defaults: - - /task/g1_motion_tracking/base - - _self_ - training: + play_steps: 1000 task_name: G1MotionTracking sim_backend: motrix play_env_num: 16 algo: + num_envs: 1024 + max_iterations: 15000 + save_interval: 500 + obs_groups: + actor: + - actor + algorithm: + entropy_coef: 0.005 noise_config: level: 1.0 scale_joint_angle: 0.01 scale_joint_vel: 1.5 scale_gyro: 0.2 +reward: + scales: + motion_global_root_pos: 1.0 + motion_global_root_ori: 0.5 + motion_body_pos: 1.0 + motion_body_ori: 1.0 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_joint_pos: 0.0 + motion_joint_vel: 0.0 + action_rate_l2: -0.05 + joint_limit: -10.0 + undesired_contacts: -0.1 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 play_profile: enabled: true env: @@ -32,8 +58,3 @@ play_profile: ground_texrepeat: - 0.25 - 0.25 -reward: - scales: - motion_global_root_pos: 1.0 - action_rate_l2: -0.05 - undesired_contacts: -0.1 diff --git a/conf/ppo/task/g1_motion_tracking/motrix_play.yaml b/conf/ppo/task/g1_motion_tracking/motrix_play.yaml deleted file mode 100644 index 9cebd0be8..000000000 --- a/conf/ppo/task/g1_motion_tracking/motrix_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/g1_motion_tracking/motrix - - _self_ - -training: - play_only: true diff --git a/conf/ppo/task/g1_motion_tracking/mujoco.yaml b/conf/ppo/task/g1_motion_tracking/mujoco.yaml index 52b079dce..b58bc3780 100644 --- a/conf/ppo/task/g1_motion_tracking/mujoco.yaml +++ b/conf/ppo/task/g1_motion_tracking/mujoco.yaml @@ -1,7 +1,38 @@ # @package _global_ -defaults: - - /task/g1_motion_tracking/base - - _self_ training: + play_steps: 1000 task_name: G1MotionTracking sim_backend: mujoco +algo: + num_envs: 1024 + max_iterations: 15000 + save_interval: 500 + obs_groups: + actor: + - actor + algorithm: + entropy_coef: 0.005 +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 1.0 + motion_body_ori: 1.0 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_joint_pos: 0.0 + motion_joint_vel: 0.0 + action_rate_l2: -0.1 + joint_limit: -10.0 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_motion_tracking/mujoco_play.yaml b/conf/ppo/task/g1_motion_tracking/mujoco_play.yaml deleted file mode 100644 index 64a0b97ff..000000000 --- a/conf/ppo/task/g1_motion_tracking/mujoco_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/g1_motion_tracking/mujoco - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/g1_walk_flat/base.yaml b/conf/ppo/task/g1_walk_flat/base.yaml deleted file mode 100644 index 962e1ecf5..000000000 --- a/conf/ppo/task/g1_walk_flat/base.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ -# Shared cross-backend contract for g1_walk_flat. See AGENTS.md (Sim2Sim). -algo: - empirical_normalization: false - obs_groups: - actor: - - actor - policy: - actor_hidden_dims: [512, 256, 128] - critic_hidden_dims: [512, 256, 128] -env: - control_config: - action_scale: 0.25 diff --git a/conf/ppo/task/g1_walk_flat/motrix.yaml b/conf/ppo/task/g1_walk_flat/motrix.yaml index fddd5f51b..fca65c0fe 100644 --- a/conf/ppo/task/g1_walk_flat/motrix.yaml +++ b/conf/ppo/task/g1_walk_flat/motrix.yaml @@ -1,37 +1,40 @@ # @package _global_ -# Inherits base.yaml, then overrides contract fields for Motrix-specific tuning -# (intentionally non-transferable from MuJoCo; drop overrides to restore parity). -defaults: - - /task/g1_walk_flat/base - - _self_ - -training: - task_name: G1WalkFlat - sim_backend: motrix algo: - num_envs: 2048 - max_iterations: 2200 empirical_normalization: true obs_groups: actor: - - policy + - policy critic: - - critic + - critic policy: + actor_hidden_dims: + - 512 + - 256 + - 128 + critic_hidden_dims: + - 512 + - 256 + - 128 init_noise_std: 0.5 + num_envs: 2048 + max_iterations: 2200 algorithm: - learning_rate: 3.0e-4 - entropy_coef: 5.0e-3 + learning_rate: 0.0003 + entropy_coef: 0.005 env: + control_config: + action_scale: 0.5 domain_rand: randomize_kp: false randomize_kd: false - control_config: - action_scale: 0.5 commands: vel_limit: - - [0.4, 0.0, 0.0] - - [0.7, 0.0, 0.0] + - - 0.4 + - 0.0 + - 0.0 + - - 0.7 + - 0.0 + - 0.0 gait_phase_init_mode: offset_phase reset_base_qvel_limit: 0.05 curriculum: @@ -41,6 +44,9 @@ env: scale_joint_angle: 0.01 scale_joint_vel: 1.5 scale_gyro: 0.2 +training: + task_name: G1WalkFlat + sim_backend: motrix reward: scales: tracking_lin_vel: 2.0 @@ -67,3 +73,7 @@ reward: min_forward_speed_for_gait_reward: 0.05 min_base_height: 0.5 max_tilt_deg: 35.0 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_walk_flat/motrix_play.yaml b/conf/ppo/task/g1_walk_flat/motrix_play.yaml deleted file mode 100644 index 2ce9d3ac4..000000000 --- a/conf/ppo/task/g1_walk_flat/motrix_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/g1_walk_flat/motrix - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/g1_walk_flat/mujoco.yaml b/conf/ppo/task/g1_walk_flat/mujoco.yaml index 54d670f5d..7d3a3f1c7 100644 --- a/conf/ppo/task/g1_walk_flat/mujoco.yaml +++ b/conf/ppo/task/g1_walk_flat/mujoco.yaml @@ -1,16 +1,23 @@ # @package _global_ -# Inherits the shared contract from base.yaml; backend-specific tuning only. -defaults: - - /task/g1_walk_flat/base - - _self_ - -training: - task_name: G1WalkFlat - sim_backend: mujoco algo: + empirical_normalization: false + obs_groups: + actor: + - actor + policy: + actor_hidden_dims: + - 512 + - 256 + - 128 + critic_hidden_dims: + - 512 + - 256 + - 128 num_envs: 2048 max_iterations: 2200 env: + control_config: + action_scale: 0.25 curriculum: enabled: false noise_config: @@ -18,6 +25,9 @@ env: scale_joint_angle: 0.01 scale_joint_vel: 1.5 scale_gyro: 0.2 +training: + task_name: G1WalkFlat + sim_backend: mujoco reward: scales: tracking_lin_vel: 2.0 @@ -36,4 +46,37 @@ reward: base_height_target: 0.754 min_base_height: 0.55 max_tilt_deg: 25.0 - pose_weights: [0.01, 1.0, 5.0, 0.01, 5.0, 5.0, 0.01, 1.0, 5.0, 0.01, 5.0, 5.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0] + pose_weights: + - 0.01 + - 1.0 + - 5.0 + - 0.01 + - 5.0 + - 5.0 + - 0.01 + - 1.0 + - 5.0 + - 0.01 + - 5.0 + - 5.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 + - 50.0 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_walk_flat/mujoco_play.yaml b/conf/ppo/task/g1_walk_flat/mujoco_play.yaml deleted file mode 100644 index 37ae39c6a..000000000 --- a/conf/ppo/task/g1_walk_flat/mujoco_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/g1_walk_flat/mujoco - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/g1_wall_flip_tracking/base.yaml b/conf/ppo/task/g1_wall_flip_tracking/base.yaml deleted file mode 100644 index a778abc01..000000000 --- a/conf/ppo/task/g1_wall_flip_tracking/base.yaml +++ /dev/null @@ -1,80 +0,0 @@ -# @package _global_ -# Shared cross-backend config for g1_wall_flip_tracking (mujoco/motrix). See AGENTS.md (Sim2Sim). - -training: - play_steps: 1000 -algo: - num_envs: 1024 - max_iterations: 20000 - save_interval: 500 - empirical_normalization: true - obs_groups: - actor: - - actor - critic: - - critic - algorithm: - entropy_coef: 0.005 - desired_kl: 0.01 -env: - sampling_mode: start - truncate_on_clip_end: false - sim_dt: 0.005 - control_config: - action_scale: - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - anchor_pos_z_threshold: 0.5 - ee_body_pos_z_threshold: 0.5 - terminate_on_undesired_contacts: true - noise_config: - level: 0.0 -reward: - scales: - motion_global_root_pos: 0.5 - motion_global_root_ori: 0.5 - motion_body_pos: 2.0 - motion_body_ori: 1.5 - motion_body_lin_vel: 1.0 - motion_body_ang_vel: 1.0 - motion_ee_body_pos_z: 2.0 - motion_joint_pos: 0.5 - motion_joint_vel: 0.25 - action_rate_l2: -0.005 - joint_limit: -10.0 - undesired_contacts: -0.1 - std_root_pos: 0.3 - std_root_ori: 0.4 - std_body_pos: 0.3 - std_body_ori: 0.4 - std_body_lin_vel: 1.0 - std_body_ang_vel: 3.14 - std_joint_pos: 0.2 - std_joint_vel: 1.0 diff --git a/conf/ppo/task/g1_wall_flip_tracking/motrix.yaml b/conf/ppo/task/g1_wall_flip_tracking/motrix.yaml index bc1bc97c2..223196042 100644 --- a/conf/ppo/task/g1_wall_flip_tracking/motrix.yaml +++ b/conf/ppo/task/g1_wall_flip_tracking/motrix.yaml @@ -1,17 +1,86 @@ # @package _global_ -defaults: - - /task/g1_wall_flip_tracking/base - - _self_ - training: + play_steps: 1000 task_name: G1WallFlipTracking sim_backend: motrix play_env_num: 16 render_spacing: 3.0 algo: + num_envs: 1024 max_iterations: 12000 + save_interval: 500 + empirical_normalization: true + obs_groups: + actor: + - actor + critic: + - critic + algorithm: + entropy_coef: 0.005 + desired_kl: 0.01 env: + sampling_mode: start + truncate_on_clip_end: false + sim_dt: 0.005 + control_config: + action_scale: + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + anchor_pos_z_threshold: 0.5 + ee_body_pos_z_threshold: 0.5 + terminate_on_undesired_contacts: true + noise_config: + level: 0.0 motrix_max_iterations: 3 +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 2.0 + motion_body_ori: 1.5 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_ee_body_pos_z: 2.0 + motion_joint_pos: 0.5 + motion_joint_vel: 0.25 + action_rate_l2: -0.005 + joint_limit: -10.0 + undesired_contacts: -0.1 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 play_profile: enabled: true env: diff --git a/conf/ppo/task/g1_wall_flip_tracking/motrix_play.yaml b/conf/ppo/task/g1_wall_flip_tracking/motrix_play.yaml deleted file mode 100644 index 78ea34f66..000000000 --- a/conf/ppo/task/g1_wall_flip_tracking/motrix_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/g1_wall_flip_tracking/motrix - - _self_ - -training: - play_only: true diff --git a/conf/ppo/task/g1_wall_flip_tracking/mujoco.yaml b/conf/ppo/task/g1_wall_flip_tracking/mujoco.yaml index b843297b7..6753ce2d7 100644 --- a/conf/ppo/task/g1_wall_flip_tracking/mujoco.yaml +++ b/conf/ppo/task/g1_wall_flip_tracking/mujoco.yaml @@ -1,7 +1,84 @@ # @package _global_ -defaults: - - /task/g1_wall_flip_tracking/base - - _self_ training: + play_steps: 1000 task_name: G1WallFlipTracking sim_backend: mujoco +algo: + num_envs: 1024 + max_iterations: 20000 + save_interval: 500 + empirical_normalization: true + obs_groups: + actor: + - actor + critic: + - critic + algorithm: + entropy_coef: 0.005 + desired_kl: 0.01 +env: + sampling_mode: start + truncate_on_clip_end: false + sim_dt: 0.005 + control_config: + action_scale: + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + anchor_pos_z_threshold: 0.5 + ee_body_pos_z_threshold: 0.5 + terminate_on_undesired_contacts: true + noise_config: + level: 0.0 +reward: + scales: + motion_global_root_pos: 0.5 + motion_global_root_ori: 0.5 + motion_body_pos: 2.0 + motion_body_ori: 1.5 + motion_body_lin_vel: 1.0 + motion_body_ang_vel: 1.0 + motion_ee_body_pos_z: 2.0 + motion_joint_pos: 0.5 + motion_joint_vel: 0.25 + action_rate_l2: -0.005 + joint_limit: -10.0 + undesired_contacts: -0.1 + std_root_pos: 0.3 + std_root_ori: 0.4 + std_body_pos: 0.3 + std_body_ori: 0.4 + std_body_lin_vel: 1.0 + std_body_ang_vel: 3.14 + std_joint_pos: 0.2 + std_joint_vel: 1.0 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/g1_wall_flip_tracking/mujoco_play.yaml b/conf/ppo/task/g1_wall_flip_tracking/mujoco_play.yaml deleted file mode 100644 index 55856ab01..000000000 --- a/conf/ppo/task/g1_wall_flip_tracking/mujoco_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/g1_wall_flip_tracking/mujoco - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/go1_joystick_flat/base.yaml b/conf/ppo/task/go1_joystick_flat/base.yaml deleted file mode 100644 index ffef37a33..000000000 --- a/conf/ppo/task/go1_joystick_flat/base.yaml +++ /dev/null @@ -1,32 +0,0 @@ -# @package _global_ -# Shared cross-backend config for go1_joystick_flat (mujoco/motrix). See AGENTS.md (Sim2Sim). - -training: - play_steps: 500 - play_env_num: 16 - cam_tracking: true - cam_tracking_env_idx: 0 - cam_tracking_extra_envs: 9 -interactive: - action_mode: policy - policy_obs_mode: auto - camera_follow_body: true - use_env_visual_model: false -algo: - num_envs: 1024 - max_iterations: 151 - obs_groups: - actor: - - actor -reward: - scales: - tracking_lin_vel: 1.0 - tracking_ang_vel: 0.2 - lin_vel_z: -5.0 - ang_vel_xy: -0.1 - base_height: -100.0 - action_rate: -0.005 - similar_to_default: -0.1 - swing_feet_z: 4.0 - tracking_sigma: 0.25 - base_height_target: 0.3 diff --git a/conf/ppo/task/go1_joystick_flat/motrix.yaml b/conf/ppo/task/go1_joystick_flat/motrix.yaml index 07938420b..fc2aa162a 100644 --- a/conf/ppo/task/go1_joystick_flat/motrix.yaml +++ b/conf/ppo/task/go1_joystick_flat/motrix.yaml @@ -1,18 +1,41 @@ # @package _global_ -defaults: - - /task/go1_joystick_flat/base - - _self_ - training: + play_steps: 500 + play_env_num: 16 + cam_tracking: true + cam_tracking_env_idx: 0 + cam_tracking_extra_envs: 9 task_name: Go1JoystickFlat sim_backend: motrix +interactive: + action_mode: policy + policy_obs_mode: auto + camera_follow_body: true + use_env_visual_model: false algo: + num_envs: 1024 + max_iterations: 151 + obs_groups: + actor: + - actor empirical_normalization: true policy: init_noise_std: 0.5 algorithm: learning_rate: 0.0003 entropy_coef: 0.001 +reward: + scales: + tracking_lin_vel: 1.0 + tracking_ang_vel: 0.2 + lin_vel_z: -5.0 + ang_vel_xy: -0.1 + base_height: -100.0 + action_rate: -0.005 + similar_to_default: -0.1 + swing_feet_z: 4.0 + tracking_sigma: 0.25 + base_height_target: 0.3 env: commands: vel_limit: @@ -22,3 +45,7 @@ env: - - 0.5 - 0.0 - 0.0 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go1_joystick_flat/motrix_play.yaml b/conf/ppo/task/go1_joystick_flat/motrix_play.yaml deleted file mode 100644 index 949a6c7e1..000000000 --- a/conf/ppo/task/go1_joystick_flat/motrix_play.yaml +++ /dev/null @@ -1,12 +0,0 @@ -# @package _global_ -defaults: - - /task/go1_joystick_flat/motrix - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/go1_joystick_flat/mujoco.yaml b/conf/ppo/task/go1_joystick_flat/mujoco.yaml index afab50df6..eb5799b20 100644 --- a/conf/ppo/task/go1_joystick_flat/mujoco.yaml +++ b/conf/ppo/task/go1_joystick_flat/mujoco.yaml @@ -1,12 +1,38 @@ # @package _global_ -defaults: - - /task/go1_joystick_flat/base - - _self_ - training: + play_steps: 500 + play_env_num: 16 + cam_tracking: true + cam_tracking_env_idx: 0 + cam_tracking_extra_envs: 9 task_name: Go1JoystickFlat sim_backend: mujoco render_spacing: 0.0 +interactive: + action_mode: policy + policy_obs_mode: auto + camera_follow_body: true + use_env_visual_model: false +algo: + num_envs: 1024 + max_iterations: 151 + obs_groups: + actor: + - actor reward: scales: + tracking_lin_vel: 1.0 + tracking_ang_vel: 0.2 + lin_vel_z: -5.0 + ang_vel_xy: -0.1 + base_height: -100.0 + action_rate: -0.005 + similar_to_default: -0.1 + swing_feet_z: 4.0 contact: 0.24 + tracking_sigma: 0.25 + base_height_target: 0.3 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go1_joystick_flat/mujoco_play.yaml b/conf/ppo/task/go1_joystick_flat/mujoco_play.yaml deleted file mode 100644 index 6f5fab2c4..000000000 --- a/conf/ppo/task/go1_joystick_flat/mujoco_play.yaml +++ /dev/null @@ -1,12 +0,0 @@ -# @package _global_ -defaults: - - /task/go1_joystick_flat/mujoco - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/go1_joystick_rough/base.yaml b/conf/ppo/task/go1_joystick_rough/base.yaml deleted file mode 100644 index 779562693..000000000 --- a/conf/ppo/task/go1_joystick_rough/base.yaml +++ /dev/null @@ -1,131 +0,0 @@ -# @package _global_ -# Shared cross-backend config for go1_joystick_rough (mujoco/motrix). See AGENTS.md (Sim2Sim). - -training: - play_steps: 500 - play_env_num: 16 - cam_tracking: true - cam_tracking_env_idx: 0 - cam_tracking_extra_envs: 9 -interactive: - action_mode: policy - policy_obs_mode: auto - camera_follow_body: true - use_env_visual_model: false -algo: - num_envs: 2048 - num_steps_per_env: 24 - max_iterations: 1000 - empirical_normalization: false - obs_groups: - actor: - - actor - critic: - - critic - policy: - init_noise_std: 1.0 - algorithm: - learning_rate: 0.001 - entropy_coef: 0.01 -env: - control_config: - action_scale: 0.25 - hip_action_scale: 0.125 - non_hip_action_scale: 0.25 - clip_actions: 100.0 - commands: - vel_limit: - - - -1.0 - - -1.0 - - -1.0 - - - 1.0 - - 1.0 - - 1.0 - resampling_time: 10.0 - heading_command: true - heading_range: - - -3.141592653589793 - - 3.141592653589793 - rel_standing_envs: 0.1 - terrain_curriculum: - enabled: false - scene: - model_file: src/unilab/assets/robots/go1/go1_mujoco.xml - fragment_files: - - src/unilab/assets/robots/go1/locomotion_task.xml - terrain: - hfield_name: terrain_hfield - geom_name: floor - generator: - seed: 42 - curriculum: false - size: - - 8.0 - - 8.0 - num_rows: 6 - num_cols: 6 - border_width: 20.0 - terrain_scan: - enabled: true - geom_name: floor - termination_config: - terrain_out_of_bounds: true - terrain_distance_buffer: 3.0 - domain_rand: - randomize_base_mass: true - added_mass_range: - - -1.0 - - 3.0 - random_com: true - randomize_kp: true - kp_multiplier_range: - - 0.5 - - 2.0 - randomize_kd: true - kd_multiplier_range: - - 0.5 - - 2.0 - push_robots: true - push_interval: 625 - max_force: - - 1.0 - - 1.0 - - 0.5 -reward: - scales: - lin_vel_z: -2.0 - ang_vel_xy: -0.05 - joint_torques_l2: -2.5e-05 - joint_acc_l2: -2.5e-07 - joint_power: -2.0e-05 - stand_still: -2.0 - hip_pos: -0.5 - joint_pos_penalty: -1.0 - joint_mirror: -0.05 - action_rate: -0.01 - undesired_contacts: -1.0 - contact_forces: -0.00015 - tracking_lin_vel: 3.0 - tracking_ang_vel: 1.5 - feet_air_time: 0.5 - feet_air_time_variance: -1.0 - feet_contact_without_cmd: 0.1 - feet_slide: -0.1 - feet_height_body: -5.0 - feet_gait: 0.5 - upward: 1.0 - tracking_sigma: 0.25 - base_height_target: 0.33 - stand_still_command_threshold: 0.1 - joint_pos_penalty_stand_still_scale: 5.0 - joint_pos_penalty_velocity_threshold: 0.5 - joint_pos_penalty_command_threshold: 0.1 - contact_threshold: 1.0 - contact_forces_threshold: 100.0 - feet_air_time_threshold: 0.5 - feet_height_body_target: -0.2 - feet_height_body_tanh_mult: 2.0 - feet_gait_std: 0.7071067811865476 - feet_gait_max_err: 0.2 - feet_gait_velocity_threshold: 0.5 - feet_gait_command_threshold: 0.1 diff --git a/conf/ppo/task/go1_joystick_rough/motrix.yaml b/conf/ppo/task/go1_joystick_rough/motrix.yaml index c382e2f5a..4d9cfdfe2 100644 --- a/conf/ppo/task/go1_joystick_rough/motrix.yaml +++ b/conf/ppo/task/go1_joystick_rough/motrix.yaml @@ -1,12 +1,136 @@ # @package _global_ -defaults: - - /task/go1_joystick_rough/base - - _self_ - training: + play_steps: 500 + play_env_num: 16 + cam_tracking: true + cam_tracking_env_idx: 0 + cam_tracking_extra_envs: 9 task_name: Go1JoystickRough sim_backend: motrix +interactive: + action_mode: policy + policy_obs_mode: auto + camera_follow_body: true + use_env_visual_model: false +algo: + num_envs: 2048 + num_steps_per_env: 24 + max_iterations: 1000 + empirical_normalization: false + obs_groups: + actor: + - actor + critic: + - critic + policy: + init_noise_std: 1.0 + algorithm: + learning_rate: 0.001 + entropy_coef: 0.01 env: - render_offset_mode: zero + control_config: + action_scale: 0.25 + hip_action_scale: 0.125 + non_hip_action_scale: 0.25 + clip_actions: 100.0 + commands: + vel_limit: + - - -1.0 + - -1.0 + - -1.0 + - - 1.0 + - 1.0 + - 1.0 + resampling_time: 10.0 + heading_command: true + heading_range: + - -3.141592653589793 + - 3.141592653589793 + rel_standing_envs: 0.1 + terrain_curriculum: + enabled: false scene: model_file: src/unilab/assets/robots/go1/go1.xml + fragment_files: + - src/unilab/assets/robots/go1/locomotion_task.xml + terrain: + hfield_name: terrain_hfield + geom_name: floor + generator: + seed: 42 + curriculum: false + size: + - 8.0 + - 8.0 + num_rows: 6 + num_cols: 6 + border_width: 20.0 + terrain_scan: + enabled: true + geom_name: floor + termination_config: + terrain_out_of_bounds: true + terrain_distance_buffer: 3.0 + domain_rand: + randomize_base_mass: true + added_mass_range: + - -1.0 + - 3.0 + random_com: true + randomize_kp: true + kp_multiplier_range: + - 0.5 + - 2.0 + randomize_kd: true + kd_multiplier_range: + - 0.5 + - 2.0 + push_robots: true + push_interval: 625 + max_force: + - 1.0 + - 1.0 + - 0.5 + render_offset_mode: zero +reward: + scales: + lin_vel_z: -2.0 + ang_vel_xy: -0.05 + joint_torques_l2: -2.5e-05 + joint_acc_l2: -2.5e-07 + joint_power: -2.0e-05 + stand_still: -2.0 + hip_pos: -0.5 + joint_pos_penalty: -1.0 + joint_mirror: -0.05 + action_rate: -0.01 + undesired_contacts: -1.0 + contact_forces: -0.00015 + tracking_lin_vel: 3.0 + tracking_ang_vel: 1.5 + feet_air_time: 0.5 + feet_air_time_variance: -1.0 + feet_contact_without_cmd: 0.1 + feet_slide: -0.1 + feet_height_body: -5.0 + feet_gait: 0.5 + upward: 1.0 + tracking_sigma: 0.25 + base_height_target: 0.33 + stand_still_command_threshold: 0.1 + joint_pos_penalty_stand_still_scale: 5.0 + joint_pos_penalty_velocity_threshold: 0.5 + joint_pos_penalty_command_threshold: 0.1 + contact_threshold: 1.0 + contact_forces_threshold: 100.0 + feet_air_time_threshold: 0.5 + feet_height_body_target: -0.2 + feet_height_body_tanh_mult: 2.0 + feet_gait_std: 0.7071067811865476 + feet_gait_max_err: 0.2 + feet_gait_velocity_threshold: 0.5 + feet_gait_command_threshold: 0.1 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go1_joystick_rough/motrix_play.yaml b/conf/ppo/task/go1_joystick_rough/motrix_play.yaml deleted file mode 100644 index 68b32511c..000000000 --- a/conf/ppo/task/go1_joystick_rough/motrix_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/go1_joystick_rough/motrix - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/go1_joystick_rough/mujoco.yaml b/conf/ppo/task/go1_joystick_rough/mujoco.yaml index d45546650..43b84ad57 100644 --- a/conf/ppo/task/go1_joystick_rough/mujoco.yaml +++ b/conf/ppo/task/go1_joystick_rough/mujoco.yaml @@ -1,11 +1,137 @@ # @package _global_ -defaults: - - /task/go1_joystick_rough/base - - _self_ - training: + play_steps: 500 + play_env_num: 16 + cam_tracking: true + cam_tracking_env_idx: 0 + cam_tracking_extra_envs: 9 task_name: Go1JoystickRough sim_backend: mujoco render_spacing: 0.0 +interactive: + action_mode: policy + policy_obs_mode: auto + camera_follow_body: true + use_env_visual_model: false +algo: + num_envs: 2048 + num_steps_per_env: 24 + max_iterations: 1000 + empirical_normalization: false + obs_groups: + actor: + - actor + critic: + - critic + policy: + init_noise_std: 1.0 + algorithm: + learning_rate: 0.001 + entropy_coef: 0.01 env: + control_config: + action_scale: 0.25 + hip_action_scale: 0.125 + non_hip_action_scale: 0.25 + clip_actions: 100.0 + commands: + vel_limit: + - - -1.0 + - -1.0 + - -1.0 + - - 1.0 + - 1.0 + - 1.0 + resampling_time: 10.0 + heading_command: true + heading_range: + - -3.141592653589793 + - 3.141592653589793 + rel_standing_envs: 0.1 + terrain_curriculum: + enabled: false + scene: + model_file: src/unilab/assets/robots/go1/go1_mujoco.xml + fragment_files: + - src/unilab/assets/robots/go1/locomotion_task.xml + terrain: + hfield_name: terrain_hfield + geom_name: floor + generator: + seed: 42 + curriculum: false + size: + - 8.0 + - 8.0 + num_rows: 6 + num_cols: 6 + border_width: 20.0 + terrain_scan: + enabled: true + geom_name: floor + termination_config: + terrain_out_of_bounds: true + terrain_distance_buffer: 3.0 + domain_rand: + randomize_base_mass: true + added_mass_range: + - -1.0 + - 3.0 + random_com: true + randomize_kp: true + kp_multiplier_range: + - 0.5 + - 2.0 + randomize_kd: true + kd_multiplier_range: + - 0.5 + - 2.0 + push_robots: true + push_interval: 625 + max_force: + - 1.0 + - 1.0 + - 0.5 sim_dt: 0.005 +reward: + scales: + lin_vel_z: -2.0 + ang_vel_xy: -0.05 + joint_torques_l2: -2.5e-05 + joint_acc_l2: -2.5e-07 + joint_power: -2.0e-05 + stand_still: -2.0 + hip_pos: -0.5 + joint_pos_penalty: -1.0 + joint_mirror: -0.05 + action_rate: -0.01 + undesired_contacts: -1.0 + contact_forces: -0.00015 + tracking_lin_vel: 3.0 + tracking_ang_vel: 1.5 + feet_air_time: 0.5 + feet_air_time_variance: -1.0 + feet_contact_without_cmd: 0.1 + feet_slide: -0.1 + feet_height_body: -5.0 + feet_gait: 0.5 + upward: 1.0 + tracking_sigma: 0.25 + base_height_target: 0.33 + stand_still_command_threshold: 0.1 + joint_pos_penalty_stand_still_scale: 5.0 + joint_pos_penalty_velocity_threshold: 0.5 + joint_pos_penalty_command_threshold: 0.1 + contact_threshold: 1.0 + contact_forces_threshold: 100.0 + feet_air_time_threshold: 0.5 + feet_height_body_target: -0.2 + feet_height_body_tanh_mult: 2.0 + feet_gait_std: 0.7071067811865476 + feet_gait_max_err: 0.2 + feet_gait_velocity_threshold: 0.5 + feet_gait_command_threshold: 0.1 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go1_joystick_rough/mujoco_play.yaml b/conf/ppo/task/go1_joystick_rough/mujoco_play.yaml deleted file mode 100644 index d180360fb..000000000 --- a/conf/ppo/task/go1_joystick_rough/mujoco_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/go1_joystick_rough/mujoco - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/go2_arm_manip_loco/base.yaml b/conf/ppo/task/go2_arm_manip_loco/base.yaml deleted file mode 100644 index e0e8465ec..000000000 --- a/conf/ppo/task/go2_arm_manip_loco/base.yaml +++ /dev/null @@ -1,149 +0,0 @@ -# @package _global_ -# Shared cross-backend config for go2_arm_manip_loco (mujoco/motrix). See AGENTS.md (Sim2Sim). - -algo: - num_envs: 4096 - max_iterations: 151 - empirical_normalization: true - obs_groups: - actor: - - actor - policy: - init_noise_std: 0.5 - algorithm: - learning_rate: 0.0003 - entropy_coef: 0.001 - num_mini_batches: 4 -reward: - scales: - tracking_lin_vel: 2.0 - tracking_ang_vel: 0.5 - lin_vel_z: -5.0 - ang_vel_xy: -0.1 - roll: -5.0 - base_height: -100 - similar_to_default: -0.0 - leg_pose: -0.1 - dof_pos_limits: 0.0 - action_rate: -0.005 - torques: 0.0 - energy: 0.0 - dof_vel: 0.0 - dof_acc: 0.0 - stand_still: -0.5 - contact: 0.24 - swing_feet_z: 4.0 - foot_drag: -0.1 - object_distance: 2.0 - object_distance_l2: -0.5 - arm_collision: -1.0 - tracking_sigma: 0.25 - base_height_target: 0.3 - object_sigma: 0.1 -env: - goal_ee: - sphere_l_range: - - 0.3 - - 0.6 - sphere_phi_range: - - -1.2566 - - 1.0472 - sphere_theta_range: - - -2.3562 - - 2.3562 - traj_time_range: - - 1.0 - - 3.0 - hold_time_range: - - 0.5 - - 2.0 - collision_upper_limits: - - 0.3 - - 0.15 - - -0.115 - collision_lower_limits: - - -0.2 - - -0.15 - - -0.515 - underground_limit: -0.57 - num_collision_check_samples: 10 - num_resample_attempts: 10 - default_orn_roll: 0.0 - arm_induced_pitch: 0.78 - delta_orn_r: - - -0.5 - - 0.5 - delta_orn_p: - - -0.5 - - 0.5 - delta_orn_y: - - -0.5 - - 0.5 - init_ee_cart: - - 0.25 - - 0.0 - - 0.25 - control_config: - arm_action_scale: 0.0 - ik: - damping: 0.05 - gain: 1.0 - dq_clip: 0.2 - use_orientation: true - orientation_mode: target - commands: - vel_limit: - - - -0.6 - - -0.4 - - -0.8 - - - 1.0 - - 0.4 - - 0.8 - zero_command_prob: 0.15 - resample_time_s: 4.0 - curriculum: - enable: false - domain_rand: - randomize_base_mass: false - added_mass_range: - - -1.0 - - 1.0 - randomize_body_mass: true - body_mass_multiplier_range: - - 0.9 - - 1.1 - random_com: true - com_offset_x: - - -0.03 - - 0.03 - randomize_gravity: false - randomize_ground_friction: true - ground_friction_multiplier_range: - - 0.8 - - 1.2 - randomize_dof_armature: true - dof_armature_multiplier_range: - - 0.8 - - 1.2 - push_robots: true - push_interval: 500 - max_force: - - 1.2 - - 1.2 - - 0.6 - push_body_name: base - randomize_kp: true - kp_multiplier_range: - - 0.9 - - 1.1 - randomize_kd: true - kd_multiplier_range: - - 0.9 - - 1.1 - arm_stage: - freeze_arm_joints: false - disable_ee_goal_trajectory: false - fixed_ee_goal_cart: - - 0.15 - - 0.0 - - 0.25 diff --git a/conf/ppo/task/go2_arm_manip_loco/motrix.yaml b/conf/ppo/task/go2_arm_manip_loco/motrix.yaml index a29528a7f..ec3b00ca5 100644 --- a/conf/ppo/task/go2_arm_manip_loco/motrix.yaml +++ b/conf/ppo/task/go2_arm_manip_loco/motrix.yaml @@ -1,13 +1,153 @@ # @package _global_ -defaults: - - /task/go2_arm_manip_loco/base - - _self_ - +algo: + num_envs: 4096 + max_iterations: 3000 + empirical_normalization: true + obs_groups: + actor: + - actor + policy: + init_noise_std: 0.5 + algorithm: + learning_rate: 0.0003 + entropy_coef: 0.001 + num_mini_batches: 4 +reward: + scales: + tracking_lin_vel: 2.0 + tracking_ang_vel: 0.5 + lin_vel_z: -5.0 + ang_vel_xy: -0.1 + roll: -5.0 + base_height: -100 + similar_to_default: -0.0 + leg_pose: -0.1 + dof_pos_limits: 0.0 + action_rate: -0.005 + torques: 0.0 + energy: 0.0 + dof_vel: 0.0 + dof_acc: 0.0 + stand_still: -0.5 + contact: 0.24 + swing_feet_z: 4.0 + foot_drag: -0.1 + object_distance: 2.0 + object_distance_l2: -0.5 + arm_collision: -1.0 + tracking_sigma: 0.25 + base_height_target: 0.3 + object_sigma: 0.1 +env: + goal_ee: + sphere_l_range: + - 0.3 + - 0.6 + sphere_phi_range: + - -1.2566 + - 1.0472 + sphere_theta_range: + - -2.3562 + - 2.3562 + traj_time_range: + - 1.0 + - 3.0 + hold_time_range: + - 0.5 + - 2.0 + collision_upper_limits: + - 0.3 + - 0.15 + - -0.115 + collision_lower_limits: + - -0.2 + - -0.15 + - -0.515 + underground_limit: -0.57 + num_collision_check_samples: 10 + num_resample_attempts: 10 + default_orn_roll: 0.0 + arm_induced_pitch: 0.78 + delta_orn_r: + - -0.5 + - 0.5 + delta_orn_p: + - -0.5 + - 0.5 + delta_orn_y: + - -0.5 + - 0.5 + init_ee_cart: + - 0.25 + - 0.0 + - 0.25 + control_config: + arm_action_scale: 0.0 + ik: + damping: 0.05 + gain: 1.0 + dq_clip: 0.2 + use_orientation: true + orientation_mode: target + commands: + vel_limit: + - - -0.6 + - -0.4 + - -0.8 + - - 1.0 + - 0.4 + - 0.8 + zero_command_prob: 0.15 + resample_time_s: 4.0 + curriculum: + enable: false + domain_rand: + randomize_base_mass: false + added_mass_range: + - -1.0 + - 1.0 + randomize_body_mass: true + body_mass_multiplier_range: + - 0.9 + - 1.1 + random_com: true + com_offset_x: + - -0.03 + - 0.03 + randomize_gravity: false + randomize_ground_friction: true + ground_friction_multiplier_range: + - 0.8 + - 1.2 + randomize_dof_armature: false + dof_armature_multiplier_range: + - 0.8 + - 1.2 + push_robots: true + push_interval: 500 + max_force: + - 1.2 + - 1.2 + - 0.6 + push_body_name: base + randomize_kp: false + kp_multiplier_range: + - 0.9 + - 1.1 + randomize_kd: false + kd_multiplier_range: + - 0.9 + - 1.1 + arm_stage: + freeze_arm_joints: false + disable_ee_goal_trajectory: false + fixed_ee_goal_cart: + - 0.15 + - 0.0 + - 0.25 training: task_name: Go2ArmManipLoco sim_backend: motrix -algo: - max_iterations: 3000 play_profile: enabled: true scene: @@ -25,8 +165,3 @@ play_profile: ground_texrepeat: - 0.25 - 0.25 -env: - domain_rand: - randomize_dof_armature: false - randomize_kp: false - randomize_kd: false diff --git a/conf/ppo/task/go2_arm_manip_loco/motrix_play.yaml b/conf/ppo/task/go2_arm_manip_loco/motrix_play.yaml deleted file mode 100644 index be532cc63..000000000 --- a/conf/ppo/task/go2_arm_manip_loco/motrix_play.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# @package _global_ - -defaults: - - /task/go2_arm_manip_loco/motrix - - _self_ - -training: - play_only: true diff --git a/conf/ppo/task/go2_arm_manip_loco/mujoco.yaml b/conf/ppo/task/go2_arm_manip_loco/mujoco.yaml index 3573222eb..83948f12d 100644 --- a/conf/ppo/task/go2_arm_manip_loco/mujoco.yaml +++ b/conf/ppo/task/go2_arm_manip_loco/mujoco.yaml @@ -1,7 +1,154 @@ # @package _global_ -defaults: - - /task/go2_arm_manip_loco/base - - _self_ +algo: + num_envs: 4096 + max_iterations: 151 + empirical_normalization: true + obs_groups: + actor: + - actor + policy: + init_noise_std: 0.5 + algorithm: + learning_rate: 0.0003 + entropy_coef: 0.001 + num_mini_batches: 4 +reward: + scales: + tracking_lin_vel: 2.0 + tracking_ang_vel: 0.5 + lin_vel_z: -5.0 + ang_vel_xy: -0.1 + roll: -5.0 + base_height: -100 + similar_to_default: -0.0 + leg_pose: -0.1 + dof_pos_limits: 0.0 + action_rate: -0.005 + torques: 0.0 + energy: 0.0 + dof_vel: 0.0 + dof_acc: 0.0 + stand_still: -0.5 + contact: 0.24 + swing_feet_z: 4.0 + foot_drag: -0.1 + object_distance: 2.0 + object_distance_l2: -0.5 + arm_collision: -1.0 + tracking_sigma: 0.25 + base_height_target: 0.3 + object_sigma: 0.1 +env: + goal_ee: + sphere_l_range: + - 0.3 + - 0.6 + sphere_phi_range: + - -1.2566 + - 1.0472 + sphere_theta_range: + - -2.3562 + - 2.3562 + traj_time_range: + - 1.0 + - 3.0 + hold_time_range: + - 0.5 + - 2.0 + collision_upper_limits: + - 0.3 + - 0.15 + - -0.115 + collision_lower_limits: + - -0.2 + - -0.15 + - -0.515 + underground_limit: -0.57 + num_collision_check_samples: 10 + num_resample_attempts: 10 + default_orn_roll: 0.0 + arm_induced_pitch: 0.78 + delta_orn_r: + - -0.5 + - 0.5 + delta_orn_p: + - -0.5 + - 0.5 + delta_orn_y: + - -0.5 + - 0.5 + init_ee_cart: + - 0.25 + - 0.0 + - 0.25 + control_config: + arm_action_scale: 0.0 + ik: + damping: 0.05 + gain: 1.0 + dq_clip: 0.2 + use_orientation: true + orientation_mode: target + commands: + vel_limit: + - - -0.6 + - -0.4 + - -0.8 + - - 1.0 + - 0.4 + - 0.8 + zero_command_prob: 0.15 + resample_time_s: 4.0 + curriculum: + enable: false + domain_rand: + randomize_base_mass: false + added_mass_range: + - -1.0 + - 1.0 + randomize_body_mass: true + body_mass_multiplier_range: + - 0.9 + - 1.1 + random_com: true + com_offset_x: + - -0.03 + - 0.03 + randomize_gravity: false + randomize_ground_friction: true + ground_friction_multiplier_range: + - 0.8 + - 1.2 + randomize_dof_armature: true + dof_armature_multiplier_range: + - 0.8 + - 1.2 + push_robots: true + push_interval: 500 + max_force: + - 1.2 + - 1.2 + - 0.6 + push_body_name: base + randomize_kp: true + kp_multiplier_range: + - 0.9 + - 1.1 + randomize_kd: true + kd_multiplier_range: + - 0.9 + - 1.1 + arm_stage: + freeze_arm_joints: false + disable_ee_goal_trajectory: false + fixed_ee_goal_cart: + - 0.15 + - 0.0 + - 0.25 training: task_name: Go2ArmManipLoco sim_backend: mujoco +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2_arm_manip_loco/mujoco_play.yaml b/conf/ppo/task/go2_arm_manip_loco/mujoco_play.yaml deleted file mode 100644 index 1c33cc956..000000000 --- a/conf/ppo/task/go2_arm_manip_loco/mujoco_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/go2_arm_manip_loco/mujoco - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/go2_joystick_flat/base.yaml b/conf/ppo/task/go2_joystick_flat/base.yaml deleted file mode 100644 index 8a69a270f..000000000 --- a/conf/ppo/task/go2_joystick_flat/base.yaml +++ /dev/null @@ -1,28 +0,0 @@ -# @package _global_ -# Shared cross-backend config for go2_joystick_flat (mujoco/motrix). See AGENTS.md (Sim2Sim). - -algo: - num_envs: 1024 - max_iterations: 151 - empirical_normalization: true - obs_groups: - actor: - - actor - policy: - init_noise_std: 0.5 - algorithm: - learning_rate: 0.0003 - entropy_coef: 0.001 -reward: - scales: - tracking_lin_vel: 1.0 - tracking_ang_vel: 0.2 - lin_vel_z: -5.0 - ang_vel_xy: -0.1 - base_height: -100.0 - action_rate: -0.005 - similar_to_default: -0.1 - contact: 0.24 - swing_feet_z: 4.0 - tracking_sigma: 0.25 - base_height_target: 0.3 diff --git a/conf/ppo/task/go2_joystick_flat/motrix.yaml b/conf/ppo/task/go2_joystick_flat/motrix.yaml index 204b72e45..920428170 100644 --- a/conf/ppo/task/go2_joystick_flat/motrix.yaml +++ b/conf/ppo/task/go2_joystick_flat/motrix.yaml @@ -1,8 +1,29 @@ # @package _global_ -defaults: - - /task/go2_joystick_flat/base - - _self_ - +algo: + num_envs: 1024 + max_iterations: 151 + empirical_normalization: true + obs_groups: + actor: + - actor + policy: + init_noise_std: 0.5 + algorithm: + learning_rate: 0.0003 + entropy_coef: 0.001 +reward: + scales: + tracking_lin_vel: 1.0 + tracking_ang_vel: 0.2 + lin_vel_z: -5.0 + ang_vel_xy: -0.1 + base_height: -100.0 + action_rate: -0.005 + similar_to_default: -0.1 + contact: 0.24 + swing_feet_z: 4.0 + tracking_sigma: 0.25 + base_height_target: 0.3 training: task_name: Go2JoystickFlat sim_backend: motrix @@ -18,3 +39,7 @@ env: domain_rand: randomize_kp: false randomize_kd: false +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2_joystick_flat/motrix_play.yaml b/conf/ppo/task/go2_joystick_flat/motrix_play.yaml deleted file mode 100644 index 87deb99da..000000000 --- a/conf/ppo/task/go2_joystick_flat/motrix_play.yaml +++ /dev/null @@ -1,12 +0,0 @@ -# @package _global_ -defaults: - - /task/go2_joystick_flat/motrix - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/go2_joystick_flat/mujoco.yaml b/conf/ppo/task/go2_joystick_flat/mujoco.yaml index 9217af855..67d77d03f 100644 --- a/conf/ppo/task/go2_joystick_flat/mujoco.yaml +++ b/conf/ppo/task/go2_joystick_flat/mujoco.yaml @@ -1,7 +1,33 @@ # @package _global_ -defaults: - - /task/go2_joystick_flat/base - - _self_ +algo: + num_envs: 1024 + max_iterations: 151 + empirical_normalization: true + obs_groups: + actor: + - actor + policy: + init_noise_std: 0.5 + algorithm: + learning_rate: 0.0003 + entropy_coef: 0.001 +reward: + scales: + tracking_lin_vel: 1.0 + tracking_ang_vel: 0.2 + lin_vel_z: -5.0 + ang_vel_xy: -0.1 + base_height: -100.0 + action_rate: -0.005 + similar_to_default: -0.1 + contact: 0.24 + swing_feet_z: 4.0 + tracking_sigma: 0.25 + base_height_target: 0.3 training: task_name: Go2JoystickFlat sim_backend: mujoco +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2_joystick_flat/mujoco_play.yaml b/conf/ppo/task/go2_joystick_flat/mujoco_play.yaml deleted file mode 100644 index a53110c28..000000000 --- a/conf/ppo/task/go2_joystick_flat/mujoco_play.yaml +++ /dev/null @@ -1,12 +0,0 @@ -# @package _global_ -defaults: - - /task/go2_joystick_flat/mujoco - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/go2_joystick_rough/base.yaml b/conf/ppo/task/go2_joystick_rough/base.yaml deleted file mode 100644 index ca0f520d1..000000000 --- a/conf/ppo/task/go2_joystick_rough/base.yaml +++ /dev/null @@ -1,131 +0,0 @@ -# @package _global_ -# Shared cross-backend config for go2_joystick_rough (mujoco/motrix). See AGENTS.md (Sim2Sim). - -training: - play_steps: 500 - play_env_num: 16 - cam_tracking: true - cam_tracking_env_idx: 0 - cam_tracking_extra_envs: 9 -interactive: - action_mode: policy - policy_obs_mode: auto - camera_follow_body: true - use_env_visual_model: false -algo: - num_envs: 2048 - num_steps_per_env: 24 - max_iterations: 1500 - empirical_normalization: false - obs_groups: - actor: - - actor - critic: - - critic - policy: - init_noise_std: 1.0 - algorithm: - learning_rate: 0.001 - entropy_coef: 0.01 -env: - control_config: - action_scale: 0.25 - hip_action_scale: 0.125 - non_hip_action_scale: 0.25 - clip_actions: 100.0 - commands: - vel_limit: - - - -1.0 - - -1.0 - - -1.0 - - - 1.0 - - 1.0 - - 1.0 - resampling_time: 10.0 - heading_command: true - heading_range: - - -3.141592653589793 - - 3.141592653589793 - rel_standing_envs: 0.1 - terrain_curriculum: - enabled: false - scene: - model_file: src/unilab/assets/robots/go2/go2_mujoco.xml - fragment_files: - - src/unilab/assets/robots/go2/locomotion_task.xml - terrain: - hfield_name: terrain_hfield - geom_name: floor - generator: - seed: 42 - curriculum: false - size: - - 8.0 - - 8.0 - num_rows: 6 - num_cols: 6 - border_width: 20.0 - terrain_scan: - enabled: true - geom_name: floor - termination_config: - terrain_out_of_bounds: true - terrain_distance_buffer: 3.0 - domain_rand: - randomize_base_mass: true - added_mass_range: - - -1.0 - - 3.0 - random_com: true - randomize_kp: true - kp_multiplier_range: - - 0.5 - - 2.0 - randomize_kd: true - kd_multiplier_range: - - 0.5 - - 2.0 - push_robots: true - push_interval: 625 - max_force: - - 1.0 - - 1.0 - - 0.5 -reward: - scales: - lin_vel_z: -2.0 - ang_vel_xy: -0.05 - joint_torques_l2: -2.5e-05 - joint_acc_l2: -2.5e-07 - joint_power: -2.0e-05 - stand_still: -2.0 - hip_pos: -0.5 - joint_pos_penalty: -1.0 - joint_mirror: -0.05 - action_rate: -0.01 - undesired_contacts: -1.0 - contact_forces: -0.00015 - tracking_lin_vel: 3.0 - tracking_ang_vel: 1.5 - feet_air_time: 0.5 - feet_air_time_variance: -1.0 - feet_contact_without_cmd: 0.1 - feet_slide: -0.1 - feet_height_body: -5.0 - feet_gait: 0.5 - upward: 1.0 - tracking_sigma: 0.25 - base_height_target: 0.3 - stand_still_command_threshold: 0.1 - joint_pos_penalty_stand_still_scale: 5.0 - joint_pos_penalty_velocity_threshold: 0.5 - joint_pos_penalty_command_threshold: 0.1 - contact_threshold: 1.0 - contact_forces_threshold: 100.0 - feet_air_time_threshold: 0.5 - feet_height_body_target: -0.2 - feet_height_body_tanh_mult: 2.0 - feet_gait_std: 0.7071067811865476 - feet_gait_max_err: 0.2 - feet_gait_velocity_threshold: 0.5 - feet_gait_command_threshold: 0.1 diff --git a/conf/ppo/task/go2_joystick_rough/motrix.yaml b/conf/ppo/task/go2_joystick_rough/motrix.yaml index b5cf9e676..6b6d07eca 100644 --- a/conf/ppo/task/go2_joystick_rough/motrix.yaml +++ b/conf/ppo/task/go2_joystick_rough/motrix.yaml @@ -1,14 +1,136 @@ # @package _global_ -defaults: - - /task/go2_joystick_rough/base - - _self_ - training: + play_steps: 500 + play_env_num: 16 + cam_tracking: true + cam_tracking_env_idx: 0 + cam_tracking_extra_envs: 9 task_name: Go2JoystickRough sim_backend: motrix +interactive: + action_mode: policy + policy_obs_mode: auto + camera_follow_body: true + use_env_visual_model: false algo: num_envs: 4096 + num_steps_per_env: 24 + max_iterations: 1500 + empirical_normalization: false + obs_groups: + actor: + - actor + critic: + - critic + policy: + init_noise_std: 1.0 + algorithm: + learning_rate: 0.001 + entropy_coef: 0.01 env: - render_offset_mode: zero + control_config: + action_scale: 0.25 + hip_action_scale: 0.125 + non_hip_action_scale: 0.25 + clip_actions: 100.0 + commands: + vel_limit: + - - -1.0 + - -1.0 + - -1.0 + - - 1.0 + - 1.0 + - 1.0 + resampling_time: 10.0 + heading_command: true + heading_range: + - -3.141592653589793 + - 3.141592653589793 + rel_standing_envs: 0.1 + terrain_curriculum: + enabled: false scene: model_file: src/unilab/assets/robots/go2/go2.xml + fragment_files: + - src/unilab/assets/robots/go2/locomotion_task.xml + terrain: + hfield_name: terrain_hfield + geom_name: floor + generator: + seed: 42 + curriculum: false + size: + - 8.0 + - 8.0 + num_rows: 6 + num_cols: 6 + border_width: 20.0 + terrain_scan: + enabled: true + geom_name: floor + termination_config: + terrain_out_of_bounds: true + terrain_distance_buffer: 3.0 + domain_rand: + randomize_base_mass: true + added_mass_range: + - -1.0 + - 3.0 + random_com: true + randomize_kp: true + kp_multiplier_range: + - 0.5 + - 2.0 + randomize_kd: true + kd_multiplier_range: + - 0.5 + - 2.0 + push_robots: true + push_interval: 625 + max_force: + - 1.0 + - 1.0 + - 0.5 + render_offset_mode: zero +reward: + scales: + lin_vel_z: -2.0 + ang_vel_xy: -0.05 + joint_torques_l2: -2.5e-05 + joint_acc_l2: -2.5e-07 + joint_power: -2.0e-05 + stand_still: -2.0 + hip_pos: -0.5 + joint_pos_penalty: -1.0 + joint_mirror: -0.05 + action_rate: -0.01 + undesired_contacts: -1.0 + contact_forces: -0.00015 + tracking_lin_vel: 3.0 + tracking_ang_vel: 1.5 + feet_air_time: 0.5 + feet_air_time_variance: -1.0 + feet_contact_without_cmd: 0.1 + feet_slide: -0.1 + feet_height_body: -5.0 + feet_gait: 0.5 + upward: 1.0 + tracking_sigma: 0.25 + base_height_target: 0.3 + stand_still_command_threshold: 0.1 + joint_pos_penalty_stand_still_scale: 5.0 + joint_pos_penalty_velocity_threshold: 0.5 + joint_pos_penalty_command_threshold: 0.1 + contact_threshold: 1.0 + contact_forces_threshold: 100.0 + feet_air_time_threshold: 0.5 + feet_height_body_target: -0.2 + feet_height_body_tanh_mult: 2.0 + feet_gait_std: 0.7071067811865476 + feet_gait_max_err: 0.2 + feet_gait_velocity_threshold: 0.5 + feet_gait_command_threshold: 0.1 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2_joystick_rough/motrix_play.yaml b/conf/ppo/task/go2_joystick_rough/motrix_play.yaml deleted file mode 100644 index d556ac266..000000000 --- a/conf/ppo/task/go2_joystick_rough/motrix_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/go2_joystick_rough/motrix - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/go2_joystick_rough/mujoco.yaml b/conf/ppo/task/go2_joystick_rough/mujoco.yaml index a5f0411f6..32a3ce0e1 100644 --- a/conf/ppo/task/go2_joystick_rough/mujoco.yaml +++ b/conf/ppo/task/go2_joystick_rough/mujoco.yaml @@ -1,11 +1,137 @@ # @package _global_ -defaults: - - /task/go2_joystick_rough/base - - _self_ - training: + play_steps: 500 + play_env_num: 16 + cam_tracking: true + cam_tracking_env_idx: 0 + cam_tracking_extra_envs: 9 task_name: Go2JoystickRough sim_backend: mujoco render_spacing: 0.0 +interactive: + action_mode: policy + policy_obs_mode: auto + camera_follow_body: true + use_env_visual_model: false +algo: + num_envs: 2048 + num_steps_per_env: 24 + max_iterations: 1500 + empirical_normalization: false + obs_groups: + actor: + - actor + critic: + - critic + policy: + init_noise_std: 1.0 + algorithm: + learning_rate: 0.001 + entropy_coef: 0.01 env: + control_config: + action_scale: 0.25 + hip_action_scale: 0.125 + non_hip_action_scale: 0.25 + clip_actions: 100.0 + commands: + vel_limit: + - - -1.0 + - -1.0 + - -1.0 + - - 1.0 + - 1.0 + - 1.0 + resampling_time: 10.0 + heading_command: true + heading_range: + - -3.141592653589793 + - 3.141592653589793 + rel_standing_envs: 0.1 + terrain_curriculum: + enabled: false + scene: + model_file: src/unilab/assets/robots/go2/go2_mujoco.xml + fragment_files: + - src/unilab/assets/robots/go2/locomotion_task.xml + terrain: + hfield_name: terrain_hfield + geom_name: floor + generator: + seed: 42 + curriculum: false + size: + - 8.0 + - 8.0 + num_rows: 6 + num_cols: 6 + border_width: 20.0 + terrain_scan: + enabled: true + geom_name: floor + termination_config: + terrain_out_of_bounds: true + terrain_distance_buffer: 3.0 + domain_rand: + randomize_base_mass: true + added_mass_range: + - -1.0 + - 3.0 + random_com: true + randomize_kp: true + kp_multiplier_range: + - 0.5 + - 2.0 + randomize_kd: true + kd_multiplier_range: + - 0.5 + - 2.0 + push_robots: true + push_interval: 625 + max_force: + - 1.0 + - 1.0 + - 0.5 sim_dt: 0.002 +reward: + scales: + lin_vel_z: -2.0 + ang_vel_xy: -0.05 + joint_torques_l2: -2.5e-05 + joint_acc_l2: -2.5e-07 + joint_power: -2.0e-05 + stand_still: -2.0 + hip_pos: -0.5 + joint_pos_penalty: -1.0 + joint_mirror: -0.05 + action_rate: -0.01 + undesired_contacts: -1.0 + contact_forces: -0.00015 + tracking_lin_vel: 3.0 + tracking_ang_vel: 1.5 + feet_air_time: 0.5 + feet_air_time_variance: -1.0 + feet_contact_without_cmd: 0.1 + feet_slide: -0.1 + feet_height_body: -5.0 + feet_gait: 0.5 + upward: 1.0 + tracking_sigma: 0.25 + base_height_target: 0.3 + stand_still_command_threshold: 0.1 + joint_pos_penalty_stand_still_scale: 5.0 + joint_pos_penalty_velocity_threshold: 0.5 + joint_pos_penalty_command_threshold: 0.1 + contact_threshold: 1.0 + contact_forces_threshold: 100.0 + feet_air_time_threshold: 0.5 + feet_height_body_target: -0.2 + feet_height_body_tanh_mult: 2.0 + feet_gait_std: 0.7071067811865476 + feet_gait_max_err: 0.2 + feet_gait_velocity_threshold: 0.5 + feet_gait_command_threshold: 0.1 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2_joystick_rough/mujoco_play.yaml b/conf/ppo/task/go2_joystick_rough/mujoco_play.yaml deleted file mode 100644 index 5e0d4b515..000000000 --- a/conf/ppo/task/go2_joystick_rough/mujoco_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/go2_joystick_rough/mujoco - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/go2w_joystick_flat/base.yaml b/conf/ppo/task/go2w_joystick_flat/base.yaml deleted file mode 100644 index 3b8034c90..000000000 --- a/conf/ppo/task/go2w_joystick_flat/base.yaml +++ /dev/null @@ -1,49 +0,0 @@ -# @package _global_ -# Shared cross-backend config for go2w_joystick_flat (mujoco/motrix). See AGENTS.md (Sim2Sim). - -algo: - num_envs: 1024 - max_iterations: 151 - empirical_normalization: true - obs_groups: - actor: - - actor - policy: - init_noise_std: 0.5 - algorithm: - learning_rate: 0.0003 - entropy_coef: 0.001 -env: - commands: - vel_limit: - - - 0.0 - - 0.0 - - -1.0 - - - 1.0 - - 0.0 - - 1.0 - control_config: - action_scale: 0.5 - wheel_action_scale: 10.0 - Kp: 50.0 - Kd: 1.5 - wheel_Kd: 0.5 - domain_rand: - randomize_kp: false - randomize_kd: false -reward: - scales: - tracking_lin_vel: 1.0 - tracking_ang_vel: 0.75 - lin_vel_z: -5.0 - ang_vel_xy: -0.1 - base_height: -100.0 - orientation: -2.0 - action_rate: -0.005 - similar_to_default: -0.5 - torques: -0.0002 - wheel_vel: 0.0 - alive: 0.5 - upward: 1.0 - tracking_sigma: 0.25 - base_height_target: 0.4 diff --git a/conf/ppo/task/go2w_joystick_flat/motrix.yaml b/conf/ppo/task/go2w_joystick_flat/motrix.yaml index 2a98d54d9..1e87e13fd 100644 --- a/conf/ppo/task/go2w_joystick_flat/motrix.yaml +++ b/conf/ppo/task/go2w_joystick_flat/motrix.yaml @@ -1,10 +1,55 @@ # @package _global_ -defaults: - - /task/go2w_joystick_flat/base - - _self_ - +algo: + num_envs: 1024 + max_iterations: 151 + empirical_normalization: true + obs_groups: + actor: + - actor + policy: + init_noise_std: 0.5 + algorithm: + learning_rate: 0.0003 + entropy_coef: 0.001 +env: + commands: + vel_limit: + - - 0.0 + - 0.0 + - -1.0 + - - 1.0 + - 0.0 + - 1.0 + control_config: + action_scale: 0.5 + wheel_action_scale: 10.0 + Kp: 50.0 + Kd: 1.5 + wheel_Kd: 0.5 + domain_rand: + randomize_kp: false + randomize_kd: false + render_offset_mode: zero +reward: + scales: + tracking_lin_vel: 1.0 + tracking_ang_vel: 0.75 + lin_vel_z: -5.0 + ang_vel_xy: -0.1 + base_height: -100.0 + orientation: -2.0 + action_rate: -0.005 + similar_to_default: -0.5 + torques: -0.0002 + wheel_vel: 0.0 + alive: 0.5 + upward: 1.0 + tracking_sigma: 0.25 + base_height_target: 0.4 training: task_name: Go2WJoystickFlat sim_backend: motrix -env: - render_offset_mode: zero +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2w_joystick_flat/motrix_play.yaml b/conf/ppo/task/go2w_joystick_flat/motrix_play.yaml deleted file mode 100644 index e3d3376fa..000000000 --- a/conf/ppo/task/go2w_joystick_flat/motrix_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/go2w_joystick_flat/motrix - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/go2w_joystick_flat/mujoco.yaml b/conf/ppo/task/go2w_joystick_flat/mujoco.yaml index 58cd7b523..548ba07e2 100644 --- a/conf/ppo/task/go2w_joystick_flat/mujoco.yaml +++ b/conf/ppo/task/go2w_joystick_flat/mujoco.yaml @@ -1,7 +1,54 @@ # @package _global_ -defaults: - - /task/go2w_joystick_flat/base - - _self_ +algo: + num_envs: 1024 + max_iterations: 151 + empirical_normalization: true + obs_groups: + actor: + - actor + policy: + init_noise_std: 0.5 + algorithm: + learning_rate: 0.0003 + entropy_coef: 0.001 +env: + commands: + vel_limit: + - - 0.0 + - 0.0 + - -1.0 + - - 1.0 + - 0.0 + - 1.0 + control_config: + action_scale: 0.5 + wheel_action_scale: 10.0 + Kp: 50.0 + Kd: 1.5 + wheel_Kd: 0.5 + domain_rand: + randomize_kp: false + randomize_kd: false +reward: + scales: + tracking_lin_vel: 1.0 + tracking_ang_vel: 0.75 + lin_vel_z: -5.0 + ang_vel_xy: -0.1 + base_height: -100.0 + orientation: -2.0 + action_rate: -0.005 + similar_to_default: -0.5 + torques: -0.0002 + wheel_vel: 0.0 + alive: 0.5 + upward: 1.0 + tracking_sigma: 0.25 + base_height_target: 0.4 training: task_name: Go2WJoystickFlat sim_backend: mujoco +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2w_joystick_flat/mujoco_play.yaml b/conf/ppo/task/go2w_joystick_flat/mujoco_play.yaml deleted file mode 100644 index e906eb02d..000000000 --- a/conf/ppo/task/go2w_joystick_flat/mujoco_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/go2w_joystick_flat/mujoco - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/go2w_joystick_rough/base.yaml b/conf/ppo/task/go2w_joystick_rough/base.yaml deleted file mode 100644 index ccd82b628..000000000 --- a/conf/ppo/task/go2w_joystick_rough/base.yaml +++ /dev/null @@ -1,113 +0,0 @@ -# @package _global_ -# Shared cross-backend config for go2w_joystick_rough (mujoco/motrix). See AGENTS.md (Sim2Sim). - -training: - play_steps: 500 - play_env_num: 16 - cam_tracking: true - cam_tracking_env_idx: 0 - cam_tracking_extra_envs: 9 -interactive: - action_mode: policy - policy_obs_mode: auto - camera_follow_body: true - use_env_visual_model: false -algo: - num_envs: 2048 - num_steps_per_env: 24 - max_iterations: 1200 - empirical_normalization: false - obs_groups: - actor: - - actor - critic: - - critic -env: - scene: - model_file: src/unilab/assets/robots/go2w/go2w_mujoco.xml - fragment_files: - - src/unilab/assets/robots/go2w/locomotion_task.xml - terrain: - hfield_name: terrain_hfield - geom_name: floor - generator: - seed: 42 - curriculum: false - size: - - 8.0 - - 8.0 - num_rows: 6 - num_cols: 6 - border_width: 20.0 - commands: - vel_limit: - - - -1.0 - - -1.0 - - -1.0 - - - 1.0 - - 1.0 - - 1.0 - resampling_time: 10.0 - heading_command: true - heading_range: - - -3.141592653589793 - - 3.141592653589793 - rel_standing_envs: 0.1 - control_config: - action_scale: 0.25 - hip_action_scale: 0.125 - wheel_action_scale: 5.0 - wheel_Kd: 0.5 - clip_actions: 100.0 - simulate_action_latency: false - terrain_scan: - enabled: true - hfield_name: terrain_hfield - geom_name: floor - termination_config: - terrain_out_of_bounds: true - terrain_distance_buffer: 3.0 - domain_rand: - randomize_base_mass: true - added_mass_range: - - -1.0 - - 3.0 - random_com: true - com_offset_x: - - -0.05 - - 0.05 - randomize_kp: true - kp_multiplier_range: - - 0.5 - - 1.0 - randomize_kd: true - kd_multiplier_range: - - 0.5 - - 1.0 - push_robots: true - push_interval: 625 - max_force: - - 1.0 - - 1.0 - - 0.5 - push_body_name: base_link -reward: - scales: - tracking_lin_vel: 3.0 - tracking_ang_vel: 1.5 - lin_vel_z: -2.0 - ang_vel_xy: -0.05 - orientation: -2.0 - joint_torques_l2: -2.5e-05 - joint_acc_l2: -2.5e-07 - joint_acc_wheel_l2: -2.5e-09 - joint_power: -2.0e-05 - action_rate: -0.01 - stand_still: -2.0 - hip_pos: -2.0 - joint_pos_penalty: -1.0 - joint_mirror: -0.05 - upward: 1.0 - tracking_sigma: 0.25 - base_height_target: 0.4 - only_positive_rewards: false diff --git a/conf/ppo/task/go2w_joystick_rough/motrix.yaml b/conf/ppo/task/go2w_joystick_rough/motrix.yaml index 209bb8c9c..30bcfcd8c 100644 --- a/conf/ppo/task/go2w_joystick_rough/motrix.yaml +++ b/conf/ppo/task/go2w_joystick_rough/motrix.yaml @@ -1,15 +1,118 @@ # @package _global_ -defaults: - - /task/go2w_joystick_rough/base - - _self_ - training: + play_steps: 500 + play_env_num: 16 + cam_tracking: true + cam_tracking_env_idx: 0 + cam_tracking_extra_envs: 9 task_name: Go2WJoystickRough sim_backend: motrix +interactive: + action_mode: policy + policy_obs_mode: auto + camera_follow_body: true + use_env_visual_model: false +algo: + num_envs: 2048 + num_steps_per_env: 24 + max_iterations: 1200 + empirical_normalization: false + obs_groups: + actor: + - actor + critic: + - critic env: - render_offset_mode: zero scene: model_file: src/unilab/assets/robots/go2w/go2w.xml + fragment_files: + - src/unilab/assets/robots/go2w/locomotion_task.xml + terrain: + hfield_name: terrain_hfield + geom_name: floor + generator: + seed: 42 + curriculum: false + size: + - 8.0 + - 8.0 + num_rows: 6 + num_cols: 6 + border_width: 20.0 + commands: + vel_limit: + - - -1.0 + - -1.0 + - -1.0 + - - 1.0 + - 1.0 + - 1.0 + resampling_time: 10.0 + heading_command: true + heading_range: + - -3.141592653589793 + - 3.141592653589793 + rel_standing_envs: 0.1 + control_config: + action_scale: 0.25 + hip_action_scale: 0.125 + wheel_action_scale: 5.0 + wheel_Kd: 0.5 + clip_actions: 100.0 + simulate_action_latency: false + terrain_scan: + enabled: true + hfield_name: terrain_hfield + geom_name: floor + termination_config: + terrain_out_of_bounds: true + terrain_distance_buffer: 3.0 + domain_rand: + randomize_base_mass: true + added_mass_range: + - -1.0 + - 3.0 + random_com: true + com_offset_x: + - -0.05 + - 0.05 + randomize_kp: true + kp_multiplier_range: + - 0.5 + - 1.0 + randomize_kd: true + kd_multiplier_range: + - 0.5 + - 1.0 + push_robots: true + push_interval: 625 + max_force: + - 1.0 + - 1.0 + - 0.5 + push_body_name: base_link + render_offset_mode: zero reward: scales: + tracking_lin_vel: 3.0 + tracking_ang_vel: 1.5 + lin_vel_z: -2.0 + ang_vel_xy: -0.05 + orientation: -2.0 + joint_torques_l2: -2.5e-05 + joint_acc_l2: -2.5e-07 + joint_acc_wheel_l2: -2.5e-09 + joint_power: -2.0e-05 + action_rate: -0.01 + stand_still: -2.0 hip_pos: -0.5 + joint_pos_penalty: -1.0 + joint_mirror: -0.05 + upward: 1.0 + tracking_sigma: 0.25 + base_height_target: 0.4 + only_positive_rewards: false +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2w_joystick_rough/motrix_play.yaml b/conf/ppo/task/go2w_joystick_rough/motrix_play.yaml deleted file mode 100644 index 1df1d2aac..000000000 --- a/conf/ppo/task/go2w_joystick_rough/motrix_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/go2w_joystick_rough/motrix - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/go2w_joystick_rough/mujoco.yaml b/conf/ppo/task/go2w_joystick_rough/mujoco.yaml index a90a37c62..628d64895 100644 --- a/conf/ppo/task/go2w_joystick_rough/mujoco.yaml +++ b/conf/ppo/task/go2w_joystick_rough/mujoco.yaml @@ -1,9 +1,118 @@ # @package _global_ -defaults: - - /task/go2w_joystick_rough/base - - _self_ - training: + play_steps: 500 + play_env_num: 16 + cam_tracking: true + cam_tracking_env_idx: 0 + cam_tracking_extra_envs: 9 task_name: Go2WJoystickRough sim_backend: mujoco render_spacing: 0.0 +interactive: + action_mode: policy + policy_obs_mode: auto + camera_follow_body: true + use_env_visual_model: false +algo: + num_envs: 2048 + num_steps_per_env: 24 + max_iterations: 1200 + empirical_normalization: false + obs_groups: + actor: + - actor + critic: + - critic +env: + scene: + model_file: src/unilab/assets/robots/go2w/go2w_mujoco.xml + fragment_files: + - src/unilab/assets/robots/go2w/locomotion_task.xml + terrain: + hfield_name: terrain_hfield + geom_name: floor + generator: + seed: 42 + curriculum: false + size: + - 8.0 + - 8.0 + num_rows: 6 + num_cols: 6 + border_width: 20.0 + commands: + vel_limit: + - - -1.0 + - -1.0 + - -1.0 + - - 1.0 + - 1.0 + - 1.0 + resampling_time: 10.0 + heading_command: true + heading_range: + - -3.141592653589793 + - 3.141592653589793 + rel_standing_envs: 0.1 + control_config: + action_scale: 0.25 + hip_action_scale: 0.125 + wheel_action_scale: 5.0 + wheel_Kd: 0.5 + clip_actions: 100.0 + simulate_action_latency: false + terrain_scan: + enabled: true + hfield_name: terrain_hfield + geom_name: floor + termination_config: + terrain_out_of_bounds: true + terrain_distance_buffer: 3.0 + domain_rand: + randomize_base_mass: true + added_mass_range: + - -1.0 + - 3.0 + random_com: true + com_offset_x: + - -0.05 + - 0.05 + randomize_kp: true + kp_multiplier_range: + - 0.5 + - 1.0 + randomize_kd: true + kd_multiplier_range: + - 0.5 + - 1.0 + push_robots: true + push_interval: 625 + max_force: + - 1.0 + - 1.0 + - 0.5 + push_body_name: base_link +reward: + scales: + tracking_lin_vel: 3.0 + tracking_ang_vel: 1.5 + lin_vel_z: -2.0 + ang_vel_xy: -0.05 + orientation: -2.0 + joint_torques_l2: -2.5e-05 + joint_acc_l2: -2.5e-07 + joint_acc_wheel_l2: -2.5e-09 + joint_power: -2.0e-05 + action_rate: -0.01 + stand_still: -2.0 + hip_pos: -2.0 + joint_pos_penalty: -1.0 + joint_mirror: -0.05 + upward: 1.0 + tracking_sigma: 0.25 + base_height_target: 0.4 + only_positive_rewards: false +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/go2w_joystick_rough/mujoco_play.yaml b/conf/ppo/task/go2w_joystick_rough/mujoco_play.yaml deleted file mode 100644 index 473eb4b36..000000000 --- a/conf/ppo/task/go2w_joystick_rough/mujoco_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/go2w_joystick_rough/mujoco - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/sharpa_inhand/motrix.yaml b/conf/ppo/task/sharpa_inhand/motrix.yaml index a71dea555..91cbf1436 100644 --- a/conf/ppo/task/sharpa_inhand/motrix.yaml +++ b/conf/ppo/task/sharpa_inhand/motrix.yaml @@ -20,3 +20,7 @@ env: randomize_gravity: false randomize_gravity_direction: true randomize_pd_gains: true +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/sharpa_inhand/motrix_play.yaml b/conf/ppo/task/sharpa_inhand/motrix_play.yaml deleted file mode 100644 index e3def6b0c..000000000 --- a/conf/ppo/task/sharpa_inhand/motrix_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/sharpa_inhand/motrix - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/sharpa_inhand/mujoco.yaml b/conf/ppo/task/sharpa_inhand/mujoco.yaml index 7ee973e50..638b8aaa4 100644 --- a/conf/ppo/task/sharpa_inhand/mujoco.yaml +++ b/conf/ppo/task/sharpa_inhand/mujoco.yaml @@ -99,3 +99,7 @@ env: joint_noise_scale: 0.02 contact_latency: 0.005 contact_sensor_noise: 0.01 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/sharpa_inhand/mujoco_play.yaml b/conf/ppo/task/sharpa_inhand/mujoco_play.yaml deleted file mode 100644 index 6712b025b..000000000 --- a/conf/ppo/task/sharpa_inhand/mujoco_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/sharpa_inhand/mujoco - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/sharpa_inhand_grasp/motrix.yaml b/conf/ppo/task/sharpa_inhand_grasp/motrix.yaml index 8002dfe89..86168005b 100644 --- a/conf/ppo/task/sharpa_inhand_grasp/motrix.yaml +++ b/conf/ppo/task/sharpa_inhand_grasp/motrix.yaml @@ -12,3 +12,7 @@ env: domain_rand: randomize_gravity: false randomize_gravity_direction: false +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/sharpa_inhand_grasp/motrix_play.yaml b/conf/ppo/task/sharpa_inhand_grasp/motrix_play.yaml deleted file mode 100644 index 2fd21b8d9..000000000 --- a/conf/ppo/task/sharpa_inhand_grasp/motrix_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/sharpa_inhand_grasp/motrix - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/sharpa_inhand_grasp/mujoco.yaml b/conf/ppo/task/sharpa_inhand_grasp/mujoco.yaml index 690c9c288..871b4be0a 100644 --- a/conf/ppo/task/sharpa_inhand_grasp/mujoco.yaml +++ b/conf/ppo/task/sharpa_inhand_grasp/mujoco.yaml @@ -100,3 +100,7 @@ env: - contact_right_pinky_elastomer_force disable_tactile_ids: [] grasp_collection_target: 10000 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/sharpa_inhand_grasp/mujoco_play.yaml b/conf/ppo/task/sharpa_inhand_grasp/mujoco_play.yaml deleted file mode 100644 index 08e7019e6..000000000 --- a/conf/ppo/task/sharpa_inhand_grasp/mujoco_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/sharpa_inhand_grasp/mujoco - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/stewart_balance/motrix.yaml b/conf/ppo/task/stewart_balance/motrix.yaml index 311be10be..f018225f8 100644 --- a/conf/ppo/task/stewart_balance/motrix.yaml +++ b/conf/ppo/task/stewart_balance/motrix.yaml @@ -46,3 +46,7 @@ reward: progress: 0.6 still: 3.0 fall_penalty: -6.0 +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/stewart_balance/motrix_play.yaml b/conf/ppo/task/stewart_balance/motrix_play.yaml deleted file mode 100644 index 8025b60c3..000000000 --- a/conf/ppo/task/stewart_balance/motrix_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/stewart_balance/motrix - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/conf/ppo/task/stewart_balance/mujoco.yaml b/conf/ppo/task/stewart_balance/mujoco.yaml index 4d3acc67f..f4facc798 100644 --- a/conf/ppo/task/stewart_balance/mujoco.yaml +++ b/conf/ppo/task/stewart_balance/mujoco.yaml @@ -11,3 +11,7 @@ defaults: training: task_name: StewartBalance sim_backend: mujoco +play_profile: + enabled: true + env: + render_spacing: 2.0 diff --git a/conf/ppo/task/stewart_balance/mujoco_play.yaml b/conf/ppo/task/stewart_balance/mujoco_play.yaml deleted file mode 100644 index 040964c0d..000000000 --- a/conf/ppo/task/stewart_balance/mujoco_play.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# @package _global_ - -defaults: - - /task/stewart_balance/mujoco - - _self_ - -training: - play_only: true - -play_profile: - enabled: true - env: - render_spacing: 2.0 diff --git a/src/unilab/cli.py b/src/unilab/cli.py index c5c60de77..e3b03897e 100644 --- a/src/unilab/cli.py +++ b/src/unilab/cli.py @@ -233,15 +233,6 @@ def build_command( if not script.is_file(): raise SystemExit(f"Entrypoint script not found: {script}") - # In eval mode, prefer the decoupled render/play config (`_play.yaml`) - # when one exists for this task, so eval automatically uses the play profile - # while train never does. Fall back to the bare backend owner if no `_play` - # variant is present. An explicit --profile always takes precedence. - if mode == "eval" and profile is None: - play_route = build_route(algo, task, sim, "play") - if _owner_yaml_path(play_route, selected_root).is_file(): - route = play_route - owner_yaml = _owner_yaml_path(route, selected_root) if not owner_yaml.is_file(): raise SystemExit( diff --git a/tests/training/test_sim2sim_resolver.py b/tests/training/test_sim2sim_resolver.py index 733160105..ef31e4137 100644 --- a/tests/training/test_sim2sim_resolver.py +++ b/tests/training/test_sim2sim_resolver.py @@ -329,7 +329,7 @@ def _compose_task(task: str) -> Any: def test_g1_walk_flat_mujoco_inherits_base_contract(): - # The MuJoCo owner must inherit the shared contract from base.yaml verbatim. + # The MuJoCo owner carries the full contract in its standalone owner config. mujoco = _compose_task("g1_walk_flat/mujoco") assert OmegaConf.select(mujoco, "env.control_config.action_scale") == 0.25 assert OmegaConf.select(mujoco, "algo.empirical_normalization") is False diff --git a/uv b/uv new file mode 100644 index 000000000..e69de29bb From bcc6f2c875d6e1e9c8748837e17812c7eff58190 Mon Sep 17 00:00:00 2001 From: LeeLeno <1846301824@qq.com> Date: Thu, 2 Jul 2026 17:52:53 +0800 Subject: [PATCH 3/3] fix:v3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 回退 YAML 格式噪声到 base 风格(flow style) - 修正过时文档:base.yaml 已删,改为 owner 自包含契约 - 删除误提交空文件 uv Co-Authored-By: Claude Opus 4.8 --- AGENTS.md | 2 +- conf/appo/task/allegro_inhand/motrix.yaml | 20 +-- conf/appo/task/allegro_inhand/mujoco.yaml | 20 +-- conf/appo/task/g1_climb_tracking/motrix.yaml | 2 +- conf/appo/task/g1_climb_tracking/mujoco.yaml | 2 +- conf/appo/task/g1_flip_tracking/motrix.yaml | 62 +++---- conf/appo/task/g1_flip_tracking/mujoco.yaml | 64 +++---- conf/appo/task/g1_motion_tracking/motrix.yaml | 2 +- conf/appo/task/g1_motion_tracking/mujoco.yaml | 2 +- .../task/g1_wall_flip_tracking/motrix.yaml | 4 +- .../task/g1_wall_flip_tracking/mujoco.yaml | 90 +++++----- conf/appo/task/go1_joystick_flat/motrix.yaml | 30 ++-- conf/appo/task/go1_joystick_flat/mujoco.yaml | 6 +- conf/appo/task/go2_joystick_flat/motrix.yaml | 12 +- conf/appo/task/go2_joystick_flat/mujoco.yaml | 6 +- .../task/sac/g1_walk_flat/motrix.yaml | 45 +---- .../task/sac/g1_walk_flat/mujoco.yaml | 39 +--- .../task/sac/g1_walk_rough/motrix.yaml | 47 +---- .../task/sac/g1_walk_rough/mujoco.yaml | 39 +--- conf/ppo/task/allegro_inhand/motrix.yaml | 23 +-- conf/ppo/task/allegro_inhand/mujoco.yaml | 34 ++-- conf/ppo/task/g1_box_tracking/motrix.yaml | 38 ++-- conf/ppo/task/g1_box_tracking/mujoco.yaml | 14 +- conf/ppo/task/g1_climb_tracking/motrix.yaml | 2 +- conf/ppo/task/g1_climb_tracking/mujoco.yaml | 64 +++---- conf/ppo/task/g1_flip_tracking/motrix.yaml | 4 +- conf/ppo/task/g1_flip_tracking/mujoco.yaml | 90 +++++----- conf/ppo/task/g1_motion_tracking/motrix.yaml | 34 ++-- conf/ppo/task/g1_motion_tracking/mujoco.yaml | 4 +- conf/ppo/task/g1_walk_flat/motrix.yaml | 43 ++--- conf/ppo/task/g1_walk_flat/mujoco.yaml | 55 ++---- .../task/g1_wall_flip_tracking/motrix.yaml | 96 +++++----- .../task/g1_wall_flip_tracking/mujoco.yaml | 64 +++---- conf/ppo/task/go1_joystick_flat/motrix.yaml | 26 ++- conf/ppo/task/go1_joystick_flat/mujoco.yaml | 12 +- conf/ppo/task/go1_joystick_rough/motrix.yaml | 61 +++---- conf/ppo/task/go1_joystick_rough/mujoco.yaml | 63 +++---- conf/ppo/task/go2_arm_manip_loco/motrix.yaml | 168 ++++++------------ conf/ppo/task/go2_arm_manip_loco/mujoco.yaml | 142 ++++++--------- conf/ppo/task/go2_joystick_flat/motrix.yaml | 32 ++-- conf/ppo/task/go2_joystick_flat/mujoco.yaml | 12 +- conf/ppo/task/go2_joystick_rough/motrix.yaml | 60 +++---- conf/ppo/task/go2_joystick_rough/mujoco.yaml | 63 +++---- conf/ppo/task/go2w_joystick_flat/motrix.yaml | 22 +-- conf/ppo/task/go2w_joystick_flat/mujoco.yaml | 20 +-- conf/ppo/task/go2w_joystick_rough/motrix.yaml | 59 +++--- conf/ppo/task/go2w_joystick_rough/mujoco.yaml | 59 +++--- .../en/3-deployment/2-sim_to_sim/0-index.md | 7 + .../2-sim_to_sim/7-config_guard.md | 47 +++++ .../3-deployment/2-sim_to_sim/0-index.md | 7 + .../2-sim_to_sim/7-config_guard.md | 47 +++++ .../9-sim2sim_contract_status.md | 8 +- uv | 0 53 files changed, 831 insertions(+), 1143 deletions(-) create mode 100644 docs/sphinx/source/en/3-deployment/2-sim_to_sim/7-config_guard.md create mode 100644 docs/sphinx/source/zh_CN/3-deployment/2-sim_to_sim/7-config_guard.md delete mode 100644 uv diff --git a/AGENTS.md b/AGENTS.md index 227770bd4..c96a0e92c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -34,7 +34,7 @@ UniLab 是一个 **高性能、模块化、contract 驱动** 的 RL infrastructu - **WARNING_LIST**：`reward.*`、`env.control_config.simulate_action_latency`、`env.ctrl_dt`。 - **ALLOWLIST**（自由覆盖）：`training.sim_backend`、`env.scene`、`training.play_steps`、`env.domain_rand`、`env.noise_config`、`env.commands.vel_limit`。 -训练时 `ExperimentTracker.start()` 把上述字段写入 `run_config.json` 的 `contract_snapshot`（不改 checkpoint 格式，旧 run 无 snapshot 时 fallback + warning）；五个 play 入口在建 env 前调用 `resolve_sim2sim_config` 校验，并用 `policy_load_dim_guard` 包裹 checkpoint 加载以把维度不匹配的隐晦报错重抛为显式诊断。设 `training.sim2sim_strict=false` 可把 DENYLIST 差异降级为 warning（默认 `true`）。DENYLIST 字段应通过 task 的 `base.yaml` 共享（范例：`conf/ppo/task/g1_walk_flat/{base,mujoco,motrix}.yaml`）；跨后端契约审计见 `scripts/audit_sim2sim_contracts.py`。 +训练时 `ExperimentTracker.start()` 把上述字段写入 `run_config.json` 的 `contract_snapshot`（不改 checkpoint 格式，旧 run 无 snapshot 时 fallback + warning）；五个 play 入口在建 env 前调用 `resolve_sim2sim_config` 校验，并用 `policy_load_dim_guard` 包裹 checkpoint 加载以把维度不匹配的隐晦报错重抛为显式诊断。设 `training.sim2sim_strict=false` 可把 DENYLIST 差异降级为 warning（默认 `true`）。DENYLIST 字段在每个后端 owner 配置中显式声明并保持跨后端一致（范例：`conf/ppo/task/g1_walk_flat/{mujoco,motrix}.yaml`）；跨后端契约审计见 `scripts/audit_sim2sim_contracts.py`。 ## Pointers diff --git a/conf/appo/task/allegro_inhand/motrix.yaml b/conf/appo/task/allegro_inhand/motrix.yaml index a582b495d..c027729a8 100644 --- a/conf/appo/task/allegro_inhand/motrix.yaml +++ b/conf/appo/task/allegro_inhand/motrix.yaml @@ -1,15 +1,12 @@ # @package _global_ training: + task_name: AllegroInhandRotation + sim_backend: motrix play_steps: 200 render_spacing: 0.5 cam_distance: 1.5 - cam_lookat: - - 0.75 - - 0.75 - - 0 + cam_lookat: [0.75, 0.75, 0] cam_elevation: -20.0 - task_name: AllegroInhandRotation - sim_backend: motrix algo: num_envs: 16384 steps_per_env: 8 @@ -31,10 +28,7 @@ algo: use_clipped_value_loss: true schedule: adaptive actor: - hidden_dims: - - 512 - - 256 - - 128 + hidden_dims: [512, 256, 128] activation: elu obs_normalization: true distribution_cfg: @@ -42,10 +36,7 @@ algo: init_std: 1.0 std_type: scalar critic: - hidden_dims: - - 512 - - 256 - - 128 + hidden_dims: [512, 256, 128] activation: elu obs_normalization: true reward: @@ -63,6 +54,7 @@ env: gen_grasp: false max_episode_seconds: 20.0 grasp_cache_path: caches/allegro_grasp_50k.npy + # Keep only grasp/pose reset variation. All online DR terms stay disabled. domain_rand: randomize_base_mass: false random_com: false diff --git a/conf/appo/task/allegro_inhand/mujoco.yaml b/conf/appo/task/allegro_inhand/mujoco.yaml index f884f40ff..c24a442bc 100644 --- a/conf/appo/task/allegro_inhand/mujoco.yaml +++ b/conf/appo/task/allegro_inhand/mujoco.yaml @@ -1,15 +1,12 @@ # @package _global_ training: + task_name: AllegroInhandRotation + sim_backend: mujoco play_steps: 200 render_spacing: 0.5 cam_distance: 0.75 - cam_lookat: - - 0.0 - - 0.0 - - 0.15 + cam_lookat: [0.0, 0.0, 0.15] cam_elevation: -25.0 - task_name: AllegroInhandRotation - sim_backend: mujoco cam_azimuth: 45.0 replay_queue_size: 4 algo: @@ -33,10 +30,7 @@ algo: use_clipped_value_loss: true schedule: adaptive actor: - hidden_dims: - - 512 - - 256 - - 128 + hidden_dims: [512, 256, 128] activation: elu obs_normalization: true distribution_cfg: @@ -44,10 +38,7 @@ algo: init_std: 1.0 std_type: scalar critic: - hidden_dims: - - 512 - - 256 - - 128 + hidden_dims: [512, 256, 128] activation: elu obs_normalization: true reward: @@ -65,6 +56,7 @@ env: gen_grasp: false max_episode_seconds: 20.0 grasp_cache_path: caches/allegro_grasp_50k.npy + # Keep only grasp/pose reset variation. All online DR terms stay disabled. domain_rand: randomize_base_mass: false random_com: false diff --git a/conf/appo/task/g1_climb_tracking/motrix.yaml b/conf/appo/task/g1_climb_tracking/motrix.yaml index f90b7aa91..ec5684fe2 100644 --- a/conf/appo/task/g1_climb_tracking/motrix.yaml +++ b/conf/appo/task/g1_climb_tracking/motrix.yaml @@ -1,8 +1,8 @@ # @package _global_ training: - play_steps: 1000 task_name: G1ClimbTracking sim_backend: motrix + play_steps: 1000 algo: num_envs: 1024 max_iterations: 20000 diff --git a/conf/appo/task/g1_climb_tracking/mujoco.yaml b/conf/appo/task/g1_climb_tracking/mujoco.yaml index d36baf4ff..bff7b13c8 100644 --- a/conf/appo/task/g1_climb_tracking/mujoco.yaml +++ b/conf/appo/task/g1_climb_tracking/mujoco.yaml @@ -1,8 +1,8 @@ # @package _global_ training: - play_steps: 1000 task_name: G1ClimbTracking sim_backend: mujoco + play_steps: 1000 algo: num_envs: 1024 max_iterations: 20000 diff --git a/conf/appo/task/g1_flip_tracking/motrix.yaml b/conf/appo/task/g1_flip_tracking/motrix.yaml index 67ea8e630..9b2fd0c57 100644 --- a/conf/appo/task/g1_flip_tracking/motrix.yaml +++ b/conf/appo/task/g1_flip_tracking/motrix.yaml @@ -1,8 +1,8 @@ # @package _global_ training: - play_steps: 1000 task_name: G1FlipTracking sim_backend: motrix + play_steps: 1000 algo: num_envs: 1024 steps_per_env: 24 @@ -16,7 +16,7 @@ algo: lam: 0.95 value_loss_coef: 1.0 entropy_coef: 0.005 - learning_rate: 0.001 + learning_rate: 1.0e-3 max_grad_norm: 1.0 use_clipped_value_loss: true schedule: adaptive @@ -35,35 +35,35 @@ env: sim_dt: 0.005 control_config: action_scale: - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 anchor_pos_z_threshold: 0.5 ee_body_pos_z_threshold: 0.5 terminate_on_undesired_contacts: true diff --git a/conf/appo/task/g1_flip_tracking/mujoco.yaml b/conf/appo/task/g1_flip_tracking/mujoco.yaml index 1389affde..628bdbb12 100644 --- a/conf/appo/task/g1_flip_tracking/mujoco.yaml +++ b/conf/appo/task/g1_flip_tracking/mujoco.yaml @@ -1,8 +1,8 @@ # @package _global_ training: - play_steps: 1000 task_name: G1FlipTracking sim_backend: mujoco + play_steps: 1000 algo: num_envs: 1024 steps_per_env: 24 @@ -16,7 +16,7 @@ algo: lam: 0.95 value_loss_coef: 1.0 entropy_coef: 0.005 - learning_rate: 0.001 + learning_rate: 1.0e-3 max_grad_norm: 1.0 use_clipped_value_loss: true schedule: adaptive @@ -35,35 +35,35 @@ env: sim_dt: 0.005 control_config: action_scale: - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 anchor_pos_z_threshold: 0.5 ee_body_pos_z_threshold: 0.5 terminate_on_undesired_contacts: true @@ -89,4 +89,4 @@ reward: std_body_lin_vel: 1.0 std_body_ang_vel: 3.14 std_joint_pos: 0.2 - std_joint_vel: 1.0 + std_joint_vel: 1.0 \ No newline at end of file diff --git a/conf/appo/task/g1_motion_tracking/motrix.yaml b/conf/appo/task/g1_motion_tracking/motrix.yaml index 5765dbc3d..567b0d745 100644 --- a/conf/appo/task/g1_motion_tracking/motrix.yaml +++ b/conf/appo/task/g1_motion_tracking/motrix.yaml @@ -1,8 +1,8 @@ # @package _global_ training: - play_steps: 1000 task_name: G1MotionTracking sim_backend: motrix + play_steps: 1000 algo: num_envs: 1024 max_iterations: 5000 diff --git a/conf/appo/task/g1_motion_tracking/mujoco.yaml b/conf/appo/task/g1_motion_tracking/mujoco.yaml index fa5c20b02..4f2dc4843 100644 --- a/conf/appo/task/g1_motion_tracking/mujoco.yaml +++ b/conf/appo/task/g1_motion_tracking/mujoco.yaml @@ -1,8 +1,8 @@ # @package _global_ training: - play_steps: 1000 task_name: G1MotionTracking sim_backend: mujoco + play_steps: 1000 algo: num_envs: 1024 max_iterations: 5000 diff --git a/conf/appo/task/g1_wall_flip_tracking/motrix.yaml b/conf/appo/task/g1_wall_flip_tracking/motrix.yaml index 06d5e8721..83cca7b39 100644 --- a/conf/appo/task/g1_wall_flip_tracking/motrix.yaml +++ b/conf/appo/task/g1_wall_flip_tracking/motrix.yaml @@ -1,8 +1,8 @@ # @package _global_ training: - play_steps: 1000 task_name: G1WallFlipTracking sim_backend: motrix + play_steps: 1000 algo: num_envs: 1024 max_iterations: 5000 @@ -15,7 +15,7 @@ algo: lam: 0.95 value_loss_coef: 1.0 entropy_coef: 0.01 - learning_rate: 0.001 + learning_rate: 1.0e-3 max_grad_norm: 1.0 use_clipped_value_loss: true schedule: adaptive diff --git a/conf/appo/task/g1_wall_flip_tracking/mujoco.yaml b/conf/appo/task/g1_wall_flip_tracking/mujoco.yaml index 62101d6b2..0083729c0 100644 --- a/conf/appo/task/g1_wall_flip_tracking/mujoco.yaml +++ b/conf/appo/task/g1_wall_flip_tracking/mujoco.yaml @@ -1,11 +1,12 @@ # @package _global_ training: - play_steps: 1000 task_name: G1WallFlipTracking sim_backend: mujoco + play_steps: 1000 replay_queue_size: 5 algo: num_envs: 1024 + steps_per_env: 20 max_iterations: 7000 save_interval: 500 algorithm: @@ -16,7 +17,7 @@ algo: lam: 0.95 value_loss_coef: 1.0 entropy_coef: 0.005 - learning_rate: 0.001 + learning_rate: 1.0e-3 max_grad_norm: 1.0 use_clipped_value_loss: true schedule: adaptive @@ -29,7 +30,46 @@ algo: vtrace_clip_rho: 1.0 vtrace_clip_c: 1.0 enable_compile: true - steps_per_env: 20 +env: + sampling_mode: start + truncate_on_clip_end: true + sim_dt: 0.005 + control_config: + action_scale: + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + anchor_pos_z_threshold: 0.5 + ee_body_pos_z_threshold: 0.5 + terminate_on_undesired_contacts: true + noise_config: + level: 0.0 reward: scales: motion_global_root_pos: 0.5 @@ -38,11 +78,11 @@ reward: motion_body_ori: 1.5 motion_body_lin_vel: 1.0 motion_body_ang_vel: 1.0 + motion_ee_body_pos_z: 2.0 motion_joint_pos: 0.0 motion_joint_vel: 0.0 action_rate_l2: -0.005 joint_limit: -10.0 - motion_ee_body_pos_z: 2.0 std_root_pos: 0.3 std_root_ori: 0.4 std_body_pos: 0.3 @@ -50,44 +90,4 @@ reward: std_body_lin_vel: 1.0 std_body_ang_vel: 3.14 std_joint_pos: 0.2 - std_joint_vel: 1.0 -env: - sampling_mode: start - truncate_on_clip_end: true - sim_dt: 0.005 - control_config: - action_scale: - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - anchor_pos_z_threshold: 0.5 - ee_body_pos_z_threshold: 0.5 - terminate_on_undesired_contacts: true - noise_config: - level: 0.0 + std_joint_vel: 1.0 \ No newline at end of file diff --git a/conf/appo/task/go1_joystick_flat/motrix.yaml b/conf/appo/task/go1_joystick_flat/motrix.yaml index f16211730..f7343fd5d 100644 --- a/conf/appo/task/go1_joystick_flat/motrix.yaml +++ b/conf/appo/task/go1_joystick_flat/motrix.yaml @@ -1,15 +1,24 @@ # @package _global_ +training: + task_name: Go1JoystickFlat + sim_backend: motrix algo: - max_iterations: 300 num_envs: 1024 steps_per_env: 24 + max_iterations: 300 actor: distribution_cfg: init_std: 0.5 algorithm: - learning_rate: 0.0005 - entropy_coef: 0.001 + learning_rate: 5.0e-4 + entropy_coef: 1.0e-3 desired_kl: 0.008 +env: + sim_dt: 0.01 + commands: + vel_limit: + - [0.5, 0.0, 0.0] + - [0.5, 0.0, 0.0] reward: scales: tracking_lin_vel: 1.0 @@ -18,21 +27,8 @@ reward: ang_vel_xy: -0.1 base_height: -100.0 action_rate: -0.015 - similar_to_default: -0.15 action_smooth: -0.01 + similar_to_default: -0.15 swing_feet_z: 2.0 tracking_sigma: 0.25 base_height_target: 0.3 -training: - task_name: Go1JoystickFlat - sim_backend: motrix -env: - sim_dt: 0.01 - commands: - vel_limit: - - - 0.5 - - 0.0 - - 0.0 - - - 0.5 - - 0.0 - - 0.0 diff --git a/conf/appo/task/go1_joystick_flat/mujoco.yaml b/conf/appo/task/go1_joystick_flat/mujoco.yaml index 5124e32cb..8646817a5 100644 --- a/conf/appo/task/go1_joystick_flat/mujoco.yaml +++ b/conf/appo/task/go1_joystick_flat/mujoco.yaml @@ -1,4 +1,7 @@ # @package _global_ +training: + task_name: Go1JoystickFlat + sim_backend: mujoco algo: max_iterations: 150 reward: @@ -13,6 +16,3 @@ reward: contact: 0.24 tracking_sigma: 0.25 base_height_target: 0.3 -training: - task_name: Go1JoystickFlat - sim_backend: mujoco diff --git a/conf/appo/task/go2_joystick_flat/motrix.yaml b/conf/appo/task/go2_joystick_flat/motrix.yaml index b76cdecc2..fc4597bbd 100644 --- a/conf/appo/task/go2_joystick_flat/motrix.yaml +++ b/conf/appo/task/go2_joystick_flat/motrix.yaml @@ -1,8 +1,13 @@ # @package _global_ +training: + task_name: Go2JoystickFlat + sim_backend: motrix algo: - max_iterations: 180 num_envs: 512 steps_per_env: 24 + max_iterations: 180 +env: + sim_dt: 0.015 reward: scales: tracking_lin_vel: 1.0 @@ -17,8 +22,3 @@ reward: swing_feet_z: 4.0 tracking_sigma: 0.25 base_height_target: 0.3 -training: - task_name: Go2JoystickFlat - sim_backend: motrix -env: - sim_dt: 0.015 diff --git a/conf/appo/task/go2_joystick_flat/mujoco.yaml b/conf/appo/task/go2_joystick_flat/mujoco.yaml index ee7ee5340..d25c356ad 100644 --- a/conf/appo/task/go2_joystick_flat/mujoco.yaml +++ b/conf/appo/task/go2_joystick_flat/mujoco.yaml @@ -1,4 +1,7 @@ # @package _global_ +training: + task_name: Go2JoystickFlat + sim_backend: mujoco algo: max_iterations: 150 reward: @@ -15,6 +18,3 @@ reward: swing_feet_z: 4.0 tracking_sigma: 0.25 base_height_target: 0.3 -training: - task_name: Go2JoystickFlat - sim_backend: mujoco diff --git a/conf/offpolicy/task/sac/g1_walk_flat/motrix.yaml b/conf/offpolicy/task/sac/g1_walk_flat/motrix.yaml index e51ae851d..9698d1d77 100644 --- a/conf/offpolicy/task/sac/g1_walk_flat/motrix.yaml +++ b/conf/offpolicy/task/sac/g1_walk_flat/motrix.yaml @@ -1,4 +1,7 @@ # @package _global_ +training: + task_name: G1WalkFlat + sim_backend: motrix algo: num_envs: 2048 learning_starts: 1 @@ -12,8 +15,11 @@ algo: env: control_config: action_scale: 1.0 - gait_phase_init_mode: offset_phase + gait_phase_init_mode: "offset_phase" reset_base_qvel_limit: 0.5 + domain_rand: + randomize_kp: false + randomize_kd: false curriculum: enabled: true initial_scale: 0.5 @@ -22,9 +28,6 @@ env: level_down_threshold: 150.0 level_up_threshold: 750.0 degree: 0.001 - domain_rand: - randomize_kp: false - randomize_kd: false reward: scales: tracking_lin_vel: 2.2 @@ -44,36 +47,4 @@ reward: feet_phase_swing_height: 0.09 feet_phase_tracking_sigma: 0.008 close_feet_threshold: 0.15 - pose_weights: - - 0.01 - - 1.0 - - 5.0 - - 0.01 - - 5.0 - - 5.0 - - 0.01 - - 1.0 - - 5.0 - - 0.01 - - 5.0 - - 5.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 -training: - task_name: G1WalkFlat - sim_backend: motrix + pose_weights: [0.01, 1.0, 5.0, 0.01, 5.0, 5.0, 0.01, 1.0, 5.0, 0.01, 5.0, 5.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0] diff --git a/conf/offpolicy/task/sac/g1_walk_flat/mujoco.yaml b/conf/offpolicy/task/sac/g1_walk_flat/mujoco.yaml index dd47efba0..37abe2d99 100644 --- a/conf/offpolicy/task/sac/g1_walk_flat/mujoco.yaml +++ b/conf/offpolicy/task/sac/g1_walk_flat/mujoco.yaml @@ -1,4 +1,7 @@ # @package _global_ +training: + task_name: G1WalkFlat + sim_backend: mujoco algo: num_envs: 2048 learning_starts: 10 @@ -12,7 +15,7 @@ algo: env: control_config: action_scale: 1.0 - gait_phase_init_mode: offset_phase + gait_phase_init_mode: "offset_phase" reset_base_qvel_limit: 0.5 curriculum: enabled: true @@ -48,36 +51,4 @@ reward: feet_phase_swing_height: 0.09 feet_phase_tracking_sigma: 0.04 close_feet_threshold: 0.15 - pose_weights: - - 0.01 - - 1.0 - - 5.0 - - 0.01 - - 5.0 - - 5.0 - - 0.01 - - 1.0 - - 5.0 - - 0.01 - - 5.0 - - 5.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 -training: - task_name: G1WalkFlat - sim_backend: mujoco + pose_weights: [0.01, 1.0, 5.0, 0.01, 5.0, 5.0, 0.01, 1.0, 5.0, 0.01, 5.0, 5.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0] diff --git a/conf/offpolicy/task/sac/g1_walk_rough/motrix.yaml b/conf/offpolicy/task/sac/g1_walk_rough/motrix.yaml index d2d7731b6..4144b1713 100644 --- a/conf/offpolicy/task/sac/g1_walk_rough/motrix.yaml +++ b/conf/offpolicy/task/sac/g1_walk_rough/motrix.yaml @@ -1,4 +1,7 @@ # @package _global_ +training: + task_name: G1WalkRough + sim_backend: motrix algo: num_envs: 2048 learning_starts: 1 @@ -10,10 +13,14 @@ algo: alpha_init: 0.001 target_entropy_ratio: 0.0 env: + sim_dt: 0.01 control_config: action_scale: 1.0 - gait_phase_init_mode: offset_phase + gait_phase_init_mode: "offset_phase" reset_base_qvel_limit: 0.5 + domain_rand: + randomize_kp: false + randomize_kd: false curriculum: enabled: true initial_scale: 0.5 @@ -22,10 +29,6 @@ env: level_down_threshold: 150.0 level_up_threshold: 750.0 degree: 0.001 - sim_dt: 0.01 - domain_rand: - randomize_kp: false - randomize_kd: false reward: scales: tracking_lin_vel: 2.2 @@ -45,36 +48,4 @@ reward: feet_phase_swing_height: 0.09 feet_phase_tracking_sigma: 0.008 close_feet_threshold: 0.15 - pose_weights: - - 0.01 - - 1.0 - - 5.0 - - 0.01 - - 5.0 - - 5.0 - - 0.01 - - 1.0 - - 5.0 - - 0.01 - - 5.0 - - 5.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 -training: - task_name: G1WalkRough - sim_backend: motrix + pose_weights: [0.01, 1.0, 5.0, 0.01, 5.0, 5.0, 0.01, 1.0, 5.0, 0.01, 5.0, 5.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0] diff --git a/conf/offpolicy/task/sac/g1_walk_rough/mujoco.yaml b/conf/offpolicy/task/sac/g1_walk_rough/mujoco.yaml index daaa8c7fd..bd9a1282a 100644 --- a/conf/offpolicy/task/sac/g1_walk_rough/mujoco.yaml +++ b/conf/offpolicy/task/sac/g1_walk_rough/mujoco.yaml @@ -1,4 +1,7 @@ # @package _global_ +training: + task_name: G1WalkRough + sim_backend: mujoco algo: num_envs: 2048 learning_starts: 10 @@ -12,7 +15,7 @@ algo: env: control_config: action_scale: 1.0 - gait_phase_init_mode: offset_phase + gait_phase_init_mode: "offset_phase" reset_base_qvel_limit: 0.5 curriculum: enabled: true @@ -48,36 +51,4 @@ reward: feet_phase_swing_height: 0.09 feet_phase_tracking_sigma: 0.04 close_feet_threshold: 0.15 - pose_weights: - - 0.01 - - 1.0 - - 5.0 - - 0.01 - - 5.0 - - 5.0 - - 0.01 - - 1.0 - - 5.0 - - 0.01 - - 5.0 - - 5.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 -training: - task_name: G1WalkRough - sim_backend: mujoco + pose_weights: [0.01, 1.0, 5.0, 0.01, 5.0, 5.0, 0.01, 1.0, 5.0, 0.01, 5.0, 5.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0] diff --git a/conf/ppo/task/allegro_inhand/motrix.yaml b/conf/ppo/task/allegro_inhand/motrix.yaml index e7b631431..3608d9743 100644 --- a/conf/ppo/task/allegro_inhand/motrix.yaml +++ b/conf/ppo/task/allegro_inhand/motrix.yaml @@ -1,19 +1,17 @@ # @package _global_ +training: + task_name: AllegroInhandRotation + sim_backend: motrix algo: num_envs: 16384 num_steps_per_env: 8 max_iterations: 201 obs_groups: - actor: - - policy - critic: - - policy + actor: [policy] + critic: [policy] actor: class_name: rsl_rl.models.MLPModel - hidden_dims: - - 512 - - 256 - - 128 + hidden_dims: [512, 256, 128] activation: elu obs_normalization: true distribution_cfg: @@ -22,10 +20,7 @@ algo: std_type: scalar critic: class_name: rsl_rl.models.MLPModel - hidden_dims: - - 512 - - 256 - - 128 + hidden_dims: [512, 256, 128] activation: elu obs_normalization: true algorithm: @@ -46,6 +41,7 @@ env: gen_grasp: false max_episode_seconds: 20.0 grasp_cache_path: caches/allegro_grasp_50k.npy + # Keep only grasp/pose reset variation. All online DR terms stay disabled. domain_rand: randomize_base_mass: false random_com: false @@ -54,9 +50,6 @@ env: joint_noise: 0.0 ball_vel_noise: 0.0 ball_z_offset: 0.0 -training: - task_name: AllegroInhandRotation - sim_backend: motrix play_profile: enabled: true env: diff --git a/conf/ppo/task/allegro_inhand/mujoco.yaml b/conf/ppo/task/allegro_inhand/mujoco.yaml index f770bdd01..17d639913 100644 --- a/conf/ppo/task/allegro_inhand/mujoco.yaml +++ b/conf/ppo/task/allegro_inhand/mujoco.yaml @@ -1,19 +1,21 @@ # @package _global_ +training: + task_name: AllegroInhandRotation + sim_backend: mujoco + render_spacing: 0.5 + cam_distance: 1.5 + cam_lookat: [0.75, 0.75, 0] + cam_elevation: -20.0 algo: num_envs: 16384 num_steps_per_env: 8 max_iterations: 201 obs_groups: - actor: - - policy - critic: - - policy + actor: [policy] + critic: [policy] actor: class_name: rsl_rl.models.MLPModel - hidden_dims: - - 512 - - 256 - - 128 + hidden_dims: [512, 256, 128] activation: elu obs_normalization: true distribution_cfg: @@ -22,10 +24,7 @@ algo: std_type: scalar critic: class_name: rsl_rl.models.MLPModel - hidden_dims: - - 512 - - 256 - - 128 + hidden_dims: [512, 256, 128] activation: elu obs_normalization: true algorithm: @@ -46,6 +45,7 @@ env: gen_grasp: false max_episode_seconds: 20.0 grasp_cache_path: caches/allegro_grasp_50k.npy + # Keep only grasp/pose reset variation. All online DR terms stay disabled. domain_rand: randomize_base_mass: false random_com: false @@ -54,16 +54,6 @@ env: joint_noise: 0.0 ball_vel_noise: 0.0 ball_z_offset: 0.0 -training: - task_name: AllegroInhandRotation - sim_backend: mujoco - render_spacing: 0.5 - cam_distance: 1.5 - cam_lookat: - - 0.75 - - 0.75 - - 0 - cam_elevation: -20.0 play_profile: enabled: true env: diff --git a/conf/ppo/task/g1_box_tracking/motrix.yaml b/conf/ppo/task/g1_box_tracking/motrix.yaml index 302b734a0..863ac1215 100644 --- a/conf/ppo/task/g1_box_tracking/motrix.yaml +++ b/conf/ppo/task/g1_box_tracking/motrix.yaml @@ -1,27 +1,38 @@ # @package _global_ training: - play_steps: 1000 task_name: G1BoxTracking sim_backend: motrix play_env_num: 16 + play_steps: 1000 algo: num_envs: 1024 max_iterations: 40000 save_interval: 500 + empirical_normalization: true obs_groups: actor: - - actor + - actor critic: - - critic + - critic algorithm: entropy_coef: 0.002 desired_kl: 0.01 - empirical_normalization: true noise_config: level: 1.0 scale_joint_angle: 0.01 scale_joint_vel: 1.5 scale_gyro: 0.2 +play_profile: + enabled: true + env: + render_spacing: 2.5 + scene: + enabled: true + source_model_file: src/unilab/assets/robots/g1/scene_flat_with_largebox.xml + ground_texture_file: src/unilab/assets/robots/g1/textures/floor.png + skybox_rgb1: [0.90, 0.90, 0.91] + skybox_rgb2: [0.68, 0.68, 0.70] + ground_texrepeat: [0.25, 0.25] reward: scales: motion_global_root_pos: 1.0 @@ -47,22 +58,3 @@ reward: std_joint_vel: 1.0 std_object_pos: 0.12 std_object_ori: 0.2 -play_profile: - enabled: true - env: - render_spacing: 2.5 - scene: - enabled: true - source_model_file: src/unilab/assets/robots/g1/scene_flat_with_largebox.xml - ground_texture_file: src/unilab/assets/robots/g1/textures/floor.png - skybox_rgb1: - - 0.9 - - 0.9 - - 0.91 - skybox_rgb2: - - 0.68 - - 0.68 - - 0.7 - ground_texrepeat: - - 0.25 - - 0.25 diff --git a/conf/ppo/task/g1_box_tracking/mujoco.yaml b/conf/ppo/task/g1_box_tracking/mujoco.yaml index aade813dd..278a8b8f3 100644 --- a/conf/ppo/task/g1_box_tracking/mujoco.yaml +++ b/conf/ppo/task/g1_box_tracking/mujoco.yaml @@ -1,17 +1,22 @@ # @package _global_ training: - play_steps: 1000 task_name: G1BoxTracking sim_backend: mujoco + play_steps: 1000 algo: num_envs: 1024 max_iterations: 30000 save_interval: 500 obs_groups: actor: - - actor + - actor algorithm: entropy_coef: 0.005 +env: + sim_dt: 0.005 + sensor: + gyro: pelvis_gyro + upvector: pelvis_upvector reward: scales: motion_global_root_pos: 0.5 @@ -37,11 +42,6 @@ reward: std_joint_vel: 1.0 std_object_pos: 0.2 std_object_ori: 0.3 -env: - sim_dt: 0.005 - sensor: - gyro: pelvis_gyro - upvector: pelvis_upvector play_profile: enabled: true env: diff --git a/conf/ppo/task/g1_climb_tracking/motrix.yaml b/conf/ppo/task/g1_climb_tracking/motrix.yaml index 63a07e012..0c30db38e 100644 --- a/conf/ppo/task/g1_climb_tracking/motrix.yaml +++ b/conf/ppo/task/g1_climb_tracking/motrix.yaml @@ -1,8 +1,8 @@ # @package _global_ training: - play_steps: 1000 task_name: G1ClimbTracking sim_backend: motrix + play_steps: 1000 algo: num_envs: 1024 max_iterations: 20000 diff --git a/conf/ppo/task/g1_climb_tracking/mujoco.yaml b/conf/ppo/task/g1_climb_tracking/mujoco.yaml index bddd42ef0..b7111e85a 100644 --- a/conf/ppo/task/g1_climb_tracking/mujoco.yaml +++ b/conf/ppo/task/g1_climb_tracking/mujoco.yaml @@ -1,8 +1,8 @@ # @package _global_ training: - play_steps: 1000 task_name: G1ClimbTracking sim_backend: mujoco + play_steps: 1000 algo: num_envs: 1024 max_iterations: 20000 @@ -10,9 +10,9 @@ algo: empirical_normalization: true obs_groups: actor: - - actor + - actor critic: - - critic + - critic algorithm: entropy_coef: 0.005 desired_kl: 0.01 @@ -22,35 +22,35 @@ env: sim_dt: 0.005 control_config: action_scale: - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 anchor_pos_z_threshold: 0.3 ee_body_pos_z_threshold: 0.3 terminate_on_undesired_contacts: true diff --git a/conf/ppo/task/g1_flip_tracking/motrix.yaml b/conf/ppo/task/g1_flip_tracking/motrix.yaml index 1978da4ef..3435bd57a 100644 --- a/conf/ppo/task/g1_flip_tracking/motrix.yaml +++ b/conf/ppo/task/g1_flip_tracking/motrix.yaml @@ -1,15 +1,15 @@ # @package _global_ training: - play_steps: 1000 task_name: G1FlipTracking sim_backend: motrix + play_steps: 1000 algo: num_envs: 1024 max_iterations: 30000 save_interval: 500 obs_groups: actor: - - actor + - actor algorithm: entropy_coef: 0.005 reward: diff --git a/conf/ppo/task/g1_flip_tracking/mujoco.yaml b/conf/ppo/task/g1_flip_tracking/mujoco.yaml index bb4d5188f..1edd403da 100644 --- a/conf/ppo/task/g1_flip_tracking/mujoco.yaml +++ b/conf/ppo/task/g1_flip_tracking/mujoco.yaml @@ -1,21 +1,61 @@ # @package _global_ training: - play_steps: 1000 task_name: G1FlipTracking sim_backend: mujoco + play_steps: 1000 algo: num_envs: 1024 max_iterations: 20000 save_interval: 500 + empirical_normalization: true obs_groups: actor: - - actor + - actor critic: - - critic + - critic algorithm: entropy_coef: 0.005 desired_kl: 0.01 - empirical_normalization: true +env: + sampling_mode: start + truncate_on_clip_end: false + sim_dt: 0.005 + control_config: + action_scale: + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + anchor_pos_z_threshold: 0.5 + ee_body_pos_z_threshold: 0.5 + terminate_on_undesired_contacts: true + noise_config: + level: 0.0 reward: scales: motion_global_root_pos: 0.5 @@ -24,11 +64,11 @@ reward: motion_body_ori: 1.5 motion_body_lin_vel: 1.0 motion_body_ang_vel: 1.0 + motion_ee_body_pos_z: 2.0 motion_joint_pos: 0.0 motion_joint_vel: 0.0 action_rate_l2: -0.005 joint_limit: -10.0 - motion_ee_body_pos_z: 2.0 undesired_contacts: -0.1 std_root_pos: 0.3 std_root_ori: 0.4 @@ -38,46 +78,6 @@ reward: std_body_ang_vel: 3.14 std_joint_pos: 0.2 std_joint_vel: 1.0 -env: - sampling_mode: start - truncate_on_clip_end: false - sim_dt: 0.005 - control_config: - action_scale: - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - anchor_pos_z_threshold: 0.5 - ee_body_pos_z_threshold: 0.5 - terminate_on_undesired_contacts: true - noise_config: - level: 0.0 play_profile: enabled: true env: diff --git a/conf/ppo/task/g1_motion_tracking/motrix.yaml b/conf/ppo/task/g1_motion_tracking/motrix.yaml index 88deb4eef..643939c54 100644 --- a/conf/ppo/task/g1_motion_tracking/motrix.yaml +++ b/conf/ppo/task/g1_motion_tracking/motrix.yaml @@ -1,16 +1,16 @@ # @package _global_ training: - play_steps: 1000 task_name: G1MotionTracking sim_backend: motrix play_env_num: 16 + play_steps: 1000 algo: num_envs: 1024 max_iterations: 15000 save_interval: 500 obs_groups: actor: - - actor + - actor algorithm: entropy_coef: 0.005 noise_config: @@ -18,6 +18,17 @@ algo: scale_joint_angle: 0.01 scale_joint_vel: 1.5 scale_gyro: 0.2 +play_profile: + enabled: true + env: + render_spacing: 2.5 + scene: + enabled: true + source_model_file: src/unilab/assets/robots/g1/scene_flat.xml + ground_texture_file: src/unilab/assets/robots/g1/textures/floor.png + skybox_rgb1: [0.90, 0.90, 0.91] + skybox_rgb2: [0.68, 0.68, 0.70] + ground_texrepeat: [0.25, 0.25] reward: scales: motion_global_root_pos: 1.0 @@ -39,22 +50,3 @@ reward: std_body_ang_vel: 3.14 std_joint_pos: 0.2 std_joint_vel: 1.0 -play_profile: - enabled: true - env: - render_spacing: 2.5 - scene: - enabled: true - source_model_file: src/unilab/assets/robots/g1/scene_flat.xml - ground_texture_file: src/unilab/assets/robots/g1/textures/floor.png - skybox_rgb1: - - 0.9 - - 0.9 - - 0.91 - skybox_rgb2: - - 0.68 - - 0.68 - - 0.7 - ground_texrepeat: - - 0.25 - - 0.25 diff --git a/conf/ppo/task/g1_motion_tracking/mujoco.yaml b/conf/ppo/task/g1_motion_tracking/mujoco.yaml index b58bc3780..7301e4b68 100644 --- a/conf/ppo/task/g1_motion_tracking/mujoco.yaml +++ b/conf/ppo/task/g1_motion_tracking/mujoco.yaml @@ -1,15 +1,15 @@ # @package _global_ training: - play_steps: 1000 task_name: G1MotionTracking sim_backend: mujoco + play_steps: 1000 algo: num_envs: 1024 max_iterations: 15000 save_interval: 500 obs_groups: actor: - - actor + - actor algorithm: entropy_coef: 0.005 reward: diff --git a/conf/ppo/task/g1_walk_flat/motrix.yaml b/conf/ppo/task/g1_walk_flat/motrix.yaml index fca65c0fe..88c971936 100644 --- a/conf/ppo/task/g1_walk_flat/motrix.yaml +++ b/conf/ppo/task/g1_walk_flat/motrix.yaml @@ -1,40 +1,36 @@ # @package _global_ +# Standalone Motrix owner config: carries the shared contract inline, then +# overrides contract fields for Motrix-specific tuning +# (intentionally non-transferable from MuJoCo; drop overrides to restore parity). +training: + task_name: G1WalkFlat + sim_backend: motrix algo: + num_envs: 2048 + max_iterations: 2200 empirical_normalization: true obs_groups: actor: - - policy + - policy critic: - - critic + - critic policy: - actor_hidden_dims: - - 512 - - 256 - - 128 - critic_hidden_dims: - - 512 - - 256 - - 128 + actor_hidden_dims: [512, 256, 128] + critic_hidden_dims: [512, 256, 128] init_noise_std: 0.5 - num_envs: 2048 - max_iterations: 2200 algorithm: - learning_rate: 0.0003 - entropy_coef: 0.005 + learning_rate: 3.0e-4 + entropy_coef: 5.0e-3 env: - control_config: - action_scale: 0.5 domain_rand: randomize_kp: false randomize_kd: false + control_config: + action_scale: 0.5 commands: vel_limit: - - - 0.4 - - 0.0 - - 0.0 - - - 0.7 - - 0.0 - - 0.0 + - [0.4, 0.0, 0.0] + - [0.7, 0.0, 0.0] gait_phase_init_mode: offset_phase reset_base_qvel_limit: 0.05 curriculum: @@ -44,9 +40,6 @@ env: scale_joint_angle: 0.01 scale_joint_vel: 1.5 scale_gyro: 0.2 -training: - task_name: G1WalkFlat - sim_backend: motrix reward: scales: tracking_lin_vel: 2.0 diff --git a/conf/ppo/task/g1_walk_flat/mujoco.yaml b/conf/ppo/task/g1_walk_flat/mujoco.yaml index 7d3a3f1c7..843d7b85f 100644 --- a/conf/ppo/task/g1_walk_flat/mujoco.yaml +++ b/conf/ppo/task/g1_walk_flat/mujoco.yaml @@ -1,20 +1,19 @@ # @package _global_ +# Standalone MuJoCo owner config: carries the shared cross-backend contract inline +# (formerly base.yaml), plus backend-specific tuning. +training: + task_name: G1WalkFlat + sim_backend: mujoco algo: + num_envs: 2048 + max_iterations: 2200 empirical_normalization: false obs_groups: actor: - - actor + - actor policy: - actor_hidden_dims: - - 512 - - 256 - - 128 - critic_hidden_dims: - - 512 - - 256 - - 128 - num_envs: 2048 - max_iterations: 2200 + actor_hidden_dims: [512, 256, 128] + critic_hidden_dims: [512, 256, 128] env: control_config: action_scale: 0.25 @@ -25,9 +24,6 @@ env: scale_joint_angle: 0.01 scale_joint_vel: 1.5 scale_gyro: 0.2 -training: - task_name: G1WalkFlat - sim_backend: mujoco reward: scales: tracking_lin_vel: 2.0 @@ -46,36 +42,7 @@ reward: base_height_target: 0.754 min_base_height: 0.55 max_tilt_deg: 25.0 - pose_weights: - - 0.01 - - 1.0 - - 5.0 - - 0.01 - - 5.0 - - 5.0 - - 0.01 - - 1.0 - - 5.0 - - 0.01 - - 5.0 - - 5.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 - - 50.0 + pose_weights: [0.01, 1.0, 5.0, 0.01, 5.0, 5.0, 0.01, 1.0, 5.0, 0.01, 5.0, 5.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0, 50.0] play_profile: enabled: true env: diff --git a/conf/ppo/task/g1_wall_flip_tracking/motrix.yaml b/conf/ppo/task/g1_wall_flip_tracking/motrix.yaml index 223196042..c595203a6 100644 --- a/conf/ppo/task/g1_wall_flip_tracking/motrix.yaml +++ b/conf/ppo/task/g1_wall_flip_tracking/motrix.yaml @@ -1,9 +1,9 @@ # @package _global_ training: - play_steps: 1000 task_name: G1WallFlipTracking sim_backend: motrix play_env_num: 16 + play_steps: 1000 render_spacing: 3.0 algo: num_envs: 1024 @@ -12,53 +12,64 @@ algo: empirical_normalization: true obs_groups: actor: - - actor + - actor critic: - - critic + - critic algorithm: entropy_coef: 0.005 desired_kl: 0.01 env: + motrix_max_iterations: 3 sampling_mode: start truncate_on_clip_end: false sim_dt: 0.005 control_config: action_scale: - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 anchor_pos_z_threshold: 0.5 ee_body_pos_z_threshold: 0.5 terminate_on_undesired_contacts: true noise_config: level: 0.0 - motrix_max_iterations: 3 +play_profile: + enabled: true + env: + render_spacing: 4.0 + scene: + enabled: true + source_model_file: src/unilab/assets/robots/g1/scene_flat_with_wall.xml + ground_texture_file: src/unilab/assets/robots/g1/textures/floor.png + skybox_rgb1: [0.90, 0.90, 0.91] + skybox_rgb2: [0.68, 0.68, 0.70] + ground_texrepeat: [0.25, 0.25] reward: scales: motion_global_root_pos: 0.5 @@ -81,22 +92,3 @@ reward: std_body_ang_vel: 3.14 std_joint_pos: 0.2 std_joint_vel: 1.0 -play_profile: - enabled: true - env: - render_spacing: 4.0 - scene: - enabled: true - source_model_file: src/unilab/assets/robots/g1/scene_flat_with_wall.xml - ground_texture_file: src/unilab/assets/robots/g1/textures/floor.png - skybox_rgb1: - - 0.9 - - 0.9 - - 0.91 - skybox_rgb2: - - 0.68 - - 0.68 - - 0.7 - ground_texrepeat: - - 0.25 - - 0.25 diff --git a/conf/ppo/task/g1_wall_flip_tracking/mujoco.yaml b/conf/ppo/task/g1_wall_flip_tracking/mujoco.yaml index 6753ce2d7..e11d9b05b 100644 --- a/conf/ppo/task/g1_wall_flip_tracking/mujoco.yaml +++ b/conf/ppo/task/g1_wall_flip_tracking/mujoco.yaml @@ -1,8 +1,8 @@ # @package _global_ training: - play_steps: 1000 task_name: G1WallFlipTracking sim_backend: mujoco + play_steps: 1000 algo: num_envs: 1024 max_iterations: 20000 @@ -10,9 +10,9 @@ algo: empirical_normalization: true obs_groups: actor: - - actor + - actor critic: - - critic + - critic algorithm: entropy_coef: 0.005 desired_kl: 0.01 @@ -22,35 +22,35 @@ env: sim_dt: 0.005 control_config: action_scale: - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.5475464629911068 - - 0.35066146637882434 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.5475464629911068 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.43857731392336724 - - 0.07450087032950714 - - 0.07450087032950714 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.5475464629911068 + - 0.35066146637882434 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.5475464629911068 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.43857731392336724 + - 0.07450087032950714 + - 0.07450087032950714 anchor_pos_z_threshold: 0.5 ee_body_pos_z_threshold: 0.5 terminate_on_undesired_contacts: true diff --git a/conf/ppo/task/go1_joystick_flat/motrix.yaml b/conf/ppo/task/go1_joystick_flat/motrix.yaml index fc2aa162a..d84ba1b03 100644 --- a/conf/ppo/task/go1_joystick_flat/motrix.yaml +++ b/conf/ppo/task/go1_joystick_flat/motrix.yaml @@ -1,29 +1,36 @@ # @package _global_ training: + task_name: Go1JoystickFlat + sim_backend: motrix play_steps: 500 play_env_num: 16 cam_tracking: true cam_tracking_env_idx: 0 cam_tracking_extra_envs: 9 - task_name: Go1JoystickFlat - sim_backend: motrix + interactive: action_mode: policy policy_obs_mode: auto camera_follow_body: true use_env_visual_model: false + algo: num_envs: 1024 max_iterations: 151 obs_groups: actor: - - actor + - actor empirical_normalization: true policy: init_noise_std: 0.5 algorithm: - learning_rate: 0.0003 - entropy_coef: 0.001 + learning_rate: 3.0e-4 + entropy_coef: 1.0e-3 +env: + commands: + vel_limit: + - [0.5, 0.0, 0.0] + - [0.5, 0.0, 0.0] reward: scales: tracking_lin_vel: 1.0 @@ -36,15 +43,6 @@ reward: swing_feet_z: 4.0 tracking_sigma: 0.25 base_height_target: 0.3 -env: - commands: - vel_limit: - - - 0.5 - - 0.0 - - 0.0 - - - 0.5 - - 0.0 - - 0.0 play_profile: enabled: true env: diff --git a/conf/ppo/task/go1_joystick_flat/mujoco.yaml b/conf/ppo/task/go1_joystick_flat/mujoco.yaml index eb5799b20..794266ad3 100644 --- a/conf/ppo/task/go1_joystick_flat/mujoco.yaml +++ b/conf/ppo/task/go1_joystick_flat/mujoco.yaml @@ -1,24 +1,26 @@ # @package _global_ training: + task_name: Go1JoystickFlat + sim_backend: mujoco play_steps: 500 play_env_num: 16 + render_spacing: 0.0 cam_tracking: true cam_tracking_env_idx: 0 cam_tracking_extra_envs: 9 - task_name: Go1JoystickFlat - sim_backend: mujoco - render_spacing: 0.0 + interactive: action_mode: policy policy_obs_mode: auto camera_follow_body: true use_env_visual_model: false + algo: num_envs: 1024 max_iterations: 151 obs_groups: actor: - - actor + - actor reward: scales: tracking_lin_vel: 1.0 @@ -28,8 +30,8 @@ reward: base_height: -100.0 action_rate: -0.005 similar_to_default: -0.1 - swing_feet_z: 4.0 contact: 0.24 + swing_feet_z: 4.0 tracking_sigma: 0.25 base_height_target: 0.3 play_profile: diff --git a/conf/ppo/task/go1_joystick_rough/motrix.yaml b/conf/ppo/task/go1_joystick_rough/motrix.yaml index 4d9cfdfe2..21be8340a 100644 --- a/conf/ppo/task/go1_joystick_rough/motrix.yaml +++ b/conf/ppo/task/go1_joystick_rough/motrix.yaml @@ -1,17 +1,19 @@ # @package _global_ training: + task_name: Go1JoystickRough + sim_backend: motrix play_steps: 500 play_env_num: 16 cam_tracking: true cam_tracking_env_idx: 0 cam_tracking_extra_envs: 9 - task_name: Go1JoystickRough - sim_backend: motrix + interactive: action_mode: policy policy_obs_mode: auto camera_follow_body: true use_env_visual_model: false + algo: num_envs: 2048 num_steps_per_env: 24 @@ -19,49 +21,41 @@ algo: empirical_normalization: false obs_groups: actor: - - actor + - actor critic: - - critic + - critic policy: init_noise_std: 1.0 algorithm: - learning_rate: 0.001 - entropy_coef: 0.01 + learning_rate: 1.0e-3 + entropy_coef: 1.0e-2 + env: + render_offset_mode: zero control_config: action_scale: 0.25 hip_action_scale: 0.125 non_hip_action_scale: 0.25 clip_actions: 100.0 commands: - vel_limit: - - - -1.0 - - -1.0 - - -1.0 - - - 1.0 - - 1.0 - - 1.0 + vel_limit: [[-1.0, -1.0, -1.0], [1.0, 1.0, 1.0]] resampling_time: 10.0 heading_command: true - heading_range: - - -3.141592653589793 - - 3.141592653589793 + heading_range: [-3.141592653589793, 3.141592653589793] rel_standing_envs: 0.1 terrain_curriculum: enabled: false scene: model_file: src/unilab/assets/robots/go1/go1.xml fragment_files: - - src/unilab/assets/robots/go1/locomotion_task.xml + - src/unilab/assets/robots/go1/locomotion_task.xml terrain: hfield_name: terrain_hfield geom_name: floor generator: seed: 42 curriculum: false - size: - - 8.0 - - 8.0 + size: [8.0, 8.0] num_rows: 6 num_cols: 6 border_width: 20.0 @@ -73,39 +67,30 @@ env: terrain_distance_buffer: 3.0 domain_rand: randomize_base_mass: true - added_mass_range: - - -1.0 - - 3.0 + added_mass_range: [-1.0, 3.0] random_com: true randomize_kp: true - kp_multiplier_range: - - 0.5 - - 2.0 + kp_multiplier_range: [0.5, 2.0] randomize_kd: true - kd_multiplier_range: - - 0.5 - - 2.0 + kd_multiplier_range: [0.5, 2.0] push_robots: true push_interval: 625 - max_force: - - 1.0 - - 1.0 - - 0.5 - render_offset_mode: zero + max_force: [1.0, 1.0, 0.5] + reward: scales: lin_vel_z: -2.0 ang_vel_xy: -0.05 - joint_torques_l2: -2.5e-05 - joint_acc_l2: -2.5e-07 - joint_power: -2.0e-05 + joint_torques_l2: -2.5e-5 + joint_acc_l2: -2.5e-7 + joint_power: -2.0e-5 stand_still: -2.0 hip_pos: -0.5 joint_pos_penalty: -1.0 joint_mirror: -0.05 action_rate: -0.01 undesired_contacts: -1.0 - contact_forces: -0.00015 + contact_forces: -1.5e-4 tracking_lin_vel: 3.0 tracking_ang_vel: 1.5 feet_air_time: 0.5 diff --git a/conf/ppo/task/go1_joystick_rough/mujoco.yaml b/conf/ppo/task/go1_joystick_rough/mujoco.yaml index 43b84ad57..04a9dc179 100644 --- a/conf/ppo/task/go1_joystick_rough/mujoco.yaml +++ b/conf/ppo/task/go1_joystick_rough/mujoco.yaml @@ -1,18 +1,20 @@ # @package _global_ training: + task_name: Go1JoystickRough + sim_backend: mujoco play_steps: 500 play_env_num: 16 + render_spacing: 0.0 cam_tracking: true cam_tracking_env_idx: 0 cam_tracking_extra_envs: 9 - task_name: Go1JoystickRough - sim_backend: mujoco - render_spacing: 0.0 + interactive: action_mode: policy policy_obs_mode: auto camera_follow_body: true use_env_visual_model: false + algo: num_envs: 2048 num_steps_per_env: 24 @@ -20,49 +22,41 @@ algo: empirical_normalization: false obs_groups: actor: - - actor + - actor critic: - - critic + - critic policy: init_noise_std: 1.0 algorithm: - learning_rate: 0.001 - entropy_coef: 0.01 + learning_rate: 1.0e-3 + entropy_coef: 1.0e-2 + env: + sim_dt: 0.005 control_config: action_scale: 0.25 hip_action_scale: 0.125 non_hip_action_scale: 0.25 clip_actions: 100.0 commands: - vel_limit: - - - -1.0 - - -1.0 - - -1.0 - - - 1.0 - - 1.0 - - 1.0 + vel_limit: [[-1.0, -1.0, -1.0], [1.0, 1.0, 1.0]] resampling_time: 10.0 heading_command: true - heading_range: - - -3.141592653589793 - - 3.141592653589793 + heading_range: [-3.141592653589793, 3.141592653589793] rel_standing_envs: 0.1 terrain_curriculum: enabled: false scene: model_file: src/unilab/assets/robots/go1/go1_mujoco.xml fragment_files: - - src/unilab/assets/robots/go1/locomotion_task.xml + - src/unilab/assets/robots/go1/locomotion_task.xml terrain: hfield_name: terrain_hfield geom_name: floor generator: seed: 42 curriculum: false - size: - - 8.0 - - 8.0 + size: [8.0, 8.0] num_rows: 6 num_cols: 6 border_width: 20.0 @@ -74,39 +68,30 @@ env: terrain_distance_buffer: 3.0 domain_rand: randomize_base_mass: true - added_mass_range: - - -1.0 - - 3.0 + added_mass_range: [-1.0, 3.0] random_com: true randomize_kp: true - kp_multiplier_range: - - 0.5 - - 2.0 + kp_multiplier_range: [0.5, 2.0] randomize_kd: true - kd_multiplier_range: - - 0.5 - - 2.0 + kd_multiplier_range: [0.5, 2.0] push_robots: true push_interval: 625 - max_force: - - 1.0 - - 1.0 - - 0.5 - sim_dt: 0.005 + max_force: [1.0, 1.0, 0.5] + reward: scales: lin_vel_z: -2.0 ang_vel_xy: -0.05 - joint_torques_l2: -2.5e-05 - joint_acc_l2: -2.5e-07 - joint_power: -2.0e-05 + joint_torques_l2: -2.5e-5 + joint_acc_l2: -2.5e-7 + joint_power: -2.0e-5 stand_still: -2.0 hip_pos: -0.5 joint_pos_penalty: -1.0 joint_mirror: -0.05 action_rate: -0.01 undesired_contacts: -1.0 - contact_forces: -0.00015 + contact_forces: -1.5e-4 tracking_lin_vel: 3.0 tracking_ang_vel: 1.5 feet_air_time: 0.5 diff --git a/conf/ppo/task/go2_arm_manip_loco/motrix.yaml b/conf/ppo/task/go2_arm_manip_loco/motrix.yaml index ec3b00ca5..1b7047bbd 100644 --- a/conf/ppo/task/go2_arm_manip_loco/motrix.yaml +++ b/conf/ppo/task/go2_arm_manip_loco/motrix.yaml @@ -1,86 +1,80 @@ # @package _global_ +training: + task_name: Go2ArmManipLoco + sim_backend: motrix algo: num_envs: 4096 max_iterations: 3000 empirical_normalization: true obs_groups: actor: - - actor + - actor policy: init_noise_std: 0.5 algorithm: - learning_rate: 0.0003 - entropy_coef: 0.001 + learning_rate: 3.0e-4 + entropy_coef: 1.0e-3 num_mini_batches: 4 +play_profile: + enabled: true + scene: + enabled: true + source_model_file: src/unilab/assets/robots/go2_arm/scene_flat.xml + ground_texture_file: src/unilab/assets/robots/g1/textures/floor.png + skybox_rgb1: [0.90, 0.90, 0.91] + skybox_rgb2: [0.68, 0.68, 0.70] + ground_texrepeat: [0.25, 0.25] reward: scales: - tracking_lin_vel: 2.0 - tracking_ang_vel: 0.5 + # locomotion: tracking + tracking_lin_vel: 2.0 #1.0 + tracking_ang_vel: 0.5 #0.2 + # locomotion: velocity / orientation penalties lin_vel_z: -5.0 ang_vel_xy: -0.1 - roll: -5.0 - base_height: -100 - similar_to_default: -0.0 - leg_pose: -0.1 - dof_pos_limits: 0.0 + roll: -5.0 #-2.0 + # locomotion: height / pose + base_height: -100 #-20.0 # Go2+arm body oscillates more with arm mass; -100 conflicts with gait. + similar_to_default: -0.0 # Align with Go2 Joystick (L1). + leg_pose: -0.1 # Kept but unused; replaced by similar_to_default. + dof_pos_limits: 0.0 # Kept but unused; soft limits are not configured. + # locomotion: effort penalties action_rate: -0.005 - torques: 0.0 - energy: 0.0 - dof_vel: 0.0 - dof_acc: 0.0 + torques: 0.0 # Kept but unused; backend does not expose torques. + energy: 0.0 # Kept but unused. + dof_vel: 0.0 # Kept but unused. + dof_acc: 0.0 # Kept but unused. + # locomotion: gait, aligned with Go2 Joystick stand_still: -0.5 contact: 0.24 swing_feet_z: 4.0 foot_drag: -0.1 + # manipulation rewards object_distance: 2.0 object_distance_l2: -0.5 + # arm collision penalty arm_collision: -1.0 tracking_sigma: 0.25 base_height_target: 0.3 object_sigma: 0.1 env: goal_ee: - sphere_l_range: - - 0.3 - - 0.6 - sphere_phi_range: - - -1.2566 - - 1.0472 - sphere_theta_range: - - -2.3562 - - 2.3562 - traj_time_range: - - 1.0 - - 3.0 - hold_time_range: - - 0.5 - - 2.0 - collision_upper_limits: - - 0.3 - - 0.15 - - -0.115 - collision_lower_limits: - - -0.2 - - -0.15 - - -0.515 + sphere_l_range: [0.3, 0.6] + sphere_phi_range: [-1.2566, 1.0472] + sphere_theta_range: [-2.3562, 2.3562] + traj_time_range: [1.0, 3.0] + hold_time_range: [0.5, 2.0] + collision_upper_limits: [0.3, 0.15, -0.115] + collision_lower_limits: [-0.2, -0.15, -0.515] underground_limit: -0.57 num_collision_check_samples: 10 num_resample_attempts: 10 default_orn_roll: 0.0 arm_induced_pitch: 0.78 - delta_orn_r: - - -0.5 - - 0.5 - delta_orn_p: - - -0.5 - - 0.5 - delta_orn_y: - - -0.5 - - 0.5 - init_ee_cart: - - 0.25 - - 0.0 - - 0.25 + delta_orn_r: [-0.5, 0.5] + delta_orn_p: [-0.5, 0.5] + delta_orn_y: [-0.5, 0.5] + init_ee_cart: [0.25, 0.0, 0.25] control_config: arm_action_scale: 0.0 ik: @@ -88,80 +82,36 @@ env: gain: 1.0 dq_clip: 0.2 use_orientation: true - orientation_mode: target + orientation_mode: target # target: track sampled orientation; zero_error: constrain orientation change through Jrot without tracking orientation. commands: vel_limit: - - - -0.6 - - -0.4 - - -0.8 - - - 1.0 - - 0.4 - - 0.8 - zero_command_prob: 0.15 - resample_time_s: 4.0 + - [-0.6, -0.4, -0.8] # Align with the default Go2 Joystick range. + - [1.0, 0.4, 0.8] + zero_command_prob: 0.15 # Explicitly sample command=0 for stable standing. + resample_time_s: 4.0 # Omit for null, which disables mid-episode resampling and matches Joystick. curriculum: - enable: false + enable: false # Align with Joystick; no curriculum. domain_rand: randomize_base_mass: false - added_mass_range: - - -1.0 - - 1.0 + added_mass_range: [-1.0, 1.0] randomize_body_mass: true - body_mass_multiplier_range: - - 0.9 - - 1.1 + body_mass_multiplier_range: [0.9, 1.1] random_com: true - com_offset_x: - - -0.03 - - 0.03 + com_offset_x: [-0.03, 0.03] randomize_gravity: false randomize_ground_friction: true - ground_friction_multiplier_range: - - 0.8 - - 1.2 + ground_friction_multiplier_range: [0.8, 1.2] randomize_dof_armature: false - dof_armature_multiplier_range: - - 0.8 - - 1.2 + dof_armature_multiplier_range: [0.8, 1.2] push_robots: true push_interval: 500 - max_force: - - 1.2 - - 1.2 - - 0.6 + max_force: [1.2, 1.2, 0.6] push_body_name: base randomize_kp: false - kp_multiplier_range: - - 0.9 - - 1.1 + kp_multiplier_range: [0.9, 1.1] randomize_kd: false - kd_multiplier_range: - - 0.9 - - 1.1 + kd_multiplier_range: [0.9, 1.1] arm_stage: freeze_arm_joints: false disable_ee_goal_trajectory: false - fixed_ee_goal_cart: - - 0.15 - - 0.0 - - 0.25 -training: - task_name: Go2ArmManipLoco - sim_backend: motrix -play_profile: - enabled: true - scene: - enabled: true - source_model_file: src/unilab/assets/robots/go2_arm/scene_flat.xml - ground_texture_file: src/unilab/assets/robots/g1/textures/floor.png - skybox_rgb1: - - 0.9 - - 0.9 - - 0.91 - skybox_rgb2: - - 0.68 - - 0.68 - - 0.7 - ground_texrepeat: - - 0.25 - - 0.25 + fixed_ee_goal_cart: [0.15, 0.0, 0.25] diff --git a/conf/ppo/task/go2_arm_manip_loco/mujoco.yaml b/conf/ppo/task/go2_arm_manip_loco/mujoco.yaml index 83948f12d..162e0a256 100644 --- a/conf/ppo/task/go2_arm_manip_loco/mujoco.yaml +++ b/conf/ppo/task/go2_arm_manip_loco/mujoco.yaml @@ -1,86 +1,71 @@ # @package _global_ +training: + task_name: Go2ArmManipLoco + sim_backend: mujoco algo: num_envs: 4096 max_iterations: 151 empirical_normalization: true obs_groups: actor: - - actor + - actor policy: init_noise_std: 0.5 algorithm: - learning_rate: 0.0003 - entropy_coef: 0.001 + learning_rate: 3.0e-4 + entropy_coef: 1.0e-3 num_mini_batches: 4 reward: scales: - tracking_lin_vel: 2.0 - tracking_ang_vel: 0.5 + # locomotion: tracking + tracking_lin_vel: 2.0 #1.0 + tracking_ang_vel: 0.5 #0.2 + # locomotion: velocity / orientation penalties lin_vel_z: -5.0 ang_vel_xy: -0.1 - roll: -5.0 - base_height: -100 - similar_to_default: -0.0 - leg_pose: -0.1 - dof_pos_limits: 0.0 + roll: -5.0 #-2.0 + # locomotion: height / pose + base_height: -100 #-20.0 # Go2+arm body oscillates more with arm mass; -100 conflicts with gait. + similar_to_default: -0.0 # Align with Go2 Joystick (L1). + leg_pose: -0.1 # Kept but unused; replaced by similar_to_default. + dof_pos_limits: 0.0 # Kept but unused; soft limits are not configured. + # locomotion: effort penalties action_rate: -0.005 - torques: 0.0 - energy: 0.0 - dof_vel: 0.0 - dof_acc: 0.0 + torques: 0.0 # Kept but unused; backend does not expose torques. + energy: 0.0 # Kept but unused. + dof_vel: 0.0 # Kept but unused. + dof_acc: 0.0 # Kept but unused. + # locomotion: gait, aligned with Go2 Joystick stand_still: -0.5 contact: 0.24 swing_feet_z: 4.0 foot_drag: -0.1 + # manipulation rewards object_distance: 2.0 object_distance_l2: -0.5 + # arm collision penalty arm_collision: -1.0 tracking_sigma: 0.25 base_height_target: 0.3 object_sigma: 0.1 env: goal_ee: - sphere_l_range: - - 0.3 - - 0.6 - sphere_phi_range: - - -1.2566 - - 1.0472 - sphere_theta_range: - - -2.3562 - - 2.3562 - traj_time_range: - - 1.0 - - 3.0 - hold_time_range: - - 0.5 - - 2.0 - collision_upper_limits: - - 0.3 - - 0.15 - - -0.115 - collision_lower_limits: - - -0.2 - - -0.15 - - -0.515 + sphere_l_range: [0.3, 0.6] + sphere_phi_range: [-1.2566, 1.0472] + sphere_theta_range: [-2.3562, 2.3562] + traj_time_range: [1.0, 3.0] + hold_time_range: [0.5, 2.0] + collision_upper_limits: [0.3, 0.15, -0.115] + collision_lower_limits: [-0.2, -0.15, -0.515] underground_limit: -0.57 num_collision_check_samples: 10 num_resample_attempts: 10 default_orn_roll: 0.0 arm_induced_pitch: 0.78 - delta_orn_r: - - -0.5 - - 0.5 - delta_orn_p: - - -0.5 - - 0.5 - delta_orn_y: - - -0.5 - - 0.5 - init_ee_cart: - - 0.25 - - 0.0 - - 0.25 + delta_orn_r: [-0.5, 0.5] + delta_orn_p: [-0.5, 0.5] + delta_orn_y: [-0.5, 0.5] + init_ee_cart: [0.25, 0.0, 0.25] control_config: arm_action_scale: 0.0 ik: @@ -88,66 +73,39 @@ env: gain: 1.0 dq_clip: 0.2 use_orientation: true - orientation_mode: target + orientation_mode: target # target: track sampled orientation; zero_error: constrain orientation change through Jrot without tracking orientation. commands: vel_limit: - - - -0.6 - - -0.4 - - -0.8 - - - 1.0 - - 0.4 - - 0.8 - zero_command_prob: 0.15 - resample_time_s: 4.0 + - [-0.6, -0.4, -0.8] # Align with the default Go2 Joystick range. + - [1.0, 0.4, 0.8] + zero_command_prob: 0.15 # Explicitly sample command=0 for stable standing. + resample_time_s: 4.0 # Omit for null, which disables mid-episode resampling and matches Joystick. curriculum: - enable: false + enable: false # Align with Joystick; no curriculum. domain_rand: randomize_base_mass: false - added_mass_range: - - -1.0 - - 1.0 + added_mass_range: [-1.0, 1.0] randomize_body_mass: true - body_mass_multiplier_range: - - 0.9 - - 1.1 + body_mass_multiplier_range: [0.9, 1.1] random_com: true - com_offset_x: - - -0.03 - - 0.03 + com_offset_x: [-0.03, 0.03] randomize_gravity: false randomize_ground_friction: true - ground_friction_multiplier_range: - - 0.8 - - 1.2 + ground_friction_multiplier_range: [0.8, 1.2] randomize_dof_armature: true - dof_armature_multiplier_range: - - 0.8 - - 1.2 + dof_armature_multiplier_range: [0.8, 1.2] push_robots: true push_interval: 500 - max_force: - - 1.2 - - 1.2 - - 0.6 + max_force: [1.2, 1.2, 0.6] push_body_name: base randomize_kp: true - kp_multiplier_range: - - 0.9 - - 1.1 + kp_multiplier_range: [0.9, 1.1] randomize_kd: true - kd_multiplier_range: - - 0.9 - - 1.1 + kd_multiplier_range: [0.9, 1.1] arm_stage: freeze_arm_joints: false disable_ee_goal_trajectory: false - fixed_ee_goal_cart: - - 0.15 - - 0.0 - - 0.25 -training: - task_name: Go2ArmManipLoco - sim_backend: mujoco + fixed_ee_goal_cart: [0.15, 0.0, 0.25] play_profile: enabled: true env: diff --git a/conf/ppo/task/go2_joystick_flat/motrix.yaml b/conf/ppo/task/go2_joystick_flat/motrix.yaml index 920428170..aca13216d 100644 --- a/conf/ppo/task/go2_joystick_flat/motrix.yaml +++ b/conf/ppo/task/go2_joystick_flat/motrix.yaml @@ -1,16 +1,27 @@ # @package _global_ +training: + task_name: Go2JoystickFlat + sim_backend: motrix algo: num_envs: 1024 max_iterations: 151 empirical_normalization: true obs_groups: actor: - - actor + - actor policy: init_noise_std: 0.5 algorithm: - learning_rate: 0.0003 - entropy_coef: 0.001 + learning_rate: 3.0e-4 + entropy_coef: 1.0e-3 +env: + commands: + vel_limit: + - [0.5, 0.0, 0.0] + - [0.5, 0.0, 0.0] + domain_rand: + randomize_kp: false + randomize_kd: false reward: scales: tracking_lin_vel: 1.0 @@ -24,21 +35,6 @@ reward: swing_feet_z: 4.0 tracking_sigma: 0.25 base_height_target: 0.3 -training: - task_name: Go2JoystickFlat - sim_backend: motrix -env: - commands: - vel_limit: - - - 0.5 - - 0.0 - - 0.0 - - - 0.5 - - 0.0 - - 0.0 - domain_rand: - randomize_kp: false - randomize_kd: false play_profile: enabled: true env: diff --git a/conf/ppo/task/go2_joystick_flat/mujoco.yaml b/conf/ppo/task/go2_joystick_flat/mujoco.yaml index 67d77d03f..35cf09bed 100644 --- a/conf/ppo/task/go2_joystick_flat/mujoco.yaml +++ b/conf/ppo/task/go2_joystick_flat/mujoco.yaml @@ -1,16 +1,19 @@ # @package _global_ +training: + task_name: Go2JoystickFlat + sim_backend: mujoco algo: num_envs: 1024 max_iterations: 151 empirical_normalization: true obs_groups: actor: - - actor + - actor policy: init_noise_std: 0.5 algorithm: - learning_rate: 0.0003 - entropy_coef: 0.001 + learning_rate: 3.0e-4 + entropy_coef: 1.0e-3 reward: scales: tracking_lin_vel: 1.0 @@ -24,9 +27,6 @@ reward: swing_feet_z: 4.0 tracking_sigma: 0.25 base_height_target: 0.3 -training: - task_name: Go2JoystickFlat - sim_backend: mujoco play_profile: enabled: true env: diff --git a/conf/ppo/task/go2_joystick_rough/motrix.yaml b/conf/ppo/task/go2_joystick_rough/motrix.yaml index 6b6d07eca..46992b1f0 100644 --- a/conf/ppo/task/go2_joystick_rough/motrix.yaml +++ b/conf/ppo/task/go2_joystick_rough/motrix.yaml @@ -1,17 +1,19 @@ # @package _global_ training: + task_name: Go2JoystickRough + sim_backend: motrix play_steps: 500 play_env_num: 16 cam_tracking: true cam_tracking_env_idx: 0 cam_tracking_extra_envs: 9 - task_name: Go2JoystickRough - sim_backend: motrix + interactive: action_mode: policy policy_obs_mode: auto camera_follow_body: true use_env_visual_model: false + algo: num_envs: 4096 num_steps_per_env: 24 @@ -19,49 +21,41 @@ algo: empirical_normalization: false obs_groups: actor: - - actor + - actor critic: - - critic + - critic policy: init_noise_std: 1.0 algorithm: - learning_rate: 0.001 - entropy_coef: 0.01 + learning_rate: 1.0e-3 + entropy_coef: 1.0e-2 + env: + render_offset_mode: zero control_config: action_scale: 0.25 hip_action_scale: 0.125 non_hip_action_scale: 0.25 clip_actions: 100.0 commands: - vel_limit: - - - -1.0 - - -1.0 - - -1.0 - - - 1.0 - - 1.0 - - 1.0 + vel_limit: [[-1.0, -1.0, -1.0], [1.0, 1.0, 1.0]] resampling_time: 10.0 heading_command: true - heading_range: - - -3.141592653589793 - - 3.141592653589793 + heading_range: [-3.141592653589793, 3.141592653589793] rel_standing_envs: 0.1 terrain_curriculum: enabled: false scene: model_file: src/unilab/assets/robots/go2/go2.xml fragment_files: - - src/unilab/assets/robots/go2/locomotion_task.xml + - src/unilab/assets/robots/go2/locomotion_task.xml terrain: hfield_name: terrain_hfield geom_name: floor generator: seed: 42 curriculum: false - size: - - 8.0 - - 8.0 + size: [8.0, 8.0] num_rows: 6 num_cols: 6 border_width: 20.0 @@ -73,39 +67,29 @@ env: terrain_distance_buffer: 3.0 domain_rand: randomize_base_mass: true - added_mass_range: - - -1.0 - - 3.0 + added_mass_range: [-1.0, 3.0] random_com: true randomize_kp: true - kp_multiplier_range: - - 0.5 - - 2.0 + kp_multiplier_range: [0.5, 2.0] randomize_kd: true - kd_multiplier_range: - - 0.5 - - 2.0 + kd_multiplier_range: [0.5, 2.0] push_robots: true push_interval: 625 - max_force: - - 1.0 - - 1.0 - - 0.5 - render_offset_mode: zero + max_force: [1.0, 1.0, 0.5] reward: scales: lin_vel_z: -2.0 ang_vel_xy: -0.05 - joint_torques_l2: -2.5e-05 - joint_acc_l2: -2.5e-07 - joint_power: -2.0e-05 + joint_torques_l2: -2.5e-5 + joint_acc_l2: -2.5e-7 + joint_power: -2.0e-5 stand_still: -2.0 hip_pos: -0.5 joint_pos_penalty: -1.0 joint_mirror: -0.05 action_rate: -0.01 undesired_contacts: -1.0 - contact_forces: -0.00015 + contact_forces: -1.5e-4 tracking_lin_vel: 3.0 tracking_ang_vel: 1.5 feet_air_time: 0.5 diff --git a/conf/ppo/task/go2_joystick_rough/mujoco.yaml b/conf/ppo/task/go2_joystick_rough/mujoco.yaml index 32a3ce0e1..ea665a5fa 100644 --- a/conf/ppo/task/go2_joystick_rough/mujoco.yaml +++ b/conf/ppo/task/go2_joystick_rough/mujoco.yaml @@ -1,18 +1,20 @@ # @package _global_ training: + task_name: Go2JoystickRough + sim_backend: mujoco play_steps: 500 play_env_num: 16 + render_spacing: 0.0 cam_tracking: true cam_tracking_env_idx: 0 cam_tracking_extra_envs: 9 - task_name: Go2JoystickRough - sim_backend: mujoco - render_spacing: 0.0 + interactive: action_mode: policy policy_obs_mode: auto camera_follow_body: true use_env_visual_model: false + algo: num_envs: 2048 num_steps_per_env: 24 @@ -20,49 +22,41 @@ algo: empirical_normalization: false obs_groups: actor: - - actor + - actor critic: - - critic + - critic policy: init_noise_std: 1.0 algorithm: - learning_rate: 0.001 - entropy_coef: 0.01 + learning_rate: 1.0e-3 + entropy_coef: 1.0e-2 + env: + sim_dt: 0.002 control_config: action_scale: 0.25 hip_action_scale: 0.125 non_hip_action_scale: 0.25 clip_actions: 100.0 commands: - vel_limit: - - - -1.0 - - -1.0 - - -1.0 - - - 1.0 - - 1.0 - - 1.0 + vel_limit: [[-1.0, -1.0, -1.0], [1.0, 1.0, 1.0]] resampling_time: 10.0 heading_command: true - heading_range: - - -3.141592653589793 - - 3.141592653589793 + heading_range: [-3.141592653589793, 3.141592653589793] rel_standing_envs: 0.1 terrain_curriculum: enabled: false scene: model_file: src/unilab/assets/robots/go2/go2_mujoco.xml fragment_files: - - src/unilab/assets/robots/go2/locomotion_task.xml + - src/unilab/assets/robots/go2/locomotion_task.xml terrain: hfield_name: terrain_hfield geom_name: floor generator: seed: 42 curriculum: false - size: - - 8.0 - - 8.0 + size: [8.0, 8.0] num_rows: 6 num_cols: 6 border_width: 20.0 @@ -74,39 +68,30 @@ env: terrain_distance_buffer: 3.0 domain_rand: randomize_base_mass: true - added_mass_range: - - -1.0 - - 3.0 + added_mass_range: [-1.0, 3.0] random_com: true randomize_kp: true - kp_multiplier_range: - - 0.5 - - 2.0 + kp_multiplier_range: [0.5, 2.0] randomize_kd: true - kd_multiplier_range: - - 0.5 - - 2.0 + kd_multiplier_range: [0.5, 2.0] push_robots: true push_interval: 625 - max_force: - - 1.0 - - 1.0 - - 0.5 - sim_dt: 0.002 + max_force: [1.0, 1.0, 0.5] + reward: scales: lin_vel_z: -2.0 ang_vel_xy: -0.05 - joint_torques_l2: -2.5e-05 - joint_acc_l2: -2.5e-07 - joint_power: -2.0e-05 + joint_torques_l2: -2.5e-5 + joint_acc_l2: -2.5e-7 + joint_power: -2.0e-5 stand_still: -2.0 hip_pos: -0.5 joint_pos_penalty: -1.0 joint_mirror: -0.05 action_rate: -0.01 undesired_contacts: -1.0 - contact_forces: -0.00015 + contact_forces: -1.5e-4 tracking_lin_vel: 3.0 tracking_ang_vel: 1.5 feet_air_time: 0.5 diff --git a/conf/ppo/task/go2w_joystick_flat/motrix.yaml b/conf/ppo/task/go2w_joystick_flat/motrix.yaml index 1e87e13fd..0e86b63b0 100644 --- a/conf/ppo/task/go2w_joystick_flat/motrix.yaml +++ b/conf/ppo/task/go2w_joystick_flat/motrix.yaml @@ -1,25 +1,25 @@ # @package _global_ +training: + task_name: Go2WJoystickFlat + sim_backend: motrix algo: num_envs: 1024 max_iterations: 151 empirical_normalization: true obs_groups: actor: - - actor + - actor policy: init_noise_std: 0.5 algorithm: - learning_rate: 0.0003 - entropy_coef: 0.001 + learning_rate: 3.0e-4 + entropy_coef: 1.0e-3 env: + render_offset_mode: zero commands: vel_limit: - - - 0.0 - - 0.0 - - -1.0 - - - 1.0 - - 0.0 - - 1.0 + - [0.0, 0.0, -1.0] + - [1.0, 0.0, 1.0] control_config: action_scale: 0.5 wheel_action_scale: 10.0 @@ -29,7 +29,6 @@ env: domain_rand: randomize_kp: false randomize_kd: false - render_offset_mode: zero reward: scales: tracking_lin_vel: 1.0 @@ -46,9 +45,6 @@ reward: upward: 1.0 tracking_sigma: 0.25 base_height_target: 0.4 -training: - task_name: Go2WJoystickFlat - sim_backend: motrix play_profile: enabled: true env: diff --git a/conf/ppo/task/go2w_joystick_flat/mujoco.yaml b/conf/ppo/task/go2w_joystick_flat/mujoco.yaml index 548ba07e2..602b95a27 100644 --- a/conf/ppo/task/go2w_joystick_flat/mujoco.yaml +++ b/conf/ppo/task/go2w_joystick_flat/mujoco.yaml @@ -1,25 +1,24 @@ # @package _global_ +training: + task_name: Go2WJoystickFlat + sim_backend: mujoco algo: num_envs: 1024 max_iterations: 151 empirical_normalization: true obs_groups: actor: - - actor + - actor policy: init_noise_std: 0.5 algorithm: - learning_rate: 0.0003 - entropy_coef: 0.001 + learning_rate: 3.0e-4 + entropy_coef: 1.0e-3 env: commands: vel_limit: - - - 0.0 - - 0.0 - - -1.0 - - - 1.0 - - 0.0 - - 1.0 + - [0.0, 0.0, -1.0] + - [1.0, 0.0, 1.0] control_config: action_scale: 0.5 wheel_action_scale: 10.0 @@ -45,9 +44,6 @@ reward: upward: 1.0 tracking_sigma: 0.25 base_height_target: 0.4 -training: - task_name: Go2WJoystickFlat - sim_backend: mujoco play_profile: enabled: true env: diff --git a/conf/ppo/task/go2w_joystick_rough/motrix.yaml b/conf/ppo/task/go2w_joystick_rough/motrix.yaml index 30bcfcd8c..d15620ddb 100644 --- a/conf/ppo/task/go2w_joystick_rough/motrix.yaml +++ b/conf/ppo/task/go2w_joystick_rough/motrix.yaml @@ -1,17 +1,19 @@ # @package _global_ training: + task_name: Go2WJoystickRough + sim_backend: motrix play_steps: 500 play_env_num: 16 cam_tracking: true cam_tracking_env_idx: 0 cam_tracking_extra_envs: 9 - task_name: Go2WJoystickRough - sim_backend: motrix + interactive: action_mode: policy policy_obs_mode: auto camera_follow_body: true use_env_visual_model: false + algo: num_envs: 2048 num_steps_per_env: 24 @@ -19,39 +21,30 @@ algo: empirical_normalization: false obs_groups: actor: - - actor + - actor critic: - - critic + - critic env: + render_offset_mode: zero scene: model_file: src/unilab/assets/robots/go2w/go2w.xml fragment_files: - - src/unilab/assets/robots/go2w/locomotion_task.xml + - src/unilab/assets/robots/go2w/locomotion_task.xml terrain: hfield_name: terrain_hfield geom_name: floor generator: seed: 42 curriculum: false - size: - - 8.0 - - 8.0 + size: [8.0, 8.0] num_rows: 6 num_cols: 6 border_width: 20.0 commands: - vel_limit: - - - -1.0 - - -1.0 - - -1.0 - - - 1.0 - - 1.0 - - 1.0 + vel_limit: [[-1.0, -1.0, -1.0], [1.0, 1.0, 1.0]] resampling_time: 10.0 heading_command: true - heading_range: - - -3.141592653589793 - - 3.141592653589793 + heading_range: [-3.141592653589793, 3.141592653589793] rel_standing_envs: 0.1 control_config: action_scale: 0.25 @@ -69,29 +62,17 @@ env: terrain_distance_buffer: 3.0 domain_rand: randomize_base_mass: true - added_mass_range: - - -1.0 - - 3.0 + added_mass_range: [-1.0, 3.0] random_com: true - com_offset_x: - - -0.05 - - 0.05 + com_offset_x: [-0.05, 0.05] randomize_kp: true - kp_multiplier_range: - - 0.5 - - 1.0 + kp_multiplier_range: [0.5, 1.0] randomize_kd: true - kd_multiplier_range: - - 0.5 - - 1.0 + kd_multiplier_range: [0.5, 1.0] push_robots: true push_interval: 625 - max_force: - - 1.0 - - 1.0 - - 0.5 + max_force: [1.0, 1.0, 0.5] push_body_name: base_link - render_offset_mode: zero reward: scales: tracking_lin_vel: 3.0 @@ -99,10 +80,10 @@ reward: lin_vel_z: -2.0 ang_vel_xy: -0.05 orientation: -2.0 - joint_torques_l2: -2.5e-05 - joint_acc_l2: -2.5e-07 - joint_acc_wheel_l2: -2.5e-09 - joint_power: -2.0e-05 + joint_torques_l2: -2.5e-5 + joint_acc_l2: -2.5e-7 + joint_acc_wheel_l2: -2.5e-9 + joint_power: -2.0e-5 action_rate: -0.01 stand_still: -2.0 hip_pos: -0.5 diff --git a/conf/ppo/task/go2w_joystick_rough/mujoco.yaml b/conf/ppo/task/go2w_joystick_rough/mujoco.yaml index 628d64895..5bcc31038 100644 --- a/conf/ppo/task/go2w_joystick_rough/mujoco.yaml +++ b/conf/ppo/task/go2w_joystick_rough/mujoco.yaml @@ -1,18 +1,20 @@ # @package _global_ training: + task_name: Go2WJoystickRough + sim_backend: mujoco play_steps: 500 play_env_num: 16 + render_spacing: 0.0 cam_tracking: true cam_tracking_env_idx: 0 cam_tracking_extra_envs: 9 - task_name: Go2WJoystickRough - sim_backend: mujoco - render_spacing: 0.0 + interactive: action_mode: policy policy_obs_mode: auto camera_follow_body: true use_env_visual_model: false + algo: num_envs: 2048 num_steps_per_env: 24 @@ -20,39 +22,29 @@ algo: empirical_normalization: false obs_groups: actor: - - actor + - actor critic: - - critic + - critic env: scene: model_file: src/unilab/assets/robots/go2w/go2w_mujoco.xml fragment_files: - - src/unilab/assets/robots/go2w/locomotion_task.xml + - src/unilab/assets/robots/go2w/locomotion_task.xml terrain: hfield_name: terrain_hfield geom_name: floor generator: seed: 42 curriculum: false - size: - - 8.0 - - 8.0 + size: [8.0, 8.0] num_rows: 6 num_cols: 6 border_width: 20.0 commands: - vel_limit: - - - -1.0 - - -1.0 - - -1.0 - - - 1.0 - - 1.0 - - 1.0 + vel_limit: [[-1.0, -1.0, -1.0], [1.0, 1.0, 1.0]] resampling_time: 10.0 heading_command: true - heading_range: - - -3.141592653589793 - - 3.141592653589793 + heading_range: [-3.141592653589793, 3.141592653589793] rel_standing_envs: 0.1 control_config: action_scale: 0.25 @@ -70,27 +62,16 @@ env: terrain_distance_buffer: 3.0 domain_rand: randomize_base_mass: true - added_mass_range: - - -1.0 - - 3.0 + added_mass_range: [-1.0, 3.0] random_com: true - com_offset_x: - - -0.05 - - 0.05 + com_offset_x: [-0.05, 0.05] randomize_kp: true - kp_multiplier_range: - - 0.5 - - 1.0 + kp_multiplier_range: [0.5, 1.0] randomize_kd: true - kd_multiplier_range: - - 0.5 - - 1.0 + kd_multiplier_range: [0.5, 1.0] push_robots: true push_interval: 625 - max_force: - - 1.0 - - 1.0 - - 0.5 + max_force: [1.0, 1.0, 0.5] push_body_name: base_link reward: scales: @@ -99,10 +80,10 @@ reward: lin_vel_z: -2.0 ang_vel_xy: -0.05 orientation: -2.0 - joint_torques_l2: -2.5e-05 - joint_acc_l2: -2.5e-07 - joint_acc_wheel_l2: -2.5e-09 - joint_power: -2.0e-05 + joint_torques_l2: -2.5e-5 + joint_acc_l2: -2.5e-7 + joint_acc_wheel_l2: -2.5e-9 + joint_power: -2.0e-5 action_rate: -0.01 stand_still: -2.0 hip_pos: -2.0 diff --git a/docs/sphinx/source/en/3-deployment/2-sim_to_sim/0-index.md b/docs/sphinx/source/en/3-deployment/2-sim_to_sim/0-index.md index f0f725fb0..09fbc9c19 100644 --- a/docs/sphinx/source/en/3-deployment/2-sim_to_sim/0-index.md +++ b/docs/sphinx/source/en/3-deployment/2-sim_to_sim/0-index.md @@ -42,6 +42,12 @@ Understand renderer and snapshot capability differences. Document unsupported backend features with evidence. ::: +:::{grid-item-card} Config guard +:link: 7-config_guard +:link-type: doc +Auto-check policy contract compatibility on cross-backend replay. +::: + :::: ```{toctree} @@ -53,4 +59,5 @@ Document unsupported backend features with evidence. 4-reward_parity 5-playback_and_snapshot_differences 6-capability_gaps +7-config_guard ``` diff --git a/docs/sphinx/source/en/3-deployment/2-sim_to_sim/7-config_guard.md b/docs/sphinx/source/en/3-deployment/2-sim_to_sim/7-config_guard.md new file mode 100644 index 000000000..3a02f4e97 --- /dev/null +++ b/docs/sphinx/source/en/3-deployment/2-sim_to_sim/7-config_guard.md @@ -0,0 +1,47 @@ +# Cross-Backend Config Guard + +When replaying across backends (train on one backend, `eval` on another), UniLab automatically checks whether the target backend config is compatible with the policy contract captured at training time, so a mismatched config can't silently load a broken policy. This happens with no manual steps. + +## An example that replays successfully + +For `go2_joystick_flat`, the MuJoCo and Motrix owners agree on every guarded field, so a cross-backend replay passes directly: + +```bash +# 1) Train in MuJoCo, producing a checkpoint +uv run train --algo ppo --task go2_joystick_flat --sim mujoco + +# 2) Replay the same checkpoint across backends in Motrix — the guard passes, playback runs +uv run eval --algo ppo --task go2_joystick_flat --sim motrix --load-run -1 +``` + +## How the guard chain works + +1. **At train time**: `ExperimentTracker` snapshots the contract fields that define policy I/O into `contract_snapshot` in `run_config.json` (the checkpoint format is untouched, so historical checkpoints stay compatible). +2. **At replay time**: `eval` loads the **target backend** owner config selected by `--sim` (e.g. `conf/ppo/task/go2_joystick_flat/motrix.yaml`) and injects `training.play_only=true`. +3. **Before env creation**: the five play entrypoints (rsl_rl / appo / offpolicy / mlx_ppo / him_ppo) call `resolve_sim2sim_config`, comparing the target config against the source run's contract snapshot field by field. +4. **At weight load**: `policy_load_dim_guard` wraps checkpoint loading, re-raising cryptic tensor shape-mismatch errors as a clear sim2sim diagnostic. + +## What the guard covers + +Fields are classified by dotted path into three tiers (see `src/unilab/training/sim2sim.py`): + +| Tier | Behavior | Fields | +|---|---|---| +| **DENYLIST** | Mismatch → `CrossBackendIncompatibleError`, aborts | `algo.obs_groups`, `env.control_config.action_scale`, `algo.policy.actor_hidden_dims` / `critic_hidden_dims`, `algo.empirical_normalization` / `algo.obs_normalization`, `env.sampling_mode` | +| **WARNING_LIST** | Prints a warning, continues | `reward.*`, `env.control_config.simulate_action_latency`, `env.ctrl_dt` | +| **ALLOWLIST** | Free to override, not checked | `training.sim_backend`, `env.scene`, `training.play_steps`, `env.domain_rand`, `env.noise_config`, `env.commands.vel_limit` | + +## When DENYLIST fields differ + +If the target backend's DENYLIST fields differ from training (e.g. a task whose two owners use different `action_scale` values), the guard aborts before env creation and lists the diverging fields. Two ways to resolve: + +- **Align the contract** (recommended): make the target owner's DENYLIST fields match the training backend, then replay. +- **Force through** (at your own risk): `uv run eval ... training.sim2sim_strict=false` downgrades DENYLIST mismatches to warnings. + +> Legacy runs: if `run_config.json` has no `contract_snapshot` (older training), the guard skips with a warning instead of breaking your workflow. + +## See also + +- {doc}`1-backend_swap` +- {doc}`4-reward_parity` +- {doc}`../../4-developer_guide/9-sim2sim_contract_status` diff --git a/docs/sphinx/source/zh_CN/3-deployment/2-sim_to_sim/0-index.md b/docs/sphinx/source/zh_CN/3-deployment/2-sim_to_sim/0-index.md index 24ba0ed5b..43c65be22 100644 --- a/docs/sphinx/source/zh_CN/3-deployment/2-sim_to_sim/0-index.md +++ b/docs/sphinx/source/zh_CN/3-deployment/2-sim_to_sim/0-index.md @@ -41,6 +41,12 @@ 用证据记录不受支持的后端特性。 ::: +:::{grid-item-card} 配置守卫 +:link: 7-config_guard +:link-type: doc +跨后端回放时自动校验策略契约兼容性。 +::: + :::: ```{toctree} @@ -52,4 +58,5 @@ 4-reward_parity 5-playback_and_snapshot_differences 6-capability_gaps +7-config_guard ``` diff --git a/docs/sphinx/source/zh_CN/3-deployment/2-sim_to_sim/7-config_guard.md b/docs/sphinx/source/zh_CN/3-deployment/2-sim_to_sim/7-config_guard.md new file mode 100644 index 000000000..c01a631fc --- /dev/null +++ b/docs/sphinx/source/zh_CN/3-deployment/2-sim_to_sim/7-config_guard.md @@ -0,0 +1,47 @@ +# 跨后端配置守卫 + +跨后端回放（在一个后端训练、到另一个后端 `eval`）时，UniLab 会自动校验目标后端配置与训练时的策略契约是否兼容，避免用错配置静默加载出行为异常的策略。整个过程无需手动干预。 + +## 一个可成功回放的例子 + +`go2_joystick_flat` 的 MuJoCo 与 Motrix owner 在守卫字段上完全一致，因此跨后端回放可以直接通过： + +```bash +# 1) MuJoCo 训练，产生 checkpoint +uv run train --algo ppo --task go2_joystick_flat --sim mujoco + +# 2) Motrix 跨后端回放同一个 checkpoint —— 守卫校验通过，正常播放 +uv run eval --algo ppo --task go2_joystick_flat --sim motrix --load-run -1 +``` + +## 生效链路 + +1. **训练时**：`ExperimentTracker` 把决定策略 I/O 的契约字段快照进 `run_config.json` 的 `contract_snapshot`（不改动 checkpoint 格式，历史 checkpoint 天然兼容）。 +2. **回放时**：`eval` 读取 `--sim` 指定的**目标后端** owner 配置（如 `conf/ppo/task/go2_joystick_flat/motrix.yaml`），并注入 `training.play_only=true`。 +3. **建 env 前**：五个 play 入口（rsl_rl / appo / offpolicy / mlx_ppo / him_ppo）调用 `resolve_sim2sim_config`，把目标配置与源 run 的契约快照逐字段比对。 +4. **加载权重时**：`policy_load_dim_guard` 包裹 checkpoint 加载，把底层 tensor 维度不匹配的晦涩报错重抛为清晰的 sim2sim 诊断。 + +## 守卫的字段 + +守卫按 dotted path 分三档（定义见 `src/unilab/training/sim2sim.py`）： + +| 档位 | 行为 | 字段 | +|---|---|---| +| **DENYLIST** | 差异即 `CrossBackendIncompatibleError`，中断 | `algo.obs_groups`、`env.control_config.action_scale`、`algo.policy.actor_hidden_dims` / `critic_hidden_dims`、`algo.empirical_normalization` / `algo.obs_normalization`、`env.sampling_mode` | +| **WARNING_LIST** | 仅打印 warning，继续 | `reward.*`、`env.control_config.simulate_action_latency`、`env.ctrl_dt` | +| **ALLOWLIST** | 自由覆盖，不检查 | `training.sim_backend`、`env.scene`、`training.play_steps`、`env.domain_rand`、`env.noise_config`、`env.commands.vel_limit` | + +## 当 DENYLIST 字段不一致时 + +若目标后端的 DENYLIST 字段与训练时不同（例如某 task 的两端 owner 在 `action_scale` 上取值不同），守卫会在建 env 前中断并列出差异字段。处理方式二选一： + +- **对齐契约**（推荐）：让目标后端 owner 的 DENYLIST 字段与训练后端一致，再回放。 +- **强制放行**（自担风险）：`uv run eval ... training.sim2sim_strict=false`，把 DENYLIST 差异降级为 warning。 + +> 兼容旧 run：若 `run_config.json` 没有 `contract_snapshot`（早期训练），守卫自动跳过并打印 warning，不会中断现有工作流。 + +## 另请参阅 + +- {doc}`1-backend_swap` +- {doc}`4-reward_parity` +- {doc}`../../4-developer_guide/9-sim2sim_contract_status` diff --git a/docs/sphinx/source/zh_CN/4-developer_guide/9-sim2sim_contract_status.md b/docs/sphinx/source/zh_CN/4-developer_guide/9-sim2sim_contract_status.md index 9505ff003..18459b02d 100644 --- a/docs/sphinx/source/zh_CN/4-developer_guide/9-sim2sim_contract_status.md +++ b/docs/sphinx/source/zh_CN/4-developer_guide/9-sim2sim_contract_status.md @@ -10,7 +10,7 @@ uv run scripts/audit_sim2sim_contracts.py ``` -该脚本只读，对每个 task 用 hydra `compose` 得到有效配置（展开 `defaults` / `base.yaml` / +该脚本只读，对每个 task 用 hydra `compose` 得到有效配置（展开 `defaults` / `# @package _global_`），再按守卫同一套归一化逐字段比对，输出本页相同的判定。 判定分三档： @@ -64,11 +64,11 @@ uv run scripts/audit_sim2sim_contracts.py | 字段 | 是否可仅改 YAML | 说明 | |---|---|---| -| `obs_groups` | 可（多数情况） | 命名差异统一进 `base.yaml`，actor 部署不变 | +| `obs_groups` | 可（多数情况） | 命名差异在两后端 owner 中统一，actor 部署不变 | | `sampling_mode` | 可（取决于 task） | 两后端运行时已同值时只需补齐显式声明 | | `action_scale` | **不可** | 改值即改训练动力学，必须 owner 决策 + 重训 | | `empirical_normalization` | **不可** | 改变网络结构，必须重训 | -试点示例：`conf/ppo/task/g1_walk_flat/{base,mujoco,motrix}.yaml`。`base.yaml` 承载共享契约， -`mujoco.yaml` 直接继承，`motrix.yaml` 为单后端调参 override 了若干契约字段——这种 override +试点示例：`conf/ppo/task/g1_walk_flat/{mujoco,motrix}.yaml`。每个后端 owner 自包含完整契约， +`motrix.yaml` 为单后端调参 override 了若干契约字段——这种 override 即令该 task 在该后端不可 sim2sim 迁移，去掉 override 即可恢复。 diff --git a/uv b/uv deleted file mode 100644 index e69de29bb..000000000