Our Physical AI Developer team drove the full XR teleop stack against two real arms (SO-101 and ReBot) for a week. The library core is solid; every serious incident happened at the boundaries the library currently leaves implicit: what the follower is assumed to do with a target, what the first frame is allowed to command, and what tracking/encoder state can be trusted after a cycle (power cycle, occlusion re-acquisition, session restart). Four upstream PRs came out of it (#727, #729 on IsaacTeleop; #3954, #3955 on LeRobot), plus a set of concrete library-level suggestions.
Our Physical AI Developer team drove the full XR teleop stack against two real arms (SO-101 and ReBot) for a week. The library core is solid; every serious incident happened at the boundaries the library currently leaves implicit: what the follower is assumed to do with a target, what the first frame is allowed to command, and what tracking/encoder state can be trusted after a cycle (power cycle, occlusion re-acquisition, session restart). Four upstream PRs came out of it (#727, #729 on IsaacTeleop; #3954, #3955 on LeRobot), plus a set of concrete library-level suggestions.