Add zarr2lerobot converter#6
Merged
Merged
Conversation
Adds tools/zarr2lerobot/, an offline converter that reads the aggregated zarr produced by tools/process_demo_data.py and emits a LeRobot Dataset v3.0 directory via the official lerobot SDK. Features: - Both joint and cartesian control modes (cartesian enforced as 8-dim: tcp_position + tcp_orientation + gripper) - Task text from CLI --task with per-episode description fallback - Bulk per-episode zarr slicing (one chunk decompression per array per episode, not per frame) so large datasets are not O(frames) in read I/O - argparse CLI entry point (python -m tools.zarr2lerobot.convert_zarr_to_lerobot) - Depth frames are intentionally skipped in v1; the v3 ecosystem has no consensus storage format for depth yet Tests: - 22 pytest cases (unit helpers + end-to-end round-trip on synthetic fixture) passing on both joint and cartesian modes - Verified on a real 50-episode / 7075-frame cartesian recording; output loads back via LeRobotDataset with correct episode/frame boundaries The lerobot extra in setup.py is pinned to >=0.4.0,<0.5 from PyPI. 0.4.x is the last series that ships Dataset v3.0 while still supporting Python 3.10 (0.5.0+ requires >=3.12, which breaks the unicon env). Also adds a README section for the new export path and a .gitignore negation for tools/zarr2lerobot/tests/ (repo has a blanket tests/ ignore).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds tools/zarr2lerobot/, an offline converter that reads the aggregated zarr produced by tools/process_demo_data.py and emits a LeRobot Dataset v3.0 directory via the official lerobot SDK.
Features:
Tests:
The lerobot extra in setup.py is pinned to >=0.4.0,<0.5 from PyPI. 0.4.x is the last series that ships Dataset v3.0 while still supporting Python 3.10 (0.5.0+ requires >=3.12, which breaks the unicon env).
Also adds a README section for the new export path and a .gitignore negation for tools/zarr2lerobot/tests/ (repo has a blanket tests/ ignore).