Skip to content

Implement codec-corpus crate per SPEC.md#1

Merged
lilith merged 2 commits into
mainfrom
implement-codec-corpus-crate
Feb 14, 2026
Merged

Implement codec-corpus crate per SPEC.md#1
lilith merged 2 commits into
mainfrom
implement-codec-corpus-crate

Conversation

@lilith

@lilith lilith commented Feb 7, 2026

Copy link
Copy Markdown
Member

Implements the full codec-corpus Rust crate in crate/ as specified in SPEC.md.

Files added

  • crate/Cargo.toml — Rust 2024 edition, dependencies: dirs, tar_light, fd-lock
  • crate/src/lib.rs (471 lines) — Corpus struct, DatasetInfo, Error types, cache management with version checking and file-based locking
  • crate/src/download.rs (227 lines) — git sparse-checkout and HTTP tarball download strategies with curl/wget/powershell fallback chain

Key features

  • Lazy download of 13 datasets from GitHub releases/tags
  • fd-lock for concurrent access safety across parallel test processes
  • Atomic version file writes and directory moves
  • Stale .tmp-* cleanup for crash recovery
  • Cross-platform cache resolution via dirs::cache_dir() and CODEC_CORPUS_CACHE env var
  • All 9 tests pass (7 unit + 2 doc-tests)

Runtime lazy-loading API for downloading and caching test image
datasets. Shell git sparse checkout as primary method, tar_light
fallback, curl/wget/powershell HTTP fallback. Per-major-version
cache subfolders, atomic downloads with fd-lock, crate version
matches GitHub release tags.
Runtime API for downloading, caching, and accessing test image datasets
from imazen/codec-corpus GitHub releases. No data ships with the crate;
datasets are fetched lazily on first access via git sparse checkout or
HTTP tarball fallback (curl/wget/powershell), then cached locally.

- #![forbid(unsafe_code)], edition 2024, MSRV 1.85
- Shell out to system tar instead of tar_light dependency
- Parameterized PowerShell invocation (no command injection)
- Warn on git tag fallback to default branch
- fd-lock for concurrent download safety
- CI: linux/macos/windows x64 + arm64 (18 jobs)
- Unit tests + network integration tests
@lilith
lilith force-pushed the implement-codec-corpus-crate branch from a072147 to e4a0850 Compare February 14, 2026 13:49
@lilith
lilith merged commit b813ac0 into main Feb 14, 2026
16 checks passed
@lilith
lilith deleted the implement-codec-corpus-crate branch February 14, 2026 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant