Skip to content

docs(codec-eval): make the API quick-start compilable (callback signature + report shape)#2

Merged
lilith merged 1 commit into
mainfrom
docfix-codeceval-readme
Jun 16, 2026
Merged

docs(codec-eval): make the API quick-start compilable (callback signature + report shape)#2
lilith merged 1 commit into
mainfrom
docfix-codeceval-readme

Conversation

@lilith

@lilith lilith commented Jun 16, 2026

Copy link
Copy Markdown
Member

Why

An "insulated external developer" usability test (given only the README) found the methodology/fair-comparison content excellent, but the API quick-start was uncompilable. Concretely:

  • The encode-callback was written against fictional types: Box::new(|image, request| my_codec::encode(image, request.quality)) with no real image/request/return types shown.
  • The example called session.evaluate_corpus("./test_images")evaluate_corpus does not exist in the crate. The real method is evaluate_image(name, ImageData).
  • The returned report was never read — no field of ImageReport/CodecResult was shown, so a reader couldn't tell how to get a metric or a byte count out.
  • quality_levels semantics were ambiguous: encoder quality (0-100) vs. target SSIMULACRA2 scores. The README's own "don't compare by quality" methodology made this genuinely confusing.
  • The install lines claimed git-only, but the library is published on crates.io as codec-eval = "0.3.2".

What this changes (README only, + a CHANGELOG note)

Rewrites the quick-start so every codec-eval symbol resolves, verified by compiling a faithful skeleton (cargo check, clean):

  • Real callback signatureBox::new(|image: &ImageData, request: &EncodeRequest| -> Result<Vec<u8>> { ... }), matching EncodeFn in src/eval/session.rs:181. Shows image.to_rgb8_vec() / image.width() / image.height() and returns encoded bytes.
  • Correct entry pointsession.evaluate_image("test.png", reference)? returning an ImageReport (src/eval/session.rs:368), with a note that it takes an already-decoded ImageData (the harness doesn't read files) and a pointer to Corpus::get_dataset(...) for the built-in corpora.
  • Report-read snippet — iterates report.results and prints codec_id, quality, file_size, bits_per_pixel, metrics.ssimulacra2, metrics.dssim (fields from src/eval/report.rs), plus write_image_report.
  • quality_levels clarified — documented inline as encoder quality settings (0-100) drawn into request.quality (a codec-specific dial), not target metric scores (EncodeRequest::quality is f64, "0-100, codec-specific interpretation", src/eval/session.rs:153).
  • Accurate installcargo add codec-eval (crates.io) as the primary line; git for unreleased; the companion codec-eval-cli flagged git-only.

Note: EncodeRequest is not re-exported at the crate root, so the example imports it via its real path codec_eval::eval::session::EncodeRequest (flagged in a comment). The same src/lib.rs doc-comment quick-start has the analogous Ok(encoded_bytes)-with-undefined-vars shape; left out of scope to keep this diff README-only, but worth a follow-up.

Methodology / fair-comparison / R-D / viewing-conditions content is unchanged.

… signature + report-read + quality_levels semantics (insulated-dev usability test)
@lilith lilith self-assigned this Jun 16, 2026
@lilith
lilith merged commit 883a4f6 into main Jun 16, 2026
10 checks passed
@lilith
lilith deleted the docfix-codeceval-readme branch June 16, 2026 09:52
@codecov-commenter

Copy link
Copy Markdown

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants