Skip to content

Docs: corpus README says 5-case smoke per PR (CI runs full 10); cassette.py docstring path is stale #75

Description

@JumpTechCode

Summary

Two stale doc strings that no longer match the code.

Evidence

  • sentinel/evals/corpus/README.md:38-39 — claims "the deterministic 5-case smoke subset CI runs on every PR." CI actually runs the full 10-case corpus per PR (.github/workflows/ci.yml:108-109, make evals with no --smoke); the corpus has 10 cases.
  • sentinel/evals/cassette.py:11-12 — docstring says cassettes live under cassettes/<prompt_version>/<model_id>/<key>.json, but the actual layout is flat (cassettes/<key>.json); prompt_version/model_id are folded into the key hash (compute_cassette_key, cassette.py:52-70; path at :153).

Fix

Update both doc strings to match reality (full 10-case per-PR run; flat cassette layout).

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions