feat(core)!: replace saturating score curve with a monotonic one#19
Open
PunGrumpy wants to merge 3 commits into
Open
feat(core)!: replace saturating score curve with a monotonic one#19PunGrumpy wants to merge 3 commits into
PunGrumpy wants to merge 3 commits into
Conversation
BREAKING CHANGE: score = round(100 * e^(-penalty / 70)) replaces `max(0, 100 - penalty)`. The old formula saturated at 0 once penalty reached ~100 (e.g. ~10 errors), making the score inert on messy repos and defeating score-regression checks. The new curve approaches but never reaches 0, staying strictly monotonic across the whole range. Per-severity weights (error 10, warning 4, info 1) and label thresholds/strings (90/75/50) are unchanged. The JSON report gains a `schemaVersion` field to make future score/shape changes detectable.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Replaces the saturating score formula with a monotonic, asymptotic curve, and versions the JSON report. Implements plan 008. Depends on 002 (CLI tests) and 006 (hardened rule suite — so a score move is attributable to the curve, not a rule regression).
Why
The score was
100 − penaltyfloored at 0 (weights: error 10, warning 4, info 1). Any project with ~10 errors or ~25 warnings scored exactly 0 and stayed there, no matter how much worse (or better) it got. That defeats the entire skill regression workflow ("re-scan, confirm the score didn't drop") — a repo at 0 never moves, so a new Dockerfile with 30 violations still reads 0 and the check green-lights it.The new curve
score = round(100 · e^(−penalty / K)), K = 70, same severity weights (10/4/1). Approaches 0 without ever saturating, so it stays strictly monotonic and responsive across the whole range.K=70 keeps a single warning at 94 (comfortably inside "Excellent" ≥90) while errors and repeated warnings still erode the score. Rationale is documented in a comment in
scoring.tsand in the README's new "How the score works" section.Scope discipline
labelblock and its 90/75/50 thresholds are byte-identical (verified in the diff) — this PR changes the number, not the presentation.rules/**untouched; thescore < 50CLI gate left as-is.Report versioning
The JSON report now carries
schemaVersion: 2(REPORT_SCHEMA_VERSIONexported frompackages/core/src/report.ts) so machine consumers can tell old scores from new. This updates plan 002's strictObject.keysassertion to include the new key (the assertion working as designed).Tests
New
packages/core/test/scoring.test.ts(first tests this module has had): empty→100, per-severity weights, strict monotonicity at 10/20/40 errors (the core regression), fixing an issue always raises the score, integer/range invariants across a wide sweep, label boundaries. Characterization commit3d0b933pins the old behavior in git history before the change.ultracite0 ·fixtures/clean --score→ 99.Changeset
@docker-doctor/climinor (pre-1.0: minor signals breaking). Score values shift; existing badges will show different numbers; label thresholds unchanged.Stacking & merge note
Base is
advisor/006-realistic-negative-test-cases(PR #18 → #16 → #15 → #14). Auto-retargets tomainonce that chain merges.packages/core/src/scoring.ts— it refactors the label block into aSCORE_BUCKETStable, while this PR changes the formula line just above it. The edits are in adjacent-but-distinct regions; whichever of #17 / this PR merges second will need a small conflict resolution inscoring.ts(keep this PR's formula line + K comment and #17'sSCORE_BUCKETSlabel block).🤖 Generated with Claude Code