Skip to content

Commit daf4bcb

Browse files
committed
fix(tests): graded/run counts claim only verdict-bearing cells
The matrix badge counted not-retained re-run gaps as graded plan-runs; both counters now count only cells carrying a recorded verdict.
1 parent 9e2a097 commit daf4bcb

1 file changed

Lines changed: 17 additions & 11 deletions

File tree

app/tests/[testId]/TestDetailClient.tsx

Lines changed: 17 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -322,10 +322,11 @@ export function TestDetailClient({
322322

323323
const phaseCols = agents[0]?.phase_history ?? [];
324324
// Phases this plan actually participates in (its own phase + every later
325-
// scored phase). Drives the "graded across N phases" copy — not the full
326-
// 9-phase roadmap, which is mostly pending columns.
325+
// scored phase). Drives the "graded across N phases" copy — counts only
326+
// phases with a recorded verdict for THIS plan (re-run gaps don't claim
327+
// a grade), not the full roadmap of pending columns.
327328
const gradedPhaseCount = phaseCols.filter(
328-
(h) => !h.pending && !h.not_applicable,
329+
(h) => !h.pending && !h.not_applicable && !h.not_retained,
329330
).length;
330331
// Phase-1 `seo` and Phase-2 `seo` share a catalog key but ship different
331332
// numbers of plans. CATEGORY_META.seo carries the phase-2 description by
@@ -338,10 +339,14 @@ export function TestDetailClient({
338339
const blocked = agents.filter((a) => a.verdict === 'blocked').length;
339340
const inconclusive = agents.length - passed - failed - blocked;
340341
const REPO_BASE = `https://github.com/TestSprite/CoderCup/blob/main/tests/world-cup-2026-v3/phase-${plan.phase}`;
341-
// Total graded plan-runs in the matrix = agents × phases this plan was graded
342-
// in (its own phase carried forward through every later scored phase).
342+
// Recorded plan-runs in the matrix — only cells carrying an actual verdict
343+
// count (not_retained re-run gaps don't claim a recorded run).
343344
const totalRuns = agents.reduce(
344-
(acc, a) => acc + a.phase_history.filter((h) => !h.pending && !h.not_applicable).length,
345+
(acc, a) =>
346+
acc +
347+
a.phase_history.filter(
348+
(h) => !h.pending && !h.not_applicable && !h.not_retained && h.verdict,
349+
).length,
345350
0,
346351
);
347352

@@ -393,8 +398,9 @@ export function TestDetailClient({
393398
<div>
394399
{/* History matrix — one column per phase. A plan is part of the
395400
cumulative suite of every scored phase ≥ its own, so it shows
396-
its verdict across all of them; phases beyond the latest scored
397-
read as a greyed roadmap. */}
401+
its recorded verdict at its authoring phase; later scored
402+
phases show an explicit not-retained gap; phases beyond the
403+
latest scored read as a greyed roadmap. */}
398404
{phaseCols.length > 0 && agents.length > 0 && (
399405
<div className="card-block">
400406
<h2>
@@ -658,10 +664,10 @@ function AgentVerdictBlock({
658664
const statKlass =
659665
v === 'passed' ? 'pass' : v === 'failed' ? 'fail' : v === 'blocked' ? 'blocked' : 'inc';
660666
// How many scored phases this plan was graded in (its own phase carried
661-
// forward through every later scored phase). Used in the "across N phases"
662-
// copy below — phase_history is the source of truth, not a padded length.
667+
// Used in the "across N phases" copy below — counts only phases carrying
668+
// a recorded verdict; phase_history is the source of truth.
663669
const gradedPhaseCount = agent.phase_history.filter(
664-
(h) => !h.pending && !h.not_applicable,
670+
(h) => !h.pending && !h.not_applicable && !h.not_retained,
665671
).length;
666672
// Prefer the per-verdict video_url that the artifact-pull pipeline lands in
667673
// each agent's fixture (1 video per (agent, plan)). Fall back to the legacy

0 commit comments

Comments
 (0)