Skip to content

Simplify the 'Improvement' signal row to a terse score instead of a text wall #5101

Description

@JSONbored

Part of #5095.

Context

The optional "Improvement" row (#4744/#4745, improvementEvidenceText in src/signals/engine.ts) is now live and rendering, but its Evidence cell is a wall of text by construction:

```ts
const deterministicPart = band === "insufficient-signal"
? "Nothing measurable for the structural-improvement analyzers on this PR (e.g. no code files changed)."
: findingSentences.slice(0, 2).join(" ") + (findingSentences.length > 2 ? ` (+${findingSentences.length - 2} more.)` : "");
const valuePart = safeValueAssessment ? ` LLM value judgment: ${safeValueAssessment.magnitude} — ${safeValueAssessment.rationale}` : "";
return `${deterministicPart}${valuePart}`;
```

Up to 2 raw finding sentences PLUS the full LLM rationale sentence get concatenated into ONE cell -- confirmed directly from a live rendering: "risk: clean · value: insufficient-signal — Nothing measurable for the structural-improvement analyzers on this PR (e.g. no code files changed). LLM value judgment: moderate — This is a small, well-targeted CI ordering fix that closes a real gap..." Maintainer feedback: they want "a quick, clean, simple score/rating/improvement" -- another indicator of the PR's quality/value vs. pre-existing code, not a paragraph.

Requirements

Deliverables

  • Redesigned improvementEvidenceText (or its replacement) with full test coverage across every ImprovementBand × risk-quadrant combination already tested today.
  • Before/after examples for at least 3 real scenarios (insufficient-signal, a genuine finding, a finding + LLM rationale both present).

Expected outcome

A maintainer glancing at the Improvement row gets a single clear read ("this PR is a moderate net improvement") without parsing a paragraph, with the detail still available (collapsible or otherwise) for anyone who wants it.

Open questions for the maintainer

  • Whether the full rationale should move to a collapsible, get truncated harder, or just be dropped from the public comment entirely (kept in the AI-reviewer's own findings, not duplicated here).

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.roadmapOn the Wave-2 agent-layer roadmap board (project 9)

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions