Skip to content

Add silent-failure severity escalation guidance to code-review skill and correctness sub-agent #33

Description

@fullsend-ai-retro

What happened

Issue #1144 in fullsend-ai/fullsend proposed adding silent-failure severity escalation guidance to the review agent's code-review skill. Two concrete instances validated the need: PR #1079 (awk path truncation rated [low], later confirmed as a real bug) and PR #2947 (annotated-tag silent 404 rated [low] across 4+ review passes while a human squad rated it [HIGH]).

PR #3156 implemented the fix but targeted the wrong repo (internal/scaffold/fullsend-repo/ in fullsend-ai/fullsend). ralphbean closed both PR #3156 and issue #1144, redirecting to the agents repo. However, no corresponding issue was created in fullsend-ai/agents, and issue #3153 remains open in the wrong repo with no path to resolution.

What could go better

When ralphbean closed #1144 with a redirect comment, no automated or manual process ensured the work was re-filed in the target repo. The well-specified content from #1144 (proposed text, validation criteria, triage analysis, RICE prioritization) and the concrete implementation from PR #3156 (88 lines of tested, well-placed guidance) are stranded — neither is discoverable from the agents repo.

Confidence: High that this is a gap — the work has two data points and a ready implementation but no tracking issue in the correct repo. Medium that the PR #3156 content can be applied directly to the agents repo's skills/code-review/SKILL.md and skills/pr-review/sub-agents/correctness.md with minimal adaptation.

Proposed change

File a new issue in fullsend-ai/agents to add silent-failure severity escalation guidance to the code-review skill (Section 4, before "Severity anchoring") and the correctness sub-agent. The content from PR #3156's diff provides a ready implementation: a subsection instructing reviewers to escalate severity by one level when a correctness bug produces silent failures (no error, no warning, appears to succeed but produces wrong results), with explicit examples of silent vs loud failures and guidance that probability-based reasoning must not override failure-mode-based severity.

The issue should reference the originating evidence: fullsend-ai/fullsend#1144, fullsend-ai/fullsend#3153, PR #2947 (annotated-tag bug), and PR #1079 (awk path truncation).

Validation criteria

  1. After the guidance is added: re-run the review agent on PR #2947's 3ce5eb7 diff. The annotated-tag finding should be rated [medium] or higher in at least 4/5 trials, with severity justification referencing the silent failure mode rather than tag-type probability.
  2. On a test PR containing a loud-failure bug (nil pointer, index out of bounds), the review agent should not over-escalate severity.
  3. Close fullsend-ai/fullsend#3153 once the corresponding agents-repo issue is filed and tracked.

Generated by retro agent from fullsend-ai/fullsend#3156

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions