refactor(#237): add silent-failure severity escalation guidance#269
refactor(#237): add silent-failure severity escalation guidance#269fullsend-ai-coder[bot] wants to merge 1 commit into
Conversation
Add explicit severity escalation rules for silent-failure findings
to the correctness sub-agent, standalone code-review skill, and
challenger sub-agent. When code silently produces wrong results
without any error signal, severity must be medium or higher
regardless of how unlikely the trigger seems today.
Changes:
- correctness.md: new "Silent-failure severity escalation" section
setting a medium severity floor for silent failures and naming
the anti-pattern (probability-based severity discounting)
- challenger.md: carve-out preventing probability-based downgrading
of silent-failure findings
- code-review/SKILL.md: matching guidance in the standalone
correctness dimension
This addresses the systematic bias where the review agent rated
silent-failure bugs as [low] using probability reasoning ("X is
likely Y") rather than failure-mode reasoning ("silent and total
when it triggers"). Evidence: PR fullsend-ai#2947 annotated-tag finding rated
[low] across 4+ passes while a human squad rated [HIGH].
Closes #237
E2E tests did not runE2E tests run automatically for org/repo members and collaborators on pull requests. For other contributors, a maintainer must add the See E2E testing guide for details. |
|
🤖 Finished Review · ❌ Failure · Started 12:58 PM UTC · Completed 12:58 PM UTC |
|
/fs-review |
|
🤖 Finished Review · ✅ Success · Started 3:57 PM UTC · Completed 4:06 PM UTC |
Review — ApprovePR: #269 — refactor(#237): add silent-failure severity escalation guidance SummaryThis PR adds explicit severity escalation rules for silent-failure findings to three agent instruction files: the correctness sub-agent, the standalone code-review skill, and the challenger sub-agent. The change is well-motivated by documented evidence (issue #237) showing the review agent systematically rated silent-failure bugs as [low] using probability reasoning across 4+ review passes, while a human squad rated the same finding [HIGH] using failure-mode reasoning. Dimensions reviewed
Labels: PR modifies agent instruction/skill documentation files in the scaffold |
|
/fs-fix |
|
🤖 Finished Fix · ✅ Success · Started 5:38 PM UTC · Completed 5:43 PM UTC |
🔧 Fix agent — iteration 1 (human-triggered)No fixes needed. The review bot approved PR #269 with zero findings across all dimensions. The human /fs-fix trigger provided no specific instruction. No code changes were made. Disagreed (1):
Tests: passed Updated by fullsend fix agent |
Add explicit severity escalation rules for silent-failure findings to the correctness sub-agent, standalone code-review skill, and challenger sub-agent. When code silently produces wrong results without any error signal, severity must be medium or higher regardless of how unlikely the trigger seems today.
Changes:
setting a medium severity floor for silent failures and naming
the anti-pattern (probability-based severity discounting)
of silent-failure findings
correctness dimension
This addresses the systematic bias where the review agent rated silent-failure bugs as [low] using probability reasoning ("X is likely Y") rather than failure-mode reasoning ("silent and total when it triggers"). Evidence: PR fullsend-ai#2947 annotated-tag finding rated [low] across 4+ passes while a human squad rated [HIGH].
Closes #237
Post-script verification
agent/237-silent-failure-severity-escalation)c776c248ac9a91f10998ca7107e974a6c9d76514..HEAD)