What happened
On PR conforma/cli#3379, the changed files were: acceptance/go.mod (+1/-1), go.mod (+1/-1), go.sum (+2/-2). Notably, acceptance/go.sum was missing from the changeset even though acceptance/go.mod was updated to require oras-go/v2 v2.6.1. This happened because Renovate's go mod tidy failed in the acceptance directory. The review agent approved the PR without noting this inconsistency. The Acceptance CI check failed confirming the build was broken. On the main branch (PR #3378), both go.sum files were correctly updated — showing what a complete update looks like.
What could go better
The review agent has demonstrated it can reason about Go module completeness — on PR #3378 it correctly identified that tools/go.mod wasn't updated. However, it missed the more obvious signal on PR #3379: a go.mod version bump without a matching go.sum update in the same module. This is a mechanical correctness check that doesn't require deep domain knowledge. The inconsistency between the root module (go.mod + go.sum both updated) and the acceptance module (go.mod updated, go.sum missing) should have been flagged. Confidence is high that this check would generalize well: any Go PR where go.mod changes a version but go.sum is absent from the diff is likely broken. The related issue #1403 covers Go toolchain validation but doesn't specifically address go.sum consistency checking.
Proposed change
Add a Go-specific correctness check to the review skill or review agent guidance that cross-references go.mod changes against the set of changed files. The check should: (1) identify all go.mod files in the PR diff, (2) for each go.mod that adds or changes a dependency version, verify the corresponding go.sum file (in the same directory) is also present in the diff, (3) flag any go.mod/go.sum mismatch as a high-severity correctness finding. This could be implemented as explicit guidance in the review skill's Go module review section, or as a structured pre-check in the harness. Example guidance: 'When reviewing Go dependency updates, verify that every go.mod in the diff has a corresponding go.sum update. A go.mod version bump without a go.sum update means go mod tidy was not run successfully, and the module will fail to build.'
Validation criteria
When presented with a Go dependency PR where go.mod is changed but the corresponding go.sum is missing from the diff, the review agent should flag this as a correctness issue and withhold approval. Can be validated against the PR #3379 diff: the agent should note that acceptance/go.mod was changed but acceptance/go.sum was not.
Generated by retro agent from conforma/cli#3379
What happened
On PR conforma/cli#3379, the changed files were:
acceptance/go.mod(+1/-1),go.mod(+1/-1),go.sum(+2/-2). Notably,acceptance/go.sumwas missing from the changeset even thoughacceptance/go.modwas updated to require oras-go/v2 v2.6.1. This happened because Renovate'sgo mod tidyfailed in the acceptance directory. The review agent approved the PR without noting this inconsistency. TheAcceptanceCI check failed confirming the build was broken. On the main branch (PR #3378), both go.sum files were correctly updated — showing what a complete update looks like.What could go better
The review agent has demonstrated it can reason about Go module completeness — on PR #3378 it correctly identified that
tools/go.modwasn't updated. However, it missed the more obvious signal on PR #3379: a go.mod version bump without a matching go.sum update in the same module. This is a mechanical correctness check that doesn't require deep domain knowledge. The inconsistency between the root module (go.mod + go.sum both updated) and the acceptance module (go.mod updated, go.sum missing) should have been flagged. Confidence is high that this check would generalize well: any Go PR where go.mod changes a version but go.sum is absent from the diff is likely broken. The related issue #1403 covers Go toolchain validation but doesn't specifically address go.sum consistency checking.Proposed change
Add a Go-specific correctness check to the review skill or review agent guidance that cross-references go.mod changes against the set of changed files. The check should: (1) identify all go.mod files in the PR diff, (2) for each go.mod that adds or changes a dependency version, verify the corresponding go.sum file (in the same directory) is also present in the diff, (3) flag any go.mod/go.sum mismatch as a high-severity correctness finding. This could be implemented as explicit guidance in the review skill's Go module review section, or as a structured pre-check in the harness. Example guidance: 'When reviewing Go dependency updates, verify that every go.mod in the diff has a corresponding go.sum update. A go.mod version bump without a go.sum update means
go mod tidywas not run successfully, and the module will fail to build.'Validation criteria
When presented with a Go dependency PR where go.mod is changed but the corresponding go.sum is missing from the diff, the review agent should flag this as a correctness issue and withhold approval. Can be validated against the PR #3379 diff: the agent should note that acceptance/go.mod was changed but acceptance/go.sum was not.
Generated by retro agent from conforma/cli#3379