fix(review-pr): stop posting a false LGTM when the agent exits 0 without a review#41
Closed
Sayt-0 wants to merge 1 commit into
Closed
fix(review-pr): stop posting a false LGTM when the agent exits 0 without a review#41Sayt-0 wants to merge 1 commit into
Sayt-0 wants to merge 1 commit into
Conversation
…out a review A degenerate agent run (exit 0, no review posted) triggered the fallback "No issues found" LGTM review, misrepresenting a failed run as a clean review, and incremental-review treated that body as a completed run, permanently skipping those commits on subsequent incremental reviews. - replace the fallback with a neutral incomplete-run notice - dedup guard: paginate and match both the new notice and the legacy prefix - drop the LGTM marker from COMPLETED_BODY_MARKERS (assessment bodies only) - adjust now-stale comments in incremental-review and rate-limit
Member
Author
|
Maybe should try something else |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When the review agent exits 0 without posting a review (a degenerate run, not a clean one), the action posted a fallback review starting with "No issues found" plus "LGTM", misrepresenting a failed run as a clean review. That body also matched
COMPLETED_BODY_MARKERSinsrc/incremental-review, so the unreviewed head SHA was recorded as reviewed and subsequent incremental reviews skipped those commits permanently (silent coverage hole).Evidence: https://github.com/docker/cli/actions/runs/28783584164 (PR docker/cli#7007). The drafter sub-agent returned a degenerate response, the re-delegated drafter got stuck in a read_file/list_directory loop, the agent exited 0 without posting anything, and the action then posted review 4634830529 as a clean LGTM.
Failure chain (before)
Changes
review-pr/action.yml(exit-0 branch only)docker-agentto retry", with a logs link. Contains neither### Assessment:nor the old LGTM prefix, so it can never count as a completed review.review-pr/action.yml(dedup guard)gh api --paginatewith per-page counts summed byjq -s 'add // 0', covering PRs with more than 30 reviews. Matches the new notice plus the legacy LGTM prefix on the reviews endpoint and keeps the legacy issue-comment check, so PRs that already received an old fallback never get a second fallback of either kind. Theecho "0"error fallback becametrue: compositeshell: bashruns with pipefail, and on a gh failure the old pattern would have emitted two lines ("0" from jq plus "0" from echo) and broken the numeric test; the existing${VAR:-0}covers the empty case.src/incremental-review/incremental-review.tsCOMPLETED_BODY_MARKERSreduced to['### Assessment:'], so past and future fallbacks never mark commits as reviewed. The legitimate zero-findings path is unaffected: the agent itself posts the### Assessment:APPROVE body, which stays a completion marker. Header comments updated.src/rate-limit/index.tssrc/incremental-review/__tests__/incremental-review.test.tsBehavior matrix
Validation
pnpm buildpnpm testpnpm lint(biome + tsc + actionlint)bash -eo pipefailwith a stubbedghPossible follow-ups (not included, out of the requested scope)
AGENTS.mdline 183 still says completed reviews are "assessment or LGTM bodies only", now inaccurate.SECURITY.mdline 114 still mentions "timeout/error/LGTM fallbacks".