-
Notifications
You must be signed in to change notification settings - Fork 28
chore(devex): Added stamphog to code #2316
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Gilbert09
wants to merge
3
commits into
main
Choose a base branch
from
tom/stamphog
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,255 @@ | ||
| name: PR Approval Agent | ||
|
|
||
| on: | ||
| pull_request: | ||
| types: [labeled, ready_for_review, synchronize] | ||
|
|
||
| permissions: | ||
| contents: read | ||
| pull-requests: write | ||
|
|
||
| concurrency: | ||
| group: pr-approval-${{ github.event.pull_request.number }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| review: | ||
| # Write access is required to apply the Stamphog label, so no | ||
| # additional author_association check is needed. | ||
| # Triggers: explicit `Stamphog` label, ready_for_review with the | ||
| # label already present, or `synchronize` where decide-delta | ||
| # asked for re-review (or itself failed — fail closed for safety). | ||
| needs: [decide-delta, dismiss] | ||
| if: >- | ||
| always() | ||
| && !github.event.pull_request.draft | ||
| && ( | ||
| github.event.label.name == 'Stamphog' | ||
| || (github.event.action == 'ready_for_review' && contains(github.event.pull_request.labels.*.name, 'Stamphog')) | ||
| || needs.decide-delta.outputs.run_review == 'true' | ||
| || needs.decide-delta.result == 'failure' | ||
| ) | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 10 | ||
|
|
||
| steps: | ||
| - name: Get app token | ||
| id: app-token | ||
| uses: actions/create-github-app-token@1b10c78c7865c340bc4f6099eb2f838309f1e8c3 # v3.1.1 | ||
| with: | ||
| client-id: ${{ secrets.GH_APP_PR_APPROVAL_AGENT_APP_ID }} | ||
| private-key: ${{ secrets.GH_APP_PR_APPROVAL_AGENT_PRIVATE_KEY }} | ||
|
|
||
| # Always run the approval script from main — hardcoded so a PR | ||
| # targeting a non-main branch can't supply a tampered script. | ||
| - name: Checkout main (blobless, full history) | ||
| uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 | ||
| with: | ||
| token: ${{ steps.app-token.outputs.token }} | ||
| ref: main | ||
| filter: blob:none | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Fetch PR head ref | ||
| run: git fetch --filter=blob:none origin pull/${{ github.event.pull_request.number }}/head | ||
|
|
||
| - name: Install uv | ||
| uses: astral-sh/setup-uv@eac588ad8def6316056a12d4907a9d4d84ff7a3b # v7.3.0 | ||
| with: | ||
| version: '0.10.2' # pinned: unpinned setup-uv calls GH API on every job, exhausts rate limit | ||
| enable-cache: false | ||
|
|
||
| - name: Run review | ||
| env: | ||
| ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} | ||
| POSTHOG_API_KEY: ${{ secrets.POSTHOG_API_TOKEN }} | ||
| GH_TOKEN: ${{ steps.app-token.outputs.token }} | ||
| run: | | ||
| uv run tools/pr-approval-agent/review_pr.py \ | ||
| ${{ github.event.pull_request.number }} \ | ||
| --repo ${{ github.repository }} \ | ||
| --output-json /tmp/review.json | ||
|
|
||
| - name: Post review | ||
| if: always() | ||
| env: | ||
| # Use GITHUB_TOKEN for approvals so github-actions[bot] is the | ||
| # reviewer — its approvals count toward branch protection rules, | ||
| # unlike GitHub App bot approvals which show author_association NONE. | ||
| GH_TOKEN_APPROVE: ${{ github.token }} | ||
| GH_TOKEN: ${{ steps.app-token.outputs.token }} | ||
| run: | | ||
| PR=${{ github.event.pull_request.number }} | ||
| REPO=${{ github.repository }} | ||
| VERDICT=$(jq -r '.final_verdict // ""' /tmp/review.json 2>/dev/null || echo "") | ||
| REASONING=$(jq -r '.reviewer.reasoning // ""' /tmp/review.json 2>/dev/null || echo "") | ||
| REVIEWED_SHA=$(jq -r '.head_sha // ""' /tmp/review.json 2>/dev/null || echo "") | ||
|
|
||
| # Lock the review to the sha the LLM actually saw — `gh pr | ||
| # review` records against the head at API-call time, which | ||
| # drifts mid-LLM-roundtrip if the author force-pushes. | ||
| SHA_ARGS=() | ||
| if [ -n "$REVIEWED_SHA" ]; then | ||
| SHA_ARGS=(-f "commit_id=$REVIEWED_SHA") | ||
| fi | ||
|
|
||
| if [ "$VERDICT" = "APPROVED" ]; then | ||
| GH_TOKEN="$GH_TOKEN_APPROVE" gh api \ | ||
| -X POST "repos/$REPO/pulls/$PR/reviews" \ | ||
| "${SHA_ARGS[@]}" \ | ||
| -f event=APPROVE \ | ||
| -f body="$REASONING" | ||
| elif [ -n "$REASONING" ]; then | ||
| gh api \ | ||
| -X POST "repos/$REPO/pulls/$PR/reviews" \ | ||
| "${SHA_ARGS[@]}" \ | ||
| -f event=COMMENT \ | ||
| -f body="$REASONING" | ||
| else | ||
| gh pr comment "$PR" \ | ||
| --body "Review agent failed — check the [workflow run](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}) and re-apply the label to retry." \ | ||
| --repo "$REPO" | ||
| fi | ||
|
|
||
| # Non-APPROVED verdict removes the label, breaking the | ||
| # auto-rerun loop until a human re-applies it after | ||
| # addressing the feedback. | ||
| if [ "$VERDICT" != "APPROVED" ]; then | ||
| gh pr edit "$PR" --remove-label Stamphog \ | ||
| --repo "$REPO" | ||
| fi | ||
|
|
||
| - name: Upload evidence | ||
| if: always() | ||
| uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0 | ||
| with: | ||
| name: review-${{ github.event.pull_request.number }} | ||
| path: /tmp/review.json | ||
| retention-days: 30 | ||
|
|
||
| # Defense-in-depth: main ruleset has dismiss_stale_reviews_on_push=false | ||
| # and require_last_push_approval=false, so a stale bot approval could | ||
| # otherwise inherit malicious commits. Two-step gate: decide-delta | ||
| # classifies the new commits since the last bot approval, dismiss only | ||
| # runs when the delta is non-trivial. Trivial deltas (test/docs/lockfile | ||
| # /generated paths and clean merges from the base branch) retain the | ||
| # prior approval — a comment on the PR records the reason. The stamphog | ||
| # label stays sticky across pushes; the review job's existing | ||
| # non-APPROVED label-strip is the auto-loop's escape hatch. | ||
| decide-delta: | ||
| if: >- | ||
| github.event.action == 'synchronize' | ||
| && !github.event.pull_request.draft | ||
| && contains(github.event.pull_request.labels.*.name, 'Stamphog') | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 5 | ||
| outputs: | ||
| dismiss_approval: ${{ steps.decide.outputs.dismiss_approval }} | ||
| run_review: ${{ steps.decide.outputs.run_review }} | ||
| reason: ${{ steps.decide.outputs.reason }} | ||
| last_approved_sha: ${{ steps.decide.outputs.last_approved_sha }} | ||
|
|
||
| steps: | ||
| - name: Get app token | ||
| id: app-token | ||
| uses: actions/create-github-app-token@1b10c78c7865c340bc4f6099eb2f838309f1e8c3 # v3.1.1 | ||
| with: | ||
| client-id: ${{ secrets.GH_APP_PR_APPROVAL_AGENT_APP_ID }} | ||
| private-key: ${{ secrets.GH_APP_PR_APPROVAL_AGENT_PRIVATE_KEY }} | ||
|
|
||
| - name: Checkout main (full history) | ||
| uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 | ||
| with: | ||
| token: ${{ steps.app-token.outputs.token }} | ||
| ref: main | ||
| filter: blob:none | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Fetch PR head | ||
| run: git fetch --filter=blob:none origin pull/${{ github.event.pull_request.number }}/head | ||
|
|
||
| - name: Install uv | ||
| uses: astral-sh/setup-uv@eac588ad8def6316056a12d4907a9d4d84ff7a3b # v7.3.0 | ||
| with: | ||
| version: '0.10.2' # pinned: unpinned setup-uv calls GH API on every job, exhausts rate limit | ||
| enable-cache: false | ||
|
|
||
| - name: Decide retain vs dismiss | ||
| id: decide | ||
| env: | ||
| GH_TOKEN: ${{ steps.app-token.outputs.token }} | ||
| REPO: ${{ github.repository }} | ||
| PR_NUMBER: ${{ github.event.pull_request.number }} | ||
| HEAD_SHA: ${{ github.event.pull_request.head.sha }} | ||
| BASE_REF: origin/${{ github.event.pull_request.base.ref }} | ||
| run: | | ||
| set -euo pipefail | ||
| decision=$(uv run tools/pr-approval-agent/dismiss_check.py) | ||
|
Gilbert09 marked this conversation as resolved.
|
||
| echo "$decision" | ||
| echo "dismiss_approval=$(echo "$decision" | jq -r .dismiss_approval)" >> "$GITHUB_OUTPUT" | ||
| echo "run_review=$(echo "$decision" | jq -r .run_review)" >> "$GITHUB_OUTPUT" | ||
| echo "reason=$(echo "$decision" | jq -r .reason)" >> "$GITHUB_OUTPUT" | ||
| echo "last_approved_sha=$(echo "$decision" | jq -r '.last_approved_sha // ""')" >> "$GITHUB_OUTPUT" | ||
|
|
||
| # Only post the comment on actual retention reasons — not on | ||
| # no_prior_approval (nothing to retain) or empty_delta (HEAD | ||
| # didn't move, comment would be noise). | ||
| - name: Note retained approval | ||
| if: contains(fromJSON('["trivial_paths", "merge_only", "mixed_trivial"]'), steps.decide.outputs.reason) | ||
| env: | ||
| GH_TOKEN: ${{ steps.app-token.outputs.token }} | ||
| PR: ${{ github.event.pull_request.number }} | ||
| REPO: ${{ github.repository }} | ||
| REASON: ${{ steps.decide.outputs.reason }} | ||
| run: | | ||
| gh pr comment "$PR" --repo "$REPO" \ | ||
| --body "Retaining stamphog approval — delta since last review classified as \`$REASON\`." | ||
|
|
||
| dismiss: | ||
| needs: decide-delta | ||
| # Fail closed on three cases: | ||
| # - decide-delta said dismiss (smart path) | ||
| # - decide-delta failed (uv install / checkout / fetch timeout) | ||
| # - decide-delta was skipped (label removed out-of-band) — mirrors | ||
| # the pre-PR unconditional dismiss-on-push behavior so a stale | ||
| # bot approval can't outlive the label under main ruleset's | ||
| # dismiss_stale_reviews_on_push=false / require_last_push_approval=false | ||
| # Explicit synchronize + draft gates stop spurious dismissal on | ||
| # labeled / ready_for_review events where decide-delta's result is | ||
| # also 'skipped'. | ||
| if: >- | ||
| always() | ||
| && github.event.action == 'synchronize' | ||
| && !github.event.pull_request.draft | ||
| && ( | ||
| needs.decide-delta.outputs.dismiss_approval == 'true' | ||
| || needs.decide-delta.result == 'failure' | ||
| || needs.decide-delta.result == 'skipped' | ||
| ) | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 5 | ||
|
|
||
| steps: | ||
| - name: Dismiss stale bot approvals | ||
| env: | ||
| # Same identity (github-actions[bot]) that posted the approval. | ||
| GH_TOKEN: ${{ github.token }} | ||
| PR: ${{ github.event.pull_request.number }} | ||
| REPO: ${{ github.repository }} | ||
| REASON: ${{ needs.decide-delta.outputs.reason || (needs.decide-delta.result == 'skipped' && 'label_absent') || 'decide_delta_failed' }} | ||
| run: | | ||
| set -euo pipefail | ||
|
|
||
| # Only dismiss APPROVED reviews made by github-actions[bot] — | ||
| # human reviews and non-approval reviews are untouched. | ||
| mapfile -t REVIEW_IDS < <( | ||
| gh api "repos/$REPO/pulls/$PR/reviews" --paginate \ | ||
| --jq '.[] | select(.user.login == "github-actions[bot]" and .state == "APPROVED") | .id' | ||
| ) | ||
|
|
||
| for id in "${REVIEW_IDS[@]}"; do | ||
| [ -z "$id" ] && continue | ||
| gh api -X PUT "repos/$REPO/pulls/$PR/reviews/$id/dismissals" \ | ||
| -f message="New commits pushed (delta classified \`$REASON\`) — stamphog approval dismissed; re-review running automatically." \ | ||
| -f event=DISMISS | ||
| done | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.