Replace persona-based PR review with unified Claude orchestrator#310
Conversation
Replace the per-persona curl pipeline (pr-review.yml + ai-persona-review.yml) with a single claude-review.yml built on anthropics/claude-code-action. An orchestrator (.github/claude-review-prompt.md) fans out to one isolated subagent per review lens (.claude/agents/review-*.md), then consolidates all findings into ONE sticky comment plus inline comments for important findings only. This ends the churn of a fresh top-level comment per persona on every push, and gives each lens a clean, uncluttered context. Cadence and cost controls: - Auto-review once when a PR opens non-draft or leaves draft; re-review on demand via an @claude comment. No per-push reviews. - concurrency group cancels an in-flight review when a new push/mention lands. - [skip-review] in the PR title (or a skip-review label) opts a PR out of both the Claude review and CodeRabbit; @claude still works as a manual override. - Reuses the ambient GITHUB_TOKEN and existing ANTHROPIC_API_KEY secret. Also quiets CodeRabbit (quiet profile, collapsed walkthrough, incremental reviews, same skip-review override) and removes the orphaned pr-review.py. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017RS1pjrpm7c5AdirmmAtwX
|
Warning Review limit reached
Next review available in: 50 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (3)
WalkthroughReplaces the previous AI review workflows with a Claude-based workflow, specialized review lenses, orchestration rules, sticky-comment updates, re-review handling, and updated CodeRabbit and repository maintenance configuration. ChangesClaude Review Migration
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant PullRequest
participant ClaudeReviewWorkflow
participant ClaudeCodeAction
participant ReviewOrchestrator
participant ReviewLenses
participant StickyComment
PullRequest->>ClaudeReviewWorkflow: open or update eligible PR
ClaudeReviewWorkflow->>ClaudeCodeAction: start review
ClaudeCodeAction->>ReviewOrchestrator: load orchestration protocol
ReviewOrchestrator->>ReviewLenses: route changed paths
ReviewLenses-->>ReviewOrchestrator: return verified findings
ReviewOrchestrator->>StickyComment: post consolidated results
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Claude finished @DiamondDagger590's task in 5m 22s —— View job Review: 2 important, 1 nitThis PR touches only workflow/config/doc files (
Important findings1. 2. Nit
Inline comments posted for both Important findings above. |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| - uses: actions/checkout@v4 | ||
| with: | ||
| fetch-depth: 0 | ||
| - uses: anthropics/claude-code-action@v1 | ||
| with: | ||
| anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }} | ||
| # Reuse the ambient token (as the old pipeline did); comments post as | ||
| # github-actions[bot]. No Claude GitHub App install required. | ||
| github_token: ${{ secrets.GITHUB_TOKEN }} | ||
| # REQUIRED alongside a custom prompt, or the sticky comment silently | ||
| # never posts (anthropics/claude-code-action#1108). | ||
| track_progress: true | ||
| use_sticky_comment: true | ||
| prompt: | | ||
| REPO: ${{ github.repository }} | ||
| PR NUMBER: ${{ github.event.pull_request.number }} | ||
|
|
||
| This is the initial automated review of this PR. Read | ||
| .github/claude-review-prompt.md in this checkout and execute the | ||
| orchestrated review protocol it defines. |
There was a problem hiding this comment.
Security regression vs. the deleted ai-persona-review.yml: that workflow's own header comment stated its entire security model — "this workflow always executes on the DEFAULT BRANCH and never checks out PR-head code, so secrets are safe" — and it enforced this by sparse-checking-out only .claude/commands/ from github.event.repository.default_branch, specifically because a PR could otherwise modify the persona prompt files and have malicious instructions executed with secrets in scope.
auto-review here does the opposite: it checks out the PR's own head (fetch-depth: 0, default pull_request ref) and then instructs Claude to "Read .github/claude-review-prompt.md in this checkout and execute the orchestrated review protocol it defines." Both that file and .claude/agents/review-*.md are themselves part of the PR diff — so a PR that edits them rewrites its own reviewer's instructions before they run, with ANTHROPIC_API_KEY and a pull-requests: write, issues: write token in scope. This very PR is a live example: it modifies exactly those files, and this run read and executed them from this PR's own checkout.
The head.repo.full_name == github.repository guard only rules out forks — it doesn't protect against a same-repo branch (e.g. any contributor/bot with push access) shipping a poisoned orchestrator prompt. Worth either restoring the old "read prompt/agent files from the default branch, diff/code from the PR" split, or explicitly deciding this tradeoff is acceptable and dropping the now-stale security rationale from history without a replacement mitigation.
| ## 3. Fan out (one subagent per lens, in parallel) | ||
|
|
||
| Spawn every applicable lens **in a single message with multiple Task calls** so they run concurrently. For each, use the matching `subagent_type` (e.g. `review-security`) and pass in the prompt: | ||
|
|
||
| - The PR diff (or, if very large, the changed-file list plus the diff hunks relevant to that lens). | ||
| - A one-line instruction to follow its own agent definition. | ||
| - On a re-review only: the list of findings previously reported for that lens, labeled "PREVIOUSLY REPORTED — re-verify each: still open, or resolved?". | ||
|
|
||
| Each subagent returns either `CLEAN` or a list of `SEVERITY/LENS/FILE/WHAT/WHY/FIX` blocks. Collect them all. |
There was a problem hiding this comment.
Unverified in this live run: §3 assumes .claude/agents/review-*.md register as Task subagent types reachable via subagent_type: review-security etc. In the session executing this exact review, only the default built-in types were available (claude, claude-code-guide, Explore, general-purpose, Plan, statusline-setup) — none of the five new lens names. The five new agent files were also absent from the working tree at session start (present in the commit, but deleted/uncommitted locally), which is consistent with whatever produced this either being a genuine checkout gap or a deliberate stripping of PR-introduced agent definitions before granting them subagent-type status (plausible given the security concern on the checkout step above).
Either way, please confirm end-to-end (a real pull_request: opened trigger, not a local dry run) that Task(subagent_type: "review-security") actually resolves to the specialized agent and doesn't silently fall back to a generic one — that failure mode wouldn't error, it would just quietly collapse "isolated per-lens context" into "one generic pass," undermining the stated benefit of this design without anyone noticing.
Gate the on-demand @claude job to OWNER/MEMBER/COLLABORATOR only, preventing untrusted commenters from triggering reviews on fork PRs. Change skip label from skip-review to skip-claude-review for more targeted control (CodeRabbit keeps its own skip-review label). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017RS1pjrpm7c5AdirmmAtwX
- Add fork-check pre-step to on-demand job that rejects @claude mentions on fork PRs (prevents attacker-controlled checkout with secrets in scope) - SHA-pin actions/checkout to 11bd7190 (v4.2.2), matching McCore - SHA-pin anthropics/claude-code-action to fffa3483 (v1) in both jobs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017RS1pjrpm7c5AdirmmAtwX
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017RS1pjrpm7c5AdirmmAtwX
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.claude/agents/review-server-owner.md:
- Around line 15-16: Update the review scope statement in the server-owner
instructions to include src/**/configuration/** alongside configuration/**,
matching the orchestrator’s routing patterns. Preserve the restriction to
reviewing only changed lines in the covered configuration files.
In @.claude/agents/review-testing.md:
- Line 4: Remove Bash from the tools list in the testing agent configuration,
leaving only Read, Grep, and Glob. Keep the workflow’s restricted --allowedTools
behavior unchanged and ensure the testing lens does not request or rely on shell
execution.
In @.github/workflows/claude-review.yml:
- Around line 83-85: Update the actions/checkout step in the issue_comment
workflow to explicitly check out the pull request’s head ref rather than the
default branch. Use the event’s PR head repository and ref values while
preserving the existing full fetch-depth configuration, so on-demand reviews
analyze the submitted PR code.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 46e97165-937f-4b31-bfc0-51e916c8fe5c
📒 Files selected for processing (12)
.claude/agents/review-extensibility.md.claude/agents/review-gui-ux.md.claude/agents/review-security.md.claude/agents/review-server-owner.md.claude/agents/review-testing.md.coderabbit.yaml.github/claude-review-prompt.md.github/scripts/pr-review.py.github/workflows/ai-persona-review.yml.github/workflows/claude-review.yml.github/workflows/pr-review.ymlCLAUDE.md
💤 Files with no reviewable changes (3)
- .github/workflows/ai-persona-review.yml
- .github/workflows/pr-review.yml
- .github/scripts/pr-review.py
- Checkout PR head ref for on-demand reviews so Read/Grep/Glob see the PR code instead of the default branch (issue_comment events default to the base branch checkout) - Remove Bash from testing agent tools (only needs Read/Grep/Glob) - Fix server-owner scope to match routing table (src/**/configuration/**) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017RS1pjrpm7c5AdirmmAtwX
2 participants
Summary
Consolidates the multi-workflow AI PR review system (pr-review.yml + ai-persona-review.yml + pr-review.py) into a single unified workflow (claude-review.yml) that uses Anthropic's
claude-code-actionto orchestrate specialized review lenses. Eliminates the Python script and replaces it with a declarative agent-based architecture where each review concern (security, testing, extensibility, GUI/UX, server config) runs as an isolated subagent, then a central orchestrator merges findings into one sticky comment.Key Changes
Removed legacy workflows:
.github/workflows/pr-review.yml— two-job pipeline (detect-changes → prepare-review).github/workflows/ai-persona-review.yml— secret-holding workflow_run job with per-persona comment posting.github/scripts/pr-review.py— Python orchestrator with API retry logic, diff sanitization, and persona routingAdded new unified workflow:
.github/workflows/claude-review.yml— singleclaude-code-actionjob with auto-review on PR open/ready and on-demand@claudemention support[skip-review]title keyword or labelAdded orchestration protocol:
.github/claude-review-prompt.md— defines the review orchestrator's responsibilities: gather context, route to lenses, fan out to subagents, consolidate findings, post one sticky comment with inline comments for Important findings onlyAdded specialized review agents:
.claude/agents/review-security.md— injection, permission bypass, SQL/DDL injection.claude/agents/review-testing.md— coverage gaps, test structure, TimeProvider usage.claude/agents/review-extensibility.md— public API stability, events, @NotNull/@nullable, backward compatibility.claude/agents/review-gui-ux.md— inventory ergonomics, navigation, click-hints, localization.claude/agents/review-server-owner.md— YAML readability, defaults, reload-safety, config-version needsEach agent applies its corresponding checklist from
.claude/commands/review-*.md(unchanged) and returns structured findings in a consistent format for the orchestrator to consolidate.Updated CodeRabbit config:
auto_incremental_reviewfor incremental feedback on pushes[skip-review]title keyword andskip-reviewlabel support to opt out of auto-reviewUpdated CLAUDE.md:
Notable Implementation Details
@claudemention also use the sticky comment.@claudemention is made on the same PR.SEVERITY/LENS/FILE/WHAT/WHY/FIXformat, which the orchestrator parses, deduplicates, ranks, and caps before posting.https://claude.ai/code/session_017RS1pjrpm7c5AdirmmAtwX
Summary by CodeRabbit
New Features
Changes