diff --git a/.github/workflows/agents/pr-review-feedback.yaml b/.github/workflows/agents/pr-review-feedback.yaml new file mode 100644 index 000000000..b1efc27f4 --- /dev/null +++ b/.github/workflows/agents/pr-review-feedback.yaml @@ -0,0 +1,26 @@ +models: + haiku: + provider: anthropic + model: claude-haiku-4-20250514 + max_tokens: 1024 + +agents: + root: + model: haiku + description: Learns from developer feedback on reviews + instruction: | + A developer replied to one of your review comments. Learn from their feedback. + + If they're correcting you, remember what you got wrong to avoid repeating it. + If they're adding context, remember it for future reviews. + + Use `add_memory` to store what you learned, then react with 👍 to acknowledge. + + toolsets: + - type: memory + path: .github/pr-review-memory.db + - type: shell + +permissions: + allow: + - shell:cmd=gh api */reactions* diff --git a/.github/workflows/agents/pr-review.yaml b/.github/workflows/agents/pr-review.yaml index 83335fc47..224cd6791 100644 --- a/.github/workflows/agents/pr-review.yaml +++ b/.github/workflows/agents/pr-review.yaml @@ -1,58 +1,120 @@ +models: + sonnet: + provider: anthropic + model: claude-sonnet-4-5 + max_tokens: 8192 + agents: root: - model: anthropic/claude-sonnet-4-5 + model: sonnet + description: PR Review Orchestrator instruction: | - You are a code reviewer. Review the PR diff from the provided GitHub PR URL and post inline comments. - The user's message contains a GitHub Pull Request URL (e.g., https://github.com/owner/repo/pull/123). - - Steps: - 1. Use the shell to call "gh" to get all of the information about the Pr - 2. Review the PR diff, focusing on lines that were ADDED (+) or MODIFIED. - 3. Make sure to get the overall picture about the changes, read the files from the current directory as needed - 4. Review the code changes in detail - 5. Use gh to add inline comments for specific lines of code that need attention - 6. Post the review to GitHub using gh CLI: - a. Create a JSON payload for the review with inline comments - b. Use shell tool to execute: - ``` - echo '{"body":"OVERALL_SUMMARY","event":"COMMENT","comments":[{"path":"FILE","line":LINE,"body":"COMMENT"},...]}' | \ - gh api repos/{owner}/{repo}/pulls/{pr}/reviews --input - - ``` - c. Map your verdict to event: "APPROVE", "REQUEST_CHANGES", or "COMMENT" - - ## Review Focus - **Code Quality:** Readability, naming, structure, DRY - **Correctness:** Logic errors, edge cases, error handling, type safety - **Security:** Input validation, SQL/XSS vulnerabilities, hardcoded secrets - **Performance:** Inefficient algorithms, unnecessary operations, memory leaks - **Best Practices:** Framework conventions, testing, documentation, accessibility - - # Go Specialization to Add + You coordinate PR reviews using specialized sub-agents. + + ## Process + + 1. Get the PR diff with `gh pr diff` + 2. Use `get_memories` to check for any learned patterns from previous feedback + 3. Delegate to `drafter` with the diff and any relevant learned patterns + 4. For each hypothesis, delegate to `verifier` to confirm or dismiss it + 5. Post your review with `gh api` - only report confirmed/likely issues + + Find **real bugs**, not style issues. If it works correctly, approve it. + + End every comment with `` for feedback tracking. + + ## Posting Reviews + + Use this format to post reviews with inline comments: + ```bash + echo '{"body":"OVERALL_SUMMARY","event":"EVENT","comments":[{"path":"FILE","line":LINE,"body":"COMMENT "},...]}' | \ + gh api repos/{owner}/{repo}/pulls/{pr}/reviews --input - + ``` + + Map your verdict to event: "APPROVE", "REQUEST_CHANGES", or "COMMENT" + + sub_agents: + - drafter + - verifier - ## Focus Areas (for `+` lines only) - - **Correctness:** Control flow, edge cases, nil checks - - **Idiomatic Go:** Conventions, stdlib patterns - - **Error Handling:** Proper wrapping (fmt.Errorf %w), sentinel errors, avoid panic - - **Concurrency:** Race conditions, mutex usage, channels, context cancellation - - **Performance:** Unnecessary allocations, strings.Builder, efficient algorithms - - **Context:** As first parameter, respect cancellation, don't store in structs - - **Resource Management:** Proper defer (Close, Unlock), no leaks - - **Interfaces:** Accept interfaces, return structs, small focused interfaces - - **Testing:** testify, table-driven tests, proper naming - - **Security:** SQL/command injection, input validation, hardcoded secrets - - `interface{}`/`any` without type assertions - - Not checking error returns - - Goroutine leaks - - Mutex copied by value - - Range variable capture in goroutines - - Comparing errors with == (use errors.Is/As) - - **Be constructive, concise, specific, respectful.** toolsets: - type: filesystem tools: [read_file, read_multiple_files, list_directory, directory_tree] - type: shell + - type: memory + path: .github/pr-review-memory.db + - type: think + + drafter: + model: sonnet + description: Bug Hypothesis Generator + instruction: | + Analyze the provided PR diff and generate specific bug hypotheses. + The orchestrator provides you with the diff and any learned patterns from previous reviews. + + ## Focus Areas (for `+` lines only) + + **General:** + - Logic errors, edge cases, off-by-one errors + - Nil/null pointer dereferences + - Resource leaks (files, connections, memory) + - Security issues (injection, validation, hardcoded secrets) + + **Go-Specific:** + - **Error Handling:** Missing error checks, improper wrapping (use `fmt.Errorf %w`), comparing with `==` instead of `errors.Is/As` + - **Concurrency:** Race conditions, mutex usage, channel deadlocks, context cancellation ignored + - **Context:** Not as first parameter, stored in structs, cancellation not respected + - **Resource Management:** Missing defer for Close/Unlock, deferred in loops + - **Interfaces:** `interface{}`/`any` without type assertions + - **Goroutines:** Leaks, range variable capture in closures, mutex copied by value + + ## Common Go Anti-Patterns to Flag + + - `interface{}`/`any` used without type assertions + - Error return values ignored (unchecked `err`) + - Goroutine leaks (no way to stop/cancel) + - Mutex copied by value (passed to function without pointer) + - Range variable captured in goroutine closure + - `err == ErrSomething` instead of `errors.Is(err, ErrSomething)` + - Context not passed through call chain + - Panic in library code (should return error) + + ## Ignore + + Style, formatting, naming, documentation, test files. + + ## Output + + For each potential bug, describe: + 1. **File and line** where the issue is + 2. **What** could go wrong + 3. **How** it could be triggered + 4. **Severity** (high/medium/low) + + toolsets: + - type: filesystem + tools: [read_file, read_multiple_files, list_directory, directory_tree] + - type: think + + verifier: + model: sonnet + description: Hypothesis Verifier + instruction: | + Verify a specific bug hypothesis by reading the full file context. + + Your job is to filter out false positives. Check if: + - The bug can actually happen given the surrounding code + - Existing safeguards already prevent it + - Tests cover this case + + Return CONFIRMED (definitely a bug), LIKELY (probably a bug), or DISMISSED (not a bug). + + toolsets: + - type: filesystem + tools: [read_file, read_multiple_files, list_directory, directory_tree] + - type: think permissions: allow: - - shell:cmd=gh * + - shell:cmd=gh * + - shell:cmd=git * diff --git a/.github/workflows/pr-review.yml b/.github/workflows/pr-review.yml index 312fade6a..36aad67d0 100644 --- a/.github/workflows/pr-review.yml +++ b/.github/workflows/pr-review.yml @@ -3,6 +3,8 @@ name: PR Review on Command on: issue_comment: types: [created] + pull_request_review_comment: + types: [created] permissions: contents: read @@ -10,19 +12,202 @@ permissions: issues: write jobs: + # ========================================================================== + # LEARN FROM FEEDBACK - Process replies to agent review comments + # ========================================================================== + learn-from-feedback: + # Trigger when someone replies to a review comment that contains our marker + if: > + github.event_name == 'pull_request_review_comment' && + github.event.comment.in_reply_to_id != null + runs-on: ubuntu-latest + steps: + - name: Check if reply is to agent comment + id: check + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + # Get the parent comment we're replying to + parent_id=${{ github.event.comment.in_reply_to_id }} + + parent=$(gh api repos/${{ github.repository }}/pulls/comments/$parent_id 2>/dev/null || echo "{}") + parent_body=$(echo "$parent" | jq -r '.body // ""') + + # Check if parent comment was from cagent (contains our marker) + if echo "$parent_body" | grep -q ""; then + echo "is_agent_comment=true" >> $GITHUB_OUTPUT + echo "Found reply to agent comment" + + # Extract PR number from the comment's pull_request_url + pr_url=$(echo "$parent" | jq -r '.pull_request_url // ""') + pr_number=$(echo "$pr_url" | grep -oE '[0-9]+$' || echo "${{ github.event.pull_request.number }}") + echo "pr_number=$pr_number" >> $GITHUB_OUTPUT + else + echo "is_agent_comment=false" >> $GITHUB_OUTPUT + echo "Not a reply to agent comment, skipping" + fi + + - name: Checkout repository + if: steps.check.outputs.is_agent_comment == 'true' + uses: actions/checkout@v4 + + - name: Restore reviewer memory database + if: steps.check.outputs.is_agent_comment == 'true' + uses: actions/cache@v4 + with: + path: .github/pr-review-memory.db + key: pr-review-memory-${{ github.repository }} + restore-keys: | + pr-review-memory-${{ github.repository }} + + - name: Process feedback and update memory + if: steps.check.outputs.is_agent_comment == 'true' + uses: docker/cagent-action@7992ea47ed2446b5a80b01c46b00037c680e35ea + with: + agent: ${{ github.workspace }}/.github/workflows/agents/pr-review-feedback.yaml + prompt: | + A developer replied to one of your previous review comments with feedback. + + **File:** ${{ github.event.comment.path }} + **Line:** ${{ github.event.comment.line }} + **Their feedback:** ${{ github.event.comment.body }} + **PR:** #${{ steps.check.outputs.pr_number }} + + Analyze this feedback: + 1. If they're correcting a false positive, add a memory to avoid this mistake + 2. If they're asking for clarification, note what was unclear + 3. If they're agreeing and adding context, store the additional insight + + Use add_memory to record what you learned. Format: + "FEEDBACK: [category] - [what you learned] - Source: PR #${{ steps.check.outputs.pr_number }}" + + Then react to their comment with 👍 to acknowledge. + anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }} + github-token: ${{ secrets.GITHUB_TOKEN }} + + - name: Save reviewer memory database + if: steps.check.outputs.is_agent_comment == 'true' + uses: actions/cache/save@v4 + with: + path: .github/pr-review-memory.db + key: pr-review-memory-${{ github.repository }} + + # ========================================================================== + # MAIN REVIEW PIPELINE + # ========================================================================== run-review: if: github.event.issue.pull_request && contains(github.event.comment.body, '/review') runs-on: ubuntu-latest + steps: - name: Checkout repository uses: actions/checkout@v4 + with: + fetch-depth: 0 - - name: Run PR Reviewer Agent - uses: docker/cagent-action@1f7ec0445e138a587639fc9c046076e22d184349 + - name: Add reaction to acknowledge + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + gh api repos/${{ github.repository }}/issues/comments/${{ github.event.comment.id }}/reactions \ + -X POST -f content='eyes' + + - name: Get PR information + id: pr-info + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + PR_NUMBER=${{ github.event.issue.number }} + echo "pr_number=$PR_NUMBER" >> $GITHUB_OUTPUT + + # Get changed files + gh pr view $PR_NUMBER --json files -q '.files[].path' > changed_files.txt + echo "Changed files:" + cat changed_files.txt + + # Get PR metadata + gh pr view $PR_NUMBER --json title,body,author,baseRefName,headRefName > pr_metadata.json + + - name: Restore reviewer memory database + uses: actions/cache@v4 + with: + path: .github/pr-review-memory.db + key: pr-review-memory-${{ github.repository }} + restore-keys: | + pr-review-memory-${{ github.repository }} + + - name: Build review context + id: context + run: | + PR_NUMBER=${{ github.event.issue.number }} + title=$(jq -r '.title' pr_metadata.json) + author=$(jq -r '.author.login' pr_metadata.json) + body=$(jq -r '.body // "No description provided."' pr_metadata.json) + base=$(jq -r '.baseRefName' pr_metadata.json) + head=$(jq -r '.headRefName' pr_metadata.json) + + cat > review_context.md << EOF + # Pull Request Review Request + + ## PR Information + - **URL**: https://github.com/${{ github.repository }}/pull/$PR_NUMBER + - **Title**: $title + - **Author**: $author + - **Branch**: $head → $base + - **Files Changed**: $(wc -l < changed_files.txt | tr -d ' ') + + ## PR Description + $body + + ## Changed Files + + EOF + + cat changed_files.txt >> review_context.md + + cat >> review_context.md << 'EOF' + + --- + + ## Instructions + + Execute the review pipeline: + + 1. **Gather**: Use `gh pr diff` to get the full diff + 2. **Draft**: Delegate to `drafter` agent to generate bug hypotheses + 3. **Verify**: For each hypothesis, delegate to `verifier` agent + 4. **Post**: Aggregate findings and post review via `gh api` + + Only report CONFIRMED and LIKELY findings. Approve if no issues found. + EOF + + echo "Context built:" + wc -l review_context.md + + - name: Run PR Review Team + uses: docker/cagent-action@7992ea47ed2446b5a80b01c46b00037c680e35ea with: - cagent-version: "v1.19.1" agent: ${{ github.workspace }}/.github/workflows/agents/pr-review.yaml - prompt: "Please review this GitHub Pull Request: https://github.com/${{ github.repository }}/pull/${{ github.event.issue.number }}" - mcp-gateway: true + prompt-file: review_context.md anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }} - github-token: ${{ secrets.GITHUB_TOKEN }} \ No newline at end of file + github-token: ${{ secrets.GITHUB_TOKEN }} + + - name: Save reviewer memory database + if: always() + uses: actions/cache/save@v4 + with: + path: .github/pr-review-memory.db + key: pr-review-memory-${{ github.repository }} + + - name: Add completion reaction + if: always() + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + if [ "${{ job.status }}" == "success" ]; then + gh api repos/${{ github.repository }}/issues/comments/${{ github.event.comment.id }}/reactions \ + -X POST -f content='rocket' + else + gh api repos/${{ github.repository }}/issues/comments/${{ github.event.comment.id }}/reactions \ + -X POST -f content='confused' + fi