Support multi-PR reviews per worker session

[neversettle17-101]

Migrated from aoagents/ReverbCode/issues/395

Source: aoagents/ReverbCode#395
Original assignees: neversettle17-101

---

Summary

Extend AO reviews so one worker session can have multiple PRs reviewed by a single shared reviewer terminal. Keep review session-level because it represents the reviewer terminal/handle for the worker session. Store PR-specific review state in review_run.

Split the work into two PRs:

1. Backend/storage/API/CLI support for multi-PR review execution.
2. Frontend UI for showing eligible and up-to-date PR review state.

PR 1: Backend Multi-PR Review Support

• Keep review as one row per worker session:
  • session_id remains unique.
  • reviewer_handle_id remains the single reviewer terminal handle.
  • review.pr_url should no longer drive behavior; leave it as legacy/deprecated or stop updating it.
• Make review_run the PR-scoped record:
  • Always persist pr_url.
  • Add a migration replacing uniqueness with (session_id, pr_url, target_sha) where target_sha != "" AND status != "failed".
  • Rename store lookup from session/SHA to session/PR/SHA.
• Add a pure planner in backend/internal/review:
  • Output per-PR items with prUrl, prNumber, targetSha, status, latestRun.
  • Status values: needs_review, running, up_to_date, changes_requested, ineligible.
  • Eligible PRs are open, non-draft, and not already AO-approved for the current pr_url + head_sha.
• Update trigger flow:
  • Trigger(sessionID) evaluates all worker PRs.
  • Create runs for needs_review and current-head changes_requested.
  • Reuse/skip running and up_to_date.
  • Spawn one reviewer terminal if needed, then send one review task per created run to that same terminal.
  • Return batch results with reviewer handle, items, and created runs.
• Update submit flow:
  • Keep ao review submit --session --run ....
  • runID identifies the PR through stored review_run.pr_url.
  • Worker changes-requested messages include the PR URL and GitHub review id.
• Add API fields in an additive way:
  • ListReviewsResponse.items.
  • TriggerReviewResponse.items.
  • TriggerReviewResponse.createdReviews.
  • Keep reviewerHandleId and reviews history for compatibility.
• Regenerate OpenAPI and frontend schema with npm run api.

PR 2: Reviews UI

• Update the Reviews tab to consume items.
• Show PR review rows/cards with status:
  • needs_review
  • running
  • changes_requested
  • up_to_date
  • optionally muted ineligible
• Use one session-level action: Run needed reviews.
• Keep one Open terminal action for the shared reviewer terminal.
• Do not create terminal controls per PR.

Test Plan

• Planner table tests for open, draft, merged, closed, approved-current-SHA, changes-requested-current-SHA, running-current-SHA, different SHA.
• Trigger creates multiple runs for multiple eligible PRs.
• Two PRs with the same head SHA do not collide.
• Approved current-head PR is skipped.
• Running current-head PR is reused.
• Failed current-head PR is retryable.
• Only one reviewer terminal is spawned per worker session.
• Submit includes PR URL in changes-requested worker notification.
• Storage test verifies unique index is (session_id, pr_url, target_sha).
• API tests verify items and batch trigger response.
• PR 2 frontend tests verify eligible/up-to-date display and one shared terminal.

Assumptions

• AO approval means a completed AO review_run with verdict=approved for the current pr_url + head_sha.
• External GitHub approval does not skip AO review unless explicitly added later.
• Draft, merged, and closed PRs are ineligible in v1.

[neversettle17-101]

Design note: review submission and worker notification batching

We discussed several ways to handle multiple PR reviews for one worker session.

Why not push review feedback to the worker one by one?

The main reason is worker context and action quality. A trigger can create reviews for several PRs owned by the same worker session. If AO sends each changes_requested result immediately as soon as that PR review finishes, the worker may start fixing PR A while the reviewer is still evaluating PR B/C. That creates a few problems:

• The worker can act on incomplete review context for the overall session.
• Multiple terminal messages can interrupt the worker repeatedly instead of giving one consolidated action set.
• If comments across PRs are related, the worker sees them fragmented.
• The shared reviewer terminal can still be mid-batch while the worker is already changing code, which makes the remaining review context stale more quickly.

So the chosen behavior is: review runs are still PR-scoped, but worker notification is batch-scoped. The reviewer submits each PR result independently, and AO sends one combined worker message only after every run in that trigger batch is terminal and at least one current-head run requested changes.

Options considered

Option A: PR-scoped submit, batch-scoped worker notification

Each trigger creates a batch_id; every review_run created by that trigger gets that same batch_id. The reviewer still runs one ao review submit --run <runID> per PR. After each submit, the daemon checks whether the whole batch is terminal. Once complete, it sends one combined worker message for the current-head changes_requested runs.

Pros:

• Keeps review runs aligned to the real domain: one GitHub PR review per PR/head SHA.
• Gives failure isolation: one failed PR review does not corrupt other PR review results.
• The daemon owns durable batch state, so restarts and retries are tractable.
• The worker receives one consolidated message instead of several interruptions.
• CLI/API surface stays simple: one submit records one run.

Cons:

• A run that remains running can block the batch indefinitely.
• Concurrent final submits need dedup/locking so the combined worker message is not sent twice.
• Send succeeds / delivered marking fails needs retry-safe dedup.
• Stale old-SHA results must be suppressed before worker notification.

Chosen for v1.

Option B: dedicated review_batch table

Create a first-class review_batch row with lifecycle fields like status, created_at, completed_at, delivered_at, and attach runs to it.

Pros:

• Cleaner durable lifecycle for completion/delivery.
• Easier to support timeout/manual flush later.
• Easier batch-level audit trail.

Cons:

• More schema and state machine surface than needed for v1.
• More store/API code before we know batch behavior needs to grow.

Deferred. The current review_run.batch_id approach can evolve into this later.

Option C: reviewer bulk-submit

The reviewer would submit all PR review results in one batch command/API call.

Pros:

• Batch completion is explicit.
• One submit maps naturally to one worker notification.

Cons:

• Bad failure isolation: one malformed or failed PR result can affect the entire batch.
• The reviewer has to hold all PR results correctly until the end.
• Partial GitHub posting/recording becomes awkward.
• The daemon still needs per-PR run state for audit, retry, and latest-status planning.

Rejected for now. GitHub reviews and AO review runs are naturally PR-scoped.

Option D: background queue/drainer

Each submit records one run and a background worker periodically drains complete batches.

Pros:

• Submit request remains small.
• Natural place for retries, timeouts, delayed flush, and future scheduling.

Cons:

• More moving parts.
• More complex deterministic tests.
• Delivery may be delayed after the final submit.
• AO does not currently have a durable generic job queue for this.

Rejected for v1 as overkill.

Option E: immediate per-run messages with worker-side batching

Each submit sends a worker message immediately, and the worker is expected to wait until all batch messages arrive before acting.

Pros:

• Simpler daemon state.

Cons:

• Pushes coordination burden to the worker agent.
• Not enforceable or durable.
• The worker may act too early.
• Violates the goal of one consolidated actionable review message.

Rejected.

Chosen v1 behavior

• Trigger(sessionID) creates one batch_id for all newly-created runs in that trigger.
• Reviewer receives one task per PR/run in one shared reviewer terminal.
• Reviewer submits one run at a time.
• Submit records the single run result, then drains that run's batch.
• Batch is complete when no run in the batch is still running.
• Failed/superseded runs are terminal and do not block.
• Worker notification is sent only if at least one deliverable run has verdict = changes_requested.
• Approved-only batches send no worker message.
• Delivery includes only current-head changes_requested runs.
• Delivery is protected by per-batch drain locking, delivered_at, and deterministic send dedup.