[safe-output-health] Safe Output Health Report - 2026-04-14

[github-actions[bot]]

Executive Summary

• Period: 2026-04-14 (last 24 hours)
• Runs Analyzed: 50 (across 31 unique workflows)
• Runs With Safe Output Processing: 5
• Total Safe Output Items Executed: 30
• Hard Failures: 1
• Warnings (non-fatal): 1
• Overall Safe Output Success Rate: 95.7% (22/23 attempted messages)
• Overall Health: 🟡 Borderline Healthy

Improvement vs. 2026-04-13: No API rate limit burst today (was 3 hard failures yesterday). Hard failure count dropped from 3 → 1. Trend is improving.

---

Safe Output Job Statistics

Job Type	Executions	Failures	Skipped	Success Rate
add_comment	7	0	0	100%
create_discussion	5	0	0	100%
submit_pull_request_review	3	0	0 (1 warn)	100%
assign_to_agent	3	0	0	100%
create_issue	2	0	0	100%
create_pull_request_review_comment	2	0	2	100%
resolve_pull_request_review_thread	0	1	0	0%
upload_artifact	1	0	0	100%
dispatch_workflow	1	0	0	100%
set_issue_type	1	0	0	100%
missing_tool	1	0	0	100%
noop	1	0	0	100%
other types	5	0	0	100%
Total	37	1	2	97.3%

---

Error Clusters

Cluster 1: PR Context Required for Thread Resolution

• Count: 1 occurrence
• Affected Jobs: resolve_pull_request_review_thread
• Affected Workflow: Smoke Claude (§24399262395)
• Sample Error:

  ✗ Message 9 (resolve_pull_request_review_thread) failed: Cannot resolve review threads outside of a pull request context
  

• Root Cause: Smoke Claude is triggered by schedule (not a PR event). The safe output handler requires a PR event context to process resolve_pull_request_review_thread, even when an explicit thread_id (PRRT_kwDOPc1QR856zwGl) is provided.
• Pattern History: Previously on 2026-04-10, the same smoke test failed with Resource not accessible by integration (permissions error). Today the error is a context check — suggesting a context guard was added to the handler, but it's too strict and rejects explicit thread_id calls.
• Impact: 1/11 messages failed in Smoke Claude; remaining 10 messages (create_issue, add_comment, update_pull_request, create_code_scanning_alert, PR review comments, submit review, add_reviewer, push_to_branch) all succeeded.

---

Warnings (Non-Fatal)

Smoke Copilot: submit_pull_request_review — No Review Context

• Run: Smoke Copilot §24399977092
• Warning: No review context set - cannot submit review
• Effect: Treated as a warning and skipped (not counted as failure). Also: create_pull_request_review_comment (×2) and reply_to_pull_request_review_comment skipped — not in PR event context.
• Overall Result: 9/13 messages succeeded, 0 failed, 3 skipped. ✅
• Recurring: Also observed 2026-04-13.

---

Agent Failures Affecting Safe Outputs (Out of Scope, Context Only)

These are not safe output job failures but caused safe output jobs to be skipped entirely:

Workflow	Run	Root Cause
Smoke Create Cross-Repo PR	§24400520199	Bad credentials — PAT for side-repo checkout expired/invalid
Smoke Update Cross-Repo PR	§24400213879	Bad credentials — same PAT issue
Smoke Gemini	§24399738673	API key not valid — invalid Gemini API key
AI Moderator	§24398686839	GitHub App rate limit during activation (lock issue step)

The cross-repo bad credentials issue is recurring (also seen 2026-04-07). PAT rotation needed.

---

Root Cause Analysis

PR Context Restriction in resolve_pull_request_review_thread

The Smoke Claude workflow runs on a schedule, not on a PR event. The safe output handler for resolve_pull_request_review_thread checks for a live PR event context before processing. Even when the agent provides an explicit thread_id, the context guard fires and rejects the call.

This behavior is overly strict: if a thread_id is explicitly provided, the handler could proceed without requiring a PR event context — it has all the information needed to make the API call. The guard is valuable for cases where no thread_id is available (preventing accidental use), but should not block explicit thread_id calls.

No Rate Limit Burst Today

Unlike 2026-04-13 (and 2026-04-02, 2026-04-07), there was no concurrent API rate limit burst at noon UTC today. This is likely because the daily workflow batch was smaller or more spread out.

---

Recommendations

Critical Issues (Immediate Action)

None today. The single hard failure is a smoke test context issue that does not affect production workflows.

Bug Fixes

1. Allow resolve_pull_request_review_thread with explicit thread_id outside PR event context
   • File: safe_output_handler_manager.cjs (or the resolve_pull_request_review_thread handler)
   • Problem: Handler rejects the call when event_name != 'pull_request', even if an explicit thread_id is in the agent output
   • Fix: If thread_id is explicitly provided in the safe output item, bypass the event context check and proceed with the API call
   • Affected Workflows: Smoke Claude (and any other schedule-triggered workflow that needs to resolve review threads)

Process Improvements

1. Renew PAT for githubnext/gh-aw-side-repo (recurring since 2026-04-07)

   • Smoke Create Cross-Repo PR and Smoke Update Cross-Repo PR both fail with Bad credentials
   • Action: Rotate the GitHub PAT used for side-repo checkout

2. Validate Gemini API Key

   • Smoke Gemini fails with invalid API key
   • Action: Check if the GEMINI_API_KEY secret in the repository is valid and not expired

---

Work Item Plans

Work Item 1: Fix resolve_pull_request_review_thread Context Check

• Type: Bug Fix
• Priority: Medium
• Description: The resolve_pull_request_review_thread safe output handler rejects calls when the workflow was not triggered by a PR event, even when an explicit thread_id is provided. This causes Smoke Claude's scheduled test to fail.
• Acceptance Criteria:
  • resolve_pull_request_review_thread succeeds when explicit thread_id is in the payload, regardless of event context
  • Handler still gracefully skips/warns when no thread_id and no PR event context
  • Smoke Claude test passes end-to-end
• Technical Approach: In the handler, check if thread_id is present in the item payload. If yes, skip the event context guard and proceed. If no thread_id, apply the existing context check.
• Estimated Effort: Small
• Dependencies: None

Work Item 2: Renew Cross-Repo PAT Credentials

• Type: Infrastructure/Secret Rotation
• Priority: High
• Description: The PAT used to checkout githubnext/gh-aw-side-repo has expired. Smoke Create Cross-Repo PR and Smoke Update Cross-Repo PR have been failing since 2026-04-07.
• Acceptance Criteria:
  • Both Smoke Create/Update Cross-Repo PR workflows complete successfully
  • New PAT stored as repo/org secret
• Technical Approach: Generate a new fine-grained PAT with repo read access to githubnext/gh-aw-side-repo, update the secret.
• Estimated Effort: Small
• Dependencies: Access to PAT rotation for githubnext org

---

Historical Context

Trend Data (Last 7 Audits)

Date	Hard Failures	Success Rate	Health	Notable Issues
2026-04-06	0	N/A	✅ Healthy	—
2026-04-07	N/A	N/A	—	Rate limit burst, cross-repo creds
2026-04-08	0	N/A	✅ Excellent	—
2026-04-09	N/A	N/A	—	—
2026-04-10	N/A	85.7%	—	upload_artifact staging, resolve_thread permissions
2026-04-11	N/A	N/A	—	—
2026-04-13	3	97.8%	🔴 Degraded	API rate limit burst (3 create_issue failures)
2026-04-14	1	95.7%	🟡 Borderline	resolve_thread context check, no rate limit burst

Metrics and KPIs

• Overall Safe Output Success Rate: 95.7% (messages attempted)
• Most Reliable Job Types: add_comment, create_discussion, create_issue, dispatch_workflow (all 100%)
• Most Problematic Job Type: resolve_pull_request_review_thread (0% — context guard too strict)
• Rate Limit Burst: None today ✅ (improvement vs. 2026-04-13)

Next Steps

• Fix resolve_pull_request_review_thread to allow explicit thread_id bypass of context check
• Rotate PAT for githubnext/gh-aw-side-repo (cross-repo smoke tests broken since 2026-04-07)
• Validate/renew Gemini API key secret
• Monitor rate limit burst at noon UTC (no burst today — may indicate smaller workflow batch)

References:

• §24399262395 — Smoke Claude (resolve_thread failure)
• §24399977092 — Smoke Copilot (no review context warning)
• §24400520199 — Smoke Create Cross-Repo PR (bad credentials)

Generated by Safe Output Health Monitor · ● 535.3K · ◷

• expires on Apr 15, 2026, 1:23 PM UTC

[pelikhan]

/plan create issue for Fix resolve_pull_request_review_thread Context Check, not the others

[github-actions[bot]]

🚀 Plan Command has started processing this discussion comment

[github-actions[bot]]

This discussion has been marked as outdated by Safe Output Health Monitor.

A newer discussion is available at Discussion #26434.