Fix pr-malicious-scan: repeat-spam + integrity-filter blocks#722
Merged
Conversation
Root causes (observed on PR #237): 1. Agent's emitted add_comment body did not include the HTML marker line (<!-- pr-malicious-scan:fingerprint=... -->), so both the orchestrator's pre-dispatch check and the agent's own Step 1 idempotency lookup failed to find a prior scan for the same head SHA. Result: hourly re-dispatch. 2. The github MCP tools (pull_request_read, list_pull_requests, search_pull_requests) are blocked by the gh-aw integrity filter on PRs from non-approved authors -- exactly the population this scanner targets. Result: 'Integrity filter blocked N items' notes in every comment. Fixes: - pr-malicious-scan.agent.md: drop the github MCP toolset, add 'gh' to the bash allowlist, and instruct the agent to use 'gh api' for all PR data reads (PAT-authenticated, not subject to the integrity filter). - Strengthen Step 5: the HTML marker MUST be the first line of the comment body. Add a defense-in-depth note that the orchestrator also accepts the visible-body sentinel. - pr-triage-batch.yml + pr-triage-act.sh: match prior scans by EITHER the HTML marker OR the visible-body sentinel ('Automated diff scan' + backticked sha7), so a missing marker on a previously-emitted comment no longer triggers re-dispatch. Workflow disabled remotely while this lands.
Replace the per-push pull_request_target trigger and the gh-api workaround with the documented gh-aw pattern: - pr-malicious-scan.agent.md: drop pull_request_target; trigger only via workflow_dispatch from the orchestrator. Restore the github MCP toolset with min-integrity: none (the documented level for spam-detection / analytics workflows; safe-outputs still gates every mutation). Drop the 'gh' bash hack and visible-body sentinel requirements. - pr-triage-batch.yml: orchestrator now posts a deterministic '<!-- pr-malicious-scan:dispatched=SHORT --> ' comment BEFORE calling gh workflow run. That comment is the source of truth for 'a scan has been initiated for this head SHA' and survives every agent-side failure mode (PAT outage, integrity block, dropped HTML marker). Dedup matches either that orchestrator marker OR the agent's own fingerprint marker. - pr-triage-act.sh: drop the visible-body-sentinel fallback; match the orchestrator dispatched marker plus the agent fingerprint marker. Validated: gh aw compile clean; bash -n clean for both worker script and orchestrator embedded script; markdownlint clean; dedup query and POST api tested live against PR #713.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the PR malicious-diff scanning pipeline to stop repeat/spam dispatches and to avoid gh-aw integrity-filter blocks by (1) moving scan deduplication to the hourly orchestrator and (2) relaxing the GitHub MCP integrity policy for this scanner workflow.
Changes:
- Orchestrator (
pr-triage-batch.yml) now posts a deterministic “dispatched” marker comment before dispatching the scanner, and dedups on either dispatched/fingerprint markers. - Scanner agent prompt/workflow updates to remove
pull_request_target, rely onworkflow_dispatch, and configure GitHub MCP withmin-integrity: none. - Worker script (
pr-triage-act.sh) now treats either dispatched or fingerprint markers as “already handled for this head”.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/pr-triage-batch.yml |
Adds pre-dispatch marker comment + expanded marker matching to prevent duplicate scanner dispatch. |
.github/workflows/pr-malicious-scan.agent.md |
Updates trigger model and integrity settings; updates scanner instructions for head SHA lookup and idempotency. |
.github/workflows/pr-malicious-scan.agent.lock.yml |
Regenerates lock workflow for the updated agent; changes MCP guard policy and container image download behavior. |
.github/scripts/pr-triage-act.sh |
Updates scan-precedence logic to recognize orchestrator “dispatched” markers as idempotency signals. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 4/4 changed files
- Comments generated: 7
Contributor
|
👋 @JanKrivanek — this PR has 7 unresolved review thread(s). When you're ready, please address the feedback and push an update; the triage bot will pick up the next state automatically. (Add the |
Evangelink
approved these changes
Jun 4, 2026
- Step 1.4 idempotency check matches only fingerprint=, not dispatched= (the orchestrator-emitted marker would otherwise self-cancel every run) - tools.github sets allowed-repos: public + min-integrity: none (replaces gh api workaround) - Drop all gh api / base64 shell snippets from agent prompt; rely on MCP pull_request_read and repos.get_file_contents - Restore container image digest pinning (sha256) in lock manifest, comment block, and download_docker_images.sh args - Header comment in pr-triage-batch.yml notes orchestrator emits pre-dispatch idempotency marker
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes scanner repeat-spam and the integrity-filter blocks observed on PR #237.
Problem
The pr-malicious-scan agent workflow was hourly-spamming new PRs with duplicate
Automated diff scan completed for <sha>comments, accompanied byIntegrity filter blocked N items. Two root causes:pull_request_target [opened, synchronize, reopened]push and could complete before the next hourly orchestrator pass, but its dedup self-check depended on the agent typing an HTML marker as the first line of its comment — which it sometimes dropped.gh apifrom bash, but the agent step runs underawfwith--exclude-env COPILOT_GITHUB_TOKENand no other GitHub token is exposed, sogh apiwould have had no credentials at runtime.Fix
Adopt the documented gh-aw pattern (per Integrity Filtering Reference — \min-integrity: none\ is the recommended level for spam-detection / analytics workflows) and move dedup ownership to the orchestrator:
pr-malicious-scan.agent.md— droppull_request_target; trigger only viaworkflow_dispatchfrom the orchestrator. Restoretools.githubwithmin-integrity: none. Drop theghbash hack and the visible-body sentinel requirements.safe-outputs(only code-scanning alerts, one comment, ≤ 2 labels) andpermissions: contents: read, pull-requests: readstill gate every mutation.pr-triage-batch.yml— orchestrator now posts a deterministic<!-- pr-malicious-scan:dispatched=SHORT -->comment BEFORE callinggh workflow run. That comment is the source of truth for ‘a scan has been initiated for this head SHA' and survives every agent-side failure mode (PAT outage, integrity block, dropped HTML marker). Dedup query matches either the orchestrator marker or the agent's own fingerprint marker.pr-triage-act.sh— drop the visible-body-sentinel fallback; match the orchestrator dispatched marker plus the agent fingerprint marker.Validation
gh aw compile pr-malicious-scan.agent— clean.bash -non the worker script and on the YAML-extracted orchestrator script — clean.markdownlinton the agent.md — clean.gh api -X POST, confirmed the orchestrator's--jqselector matches it, confirmed the bot-author filter excludes non-bot comments, and cleaned up the test comment.The workflow is currently \disabled_manually\ on the repo and will stay that way until this PR merges. Merging will re-enable the cron-driven orchestrator, which is now the only dispatch path.