Skip to content

Content scope dependabot#2727

Open
jonathanKingston wants to merge 23 commits into
mainfrom
jkt/auto/content-scope-dependabot-591f
Open

Content scope dependabot#2727
jonathanKingston wants to merge 23 commits into
mainfrom
jkt/auto/content-scope-dependabot-591f

Conversation

@jonathanKingston
Copy link
Copy Markdown
Contributor

@jonathanKingston jonathanKingston commented May 27, 2026

Asana Task/Github Issue: https://app.asana.com/1/137249556945/project/1201614831475344/task/1211495549578747?focus=true

Description

This change auto reviews Dependabot changes that have 14 days cooldown. It ensures that we pass all the other internal checks and then auto validates that with the model. It should be restricted only to Dependabot and has limited risk as upstreams will still manage their merges.

Testing Steps

Checklist

Please tick all that apply:

  • I have tested this change locally
  • I have tested this change locally in all supported browsers
  • This change will be visible to users
  • I have added automated tests that cover this change
  • I have ensured the change is gated by config
  • This change was covered by a ship review
  • This change was covered by a tech design
  • Any dependent config has been merged

Note

High Risk
Automated approve/merge on pull_request_target with write permissions and model-driven decisions materially increases supply-chain and workflow-trust risk if checks or parsing are bypassed.

Overview
Replaces patch-only, semver-gated Dependabot auto-approval with an Anthropic-backed safety gate for all npm updates (still Dependabot-only). Dependabot is tuned to daily opens and a 14-day cooldown (was weekly / 2-day).

Adds dependabot-anthropic-gate.mjs (and unit tests): polls until CI and three trusted Cursor checks finish, builds evidence from check output plus cursor[bot] reviews/comments (including inline Bugbot), calls Anthropic, and emits safe_to_merge. The workflow only approves and enables auto-merge when that flag is true; draft/conflict/merge failures are skipped without failing the job.

The auto-merge workflow moves to pull_request_target, checks out the gate script from the base branch, and grants actions:read / checks:read so the in-repo waiter does not need a third-party “wait for checks” action.

Reviewed by Cursor Bugbot for commit 571fd09. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions github-actions Bot added the semver-patch Bug fix / internal — no release needed label May 27, 2026
@github-actions github-actions Bot requested review from dbajpeyi, dharb and shakyShane May 27, 2026 23:21
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 27, 2026

[Beta] Generated file diff

Time updated: Fri, 29 May 2026 11:03:41 GMT

@jonathanKingston
Copy link
Copy Markdown
Contributor Author

@shakyShane, @dbajpeyi or @dharb This is to add auto-merging of npm package changes that have happened after a 14-day cooldown period. We validate with Cursor and the various other automations that we have, and use Anthropic to validate all of these automations. If they all pass, then we merge automatically, assuming that all tests pass.

@jonathanKingston jonathanKingston marked this pull request as ready for review May 27, 2026 23:26
@jonathanKingston jonathanKingston requested a review from a team as a code owner May 27, 2026 23:26
Comment thread .github/workflows/dependabot-auto-merge.yml Outdated
Comment thread .github/scripts/dependabot-anthropic-gate.mjs
Comment thread .github/scripts/dependabot-anthropic-gate.mjs
@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Cursor review was not successful.

This PR requires a manual review and approval from a member of one of the following teams:

  • @duckduckgo/content-scope-scripts-owners
  • @duckduckgo/apple-devs
  • @duckduckgo/android-devs
  • @duckduckgo/team-windows-development
  • @duckduckgo/extension-owners
  • @duckduckgo/config-aor
  • @duckduckgo/breakage-aor
  • @duckduckgo/breakage

Comment thread .github/scripts/dependabot-anthropic-gate.mjs Outdated
The previous self-exclusion compared run.run_id (a field that does not
exist on the Check Runs API response) and then fell through to a
hardcoded run.name === 'dependabot' filter. That created a fragile
coupling to the YAML job key: any rename would silently include the
gate's own check run in waitForOtherChecksToPass and deadlock until
the 30-minute timeout.

Resolve the current workflow run's job IDs via the Actions Jobs API
(GitHub Actions sets job.id == check_run.id) and exclude any matching
check run from the wait set.

Co-authored-by: Jonathan Kingston <jonathanKingston@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 27, 2026

Build Branch

Branch pr-releases/jkt/auto/content-scope-dependabot-591f
Commit 788ae9ba8a
Updated May 29, 2026 at 11:03:08 AM UTC

Static preview entry points

QR codes (mobile preview)
Entry point QR code
Docs QR for docs preview
Static pages QR for static pages preview
Integration pages QR for integration pages preview

Integration commands

npm (Android / Extension):

npm i github:duckduckgo/content-scope-scripts#pr-releases/jkt/auto/content-scope-dependabot-591f

Swift Package Manager (Apple):

.package(url: "https://github.com/duckduckgo/content-scope-scripts.git", branch: "pr-releases/jkt/auto/content-scope-dependabot-591f")

git submodule (Windows):

git -C submodules/content-scope-scripts fetch origin pr-releases/jkt/auto/content-scope-dependabot-591f
git -C submodules/content-scope-scripts checkout origin/pr-releases/jkt/auto/content-scope-dependabot-591f
Pin to exact commit

npm (Android / Extension):

npm i github:duckduckgo/content-scope-scripts#788ae9ba8a4b878902b69ac4db3dc265bca2b2ae

Swift Package Manager (Apple):

.package(url: "https://github.com/duckduckgo/content-scope-scripts.git", revision: "788ae9ba8a4b878902b69ac4db3dc265bca2b2ae")

git submodule (Windows):

git -C submodules/content-scope-scripts fetch origin pr-releases/jkt/auto/content-scope-dependabot-591f
git -C submodules/content-scope-scripts checkout 788ae9ba8a4b878902b69ac4db3dc265bca2b2ae

Comment thread .github/workflows/dependabot-auto-merge.yml
Comment thread .github/scripts/dependabot-anthropic-gate.mjs Outdated
matchedCursorSources() previously matched any review or issue comment
whose body contained the public Cursor agent id. Because the agent id
appears in plain text in PR comments, anyone with comment access could
echo it (or a fabricated transcript around it) and have the Anthropic
gate treat that text as authenticated Cursor-authored evidence —
prompt injection against the auto-merge decision.

Filter sources at construction time: drop any review/comment whose
author is not a GitHub App bot in the explicit
TRUSTED_AUTOMATION_AUTHORS allow-list (currently just 'cursor[bot]').
Untrusted authors are returned as null and stripped before sources
reach matchedCursorSources(). Also tighten the Anthropic system prompt
to reflect that filtering happens upstream.

Co-authored-by: Jonathan Kingston <jonathanKingston@users.noreply.github.com>
Comment thread .github/scripts/dependabot-anthropic-gate.mjs
cursoragent and others added 2 commits May 28, 2026 00:05
latestCheckRunsByName() previously matched expected Cursor checks
solely by display name. Any installed GitHub App with checks:write
permission could have published a later success with the same display
name, and the Anthropic gate would have treated it as authenticated
Cursor evidence.

Convert EXPECTED_CHECKS to structured entries that pin each check to:
 - run.name              (display name)
 - run.app.slug          (globally unique GitHub App slug 'cursor')
 - URL(run.details_url).host  ('cursor.com')

A new matchExpectedCheck() returns the matching expected entry only
when all three signals line up; latestCheckRunsByName() ignores
anything else. Updated main()'s missingChecks computation to read
expected.name from the new shape.

Co-authored-by: Jonathan Kingston <jonathanKingston@users.noreply.github.com>
Removed the three fountainhead/action-wait-for-check@v1.2.0 steps and
the Validate Cursor checks step from dependabot-auto-merge.yml. That
action was referenced by mutable tag and was being handed GITHUB_TOKEN
on a pull_request_target job with contents:write / pull-requests:write,
which would let a tag retag or repo compromise of fountainhead use the
write-scoped token during Dependabot PR processing.

Folded the wait inline into dependabot-anthropic-gate.mjs, which is
already checked out from the trusted base branch via sparse-checkout
and runs with the same permissions:

  - Renamed waitForOtherChecksToPass -> waitForChecksToSettle and made
    it block until (a) every non-current check run on the head SHA is
    completed and passing, (b) commit statuses are non-pending and
    non-failed, AND (c) every EXPECTED_CHECKS entry has appeared as a
    matching trusted check run and reached 'completed' state. Without
    (c) the loop could exit before Cursor's checks were even
    registered, then fail with 'Missing expected Cursor checks'.
  - Renamed OTHER_CHECK_TIMEOUT_MS / OTHER_CHECK_POLL_INTERVAL_MS to
    CHECK_WAIT_TIMEOUT_MS / CHECK_WAIT_POLL_INTERVAL_MS to match.
  - Added missingExpectedCheckNames() and pendingExpectedCheckRuns()
    helpers reused for both the wait and the timeout error messages.

The Anthropic step now runs whenever the npm classification matches;
the script itself enforces all the previous gating, so failure modes
are unchanged from a caller's perspective.

Co-authored-by: Jonathan Kingston <jonathanKingston@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Web Compatibility Assessment

No injected-runtime findings. The current diff is limited to .github/dependabot.yml, .github/workflows/dependabot-auto-merge.yml, and .github/scripts/dependabot-anthropic-gate.mjs; it does not touch injected/src, wrapper utilities, captured globals, messaging transports, platform entry points, package manifests, or lockfiles.

Security Assessment

  1. .github/scripts/dependabot-anthropic-gate.mjs lines 375-379, severity: error
    Existing open finding still applies: the Anthropic gate fetches top-level review bodies and issue comments, but not inline PR review comments. Cursor/Bugbot findings are often inline-only with empty review bodies, so the final gate can miss blocking automation feedback.

  2. .github/workflows/dependabot-auto-merge.yml lines 7-10, severity: warning
    Existing open finding still applies: fetchCurrentWorkflowCheckRunIds() calls the Actions Jobs API, but the explicit permissions block omits actions: read. That can 403 before the gate reaches its fail-closed decision.

  3. .github/workflows/dependabot-auto-merge.yml lines 43-71, severity: warning
    Existing open finding still applies: this privileged pull_request_target workflow passes GITHUB_TOKEN to fountainhead/action-wait-for-check@v1.2.0, a mutable third-party tag. Pin it to a full commit SHA or move the wait logic into checked-in base-branch code.

Previously reported issues for name-only Cursor check trust and untrusted review/comment authors are addressed in the latest commit via run.app.slug / details_url host validation and TRUSTED_AUTOMATION_AUTHORS filtering.

Risk Level

High Risk for workflow/security automation. Injected runtime risk is low, but this PR adds privileged auto-approval and auto-merge machinery for future dependency updates.

Recommendations

Fetch trusted inline review comments before asking Anthropic, add actions: read, and pin or replace the third-party wait action.

Validation: node --check .github/scripts/dependabot-anthropic-gate.mjs passes.

Open in Web View Automation 

Sent by Cursor Automation: Web compat and sec

Comment thread .github/scripts/dependabot-anthropic-gate.mjs
Comment thread .github/workflows/dependabot-auto-merge.yml
Comment thread .github/workflows/dependabot-auto-merge.yml
parseAnthropicDecision used /\{[\s\S]*\}/, which matches greedily
from the first '{' to the *last* '}' in the response. Any preamble
containing curly braces — e.g. 'Based on {evidence}, here is the JSON:
{...}' — would make the match span non-JSON content, JSON.parse would
throw, and a legitimate safe_to_merge=true decision would be lost.

Replace it with a generator candidateJsonObjects() that walks the text
emitting every brace-balanced '{...}' substring, tracking JSON string
literals and escapes so braces inside quoted values do not disturb
depth. parseAnthropicDecision now iterates candidates and returns the
first one that both parses as JSON and has the expected
{safe_to_merge:boolean, reason:string} shape, surfacing the last
parse/shape error if no candidate matches.

Co-authored-by: Jonathan Kingston <jonathanKingston@users.noreply.github.com>
@cursor cursor Bot force-pushed the jkt/auto/content-scope-dependabot-591f branch from 6e35b15 to 6121e20 Compare May 28, 2026 10:17
Comment thread .github/scripts/dependabot-anthropic-gate.mjs
Comment thread .github/scripts/dependabot-anthropic-gate.mjs
Comment thread .github/scripts/dependabot-anthropic-gate.mjs Outdated
cursoragent and others added 2 commits May 28, 2026 10:20
The gate previously assembled evidence from two sources only:
  - GET /pulls/{pr}/reviews   (top-level review bodies)
  - GET /issues/{pr}/comments (issue / conversation comments)

It did not pull inline review comments (those attached to a diff
hunk) from GET /pulls/{pr}/comments. Cursor Bugbot publishes its
blocking findings as inline review comments with an empty parent
review body, so omitting that feed would let the Anthropic gate
auto-approve while concrete blocking findings sit unread on the PR.

Add a sourceFromInlineReviewComment builder (typed as
'inline_review_comment', carrying path/line/in_reply_to_id for audit)
gated by the existing isTrustedAutomationActor allow-list, and
include the new feed in main()'s Promise.all + sources merge. Trust
boundary is unchanged: untrusted authors still drop to null and are
filtered out before evidence is sent to Anthropic.

Co-authored-by: Jonathan Kingston <jonathanKingston@users.noreply.github.com>
fetchCurrentWorkflowCheckRunIds() calls GET /actions/runs/{run_id}/jobs
to map the current workflow run's jobs to their corresponding check-run
IDs, which is what lets waitForChecksToSettle() ignore its own check
run when deciding whether 'other' checks are pending.

Once a workflow declares an explicit permissions block, every scope
not listed defaults to 'none'. The block listed checks/contents/
pull-requests but omitted actions, so GITHUB_TOKEN would 403 against
the Actions Jobs API and the gate would error out before reaching its
fail-closed Anthropic decision.

Add 'actions: read' with an explanatory comment so the dependency is
documented at the permission site.

Co-authored-by: Jonathan Kingston <jonathanKingston@users.noreply.github.com>
@cursor cursor Bot force-pushed the jkt/auto/content-scope-dependabot-591f branch from dab5c39 to 7e81412 Compare May 28, 2026 10:22
Copy link
Copy Markdown
Contributor

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Web Compatibility Assessment

No injected-runtime findings. The diff is limited to .github/dependabot.yml, .github/workflows/dependabot-auto-merge.yml, and .github/scripts/dependabot-anthropic-gate.mjs; it does not touch injected/src, wrapper utilities, captured globals, messaging transports, platform entry points, package manifests, or lockfiles.

Security Assessment

  1. .github/scripts/dependabot-anthropic-gate.mjs lines 467-470, severity: error
    The Anthropic gate still gathers top-level PR review bodies and issue comments, but not inline PR review comments / review threads. Cursor/Bugbot findings are often inline-only with empty review bodies, so the final merge gate can miss blocking automation feedback.

  2. .github/workflows/dependabot-auto-merge.yml lines 7-10, severity: warning
    The script calls GET /actions/runs/{run_id}/jobs via fetchCurrentWorkflowCheckRunIds(), but the explicit workflow permissions block grants checks: read and omits actions: read. With unspecified permissions set to none, this can 403 before the gate reaches its assessment.

Previously reported issues around Cursor check spoofing and untrusted review/comment author evidence are addressed in the current diff via run.app.slug / details_url host validation and TRUSTED_AUTOMATION_AUTHORS filtering.

Risk Level

High Risk for workflow/security automation. Injected runtime risk is low, but this PR adds privileged pull_request_target auto-approval/auto-merge machinery for future Dependabot npm updates.

Recommendations

  1. Fetch trusted inline review findings before asking Anthropic, preferably via GraphQL reviewThreads so resolved/outdated threads can be handled deliberately; /pulls/{pull_number}/comments is the simpler alternative if thread state is not needed.
  2. Add actions: read to the workflow permissions, or remove the Actions Jobs API dependency and self-exclude the gate another way.
  3. Add a mocked test/fixture where a Cursor run has an empty review body plus an inline blocking comment, and assert the gate evidence includes that finding.

Validation: git show 6121e20a8e159b7244989eff956251c9d8929f50:.github/scripts/dependabot-anthropic-gate.mjs | node --check --input-type=module - passes.

Open in Web View Automation 

Sent by Cursor Automation: Web compat and sec

Comment thread .github/workflows/dependabot-auto-merge.yml
Comment thread .github/workflows/dependabot-auto-merge.yml
Comment thread .github/scripts/dependabot-anthropic-gate.mjs
matchedCursorSources() only matched bodies containing the agent id
extracted from a check run's details_url (/agents/<id>). That works
for the two 'Cursor Automation: ...' runs, but Cursor Bugbot uses a
generic https://cursor.com/docs/bugbot URL with no agent id, so
cursorAgentId() returned null and matchedCursorSources() returned []
for every Bugbot check — even after the inline-comment fetch landed.
The Anthropic gate could therefore auto-approve without ever seeing
the Bugbot inline findings on the PR.

Add a small sourceMatchesCheckRun() helper. Default match is still
the agent id; when the run is 'Cursor Bugbot' specifically, fall
back to matching trusted-author bodies that mention the check run's
head_sha. Bugbot's review summary, per-finding inline comments, and
threaded follow-ups all include a 'Reviewed by Cursor Bugbot for
commit <head_sha>' footer, so head_sha presence is a reliable
attribution signal that:

  - never re-trusts arbitrary comment authors (isTrustedAutomationActor
    has already filtered sources to cursor[bot] only),
  - scopes findings to the current revision (Bugbot comments for
    earlier commits with a different head_sha don't match),
  - does not depend on Bugbot's URL scheme or the 'BUGBOT_BUG_ID' /
    'BUGBOT_REVIEW' marker shape, so it survives Bugbot UI tweaks.

Verified with 5 cases: Bugbot inline finding for this head_sha
matches; Bugbot review summary for this head_sha matches; Bugbot
comment referencing a different head_sha is rejected; Cursor
Automation runs still match by agent id; Cursor Automation does not
fall back to head_sha.

Co-authored-by: Jonathan Kingston <jonathanKingston@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Web Compatibility Assessment

No findings. The current diff is limited to .github/dependabot.yml, .github/workflows/dependabot-auto-merge.yml, and .github/scripts/dependabot-anthropic-gate.mjs; it does not touch injected/src, wrapper utilities, captured globals, messaging transports, platform entry points, package manifests, or lockfiles.

Security Assessment

No new findings in the current head. I rechecked the previously reported automation-gate concerns and they appear addressed: Cursor check runs are pinned by name + app slug + details URL host, review/comment evidence is filtered to trusted cursor[bot] authors, inline PR review comments are fetched, Bugbot evidence is matched by the trusted check head SHA, actions: read is granted for the Jobs API call, and the mutable third-party wait action is no longer used.

Risk Level

High Risk for workflow/security automation: injected runtime risk is low, but this adds privileged pull_request_target auto-approval/auto-merge machinery for future Dependabot npm updates.

Recommendations

Add fixture coverage for the gate script if practical, especially: spoofed check names from another app, untrusted comments containing agent IDs, inline-only Bugbot findings with details_url: https://cursor.com/docs/bugbot, and brace-balanced Anthropic JSON parsing.

Validation: node --check .github/scripts/dependabot-anthropic-gate.mjs passes.

Open in Web View Automation 

Sent by Cursor Automation: Web compat and sec

The last Cursor automation review reported no remaining security
findings but recommended fixture coverage for the gate script,
particularly: spoofed check names from another GitHub App, untrusted
comments containing agent ids, inline-only Bugbot findings with
details_url=https://cursor.com/docs/bugbot, and brace-balanced
Anthropic JSON parsing.

Make the script importable so it can be unit-tested without invoking
main():

  - Export the per-helper functions and the configuration constants
    (EXPECTED_CHECKS, TRUSTED_AUTOMATION_AUTHORS,
    PASSING_CHECK_CONCLUSIONS).
  - Guard the previous unconditional 'await main()' behind a
    fileURLToPath(import.meta.url) === process.argv[1] check so the
    workflow invocation still runs the gate but tests can import
    helpers safely.

Add dependabot-anthropic-gate.test.mjs following the existing
.github/scripts/review-helpers.test.mjs node:test / node:assert
convention, with 41 assertions across 16 suites covering:

  - matchExpectedCheck: trusted check accepted; spoofed app slug,
    wrong host, and unknown display name rejected.
  - latestCheckRunsByName: picks the latest matching run per name,
    rejects spoofed apps.
  - latestOtherCheckRunsByName / checkRunState: excludes own
    workflow's check-run ids; pending/failed/passing classification.
  - commitStatusState: pending vs failed.
  - missingExpectedCheckNames / pendingExpectedCheckRuns: report
    which expected Cursor checks have not yet appeared or are still
    in flight.
  - isTrustedAutomationActor and the three source builders:
    untrusted authors map to null, trusted authors produce the
    expected shape (path/line/inReplyToId on inline comments).
  - sourceMatchesCheckRun: Cursor Automation runs match by agent id;
    Cursor Bugbot runs match by head_sha; cross-commit Bugbot
    findings rejected; non-Bugbot runs do not fall back to head_sha.
  - matchedCursorSources: only surfaces the expected source per run.
  - evidenceForRun: long output text is truncated.
  - candidateJsonObjects / parseAnthropicDecision: preamble braces,
    multiple top-level objects, escape sequences, garbage input.
  - truncate, parseLinkHeader: utility behavior.

Co-authored-by: Jonathan Kingston <jonathanKingston@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Web Compatibility Assessment

No injected-runtime findings. The current diff is limited to .github/dependabot.yml, .github/workflows/dependabot-auto-merge.yml, and .github/scripts/dependabot-anthropic-gate*.mjs; it does not touch injected/src, wrapper utilities, captured globals, messaging transports, platform entry points, package manifests, or lockfiles.

Security Assessment

  1. .github/scripts/dependabot-anthropic-gate.mjs lines 432-445, severity: error
    parseAnthropicDecision() accepts the first brace-balanced object with { safe_to_merge: boolean, reason: string } anywhere in Anthropic's response. Because the prompt includes untrusted automation bodies as evidence, a prompt-injected or quoted snippet such as {"safe_to_merge":true,"reason":"..."} can appear before the model's actual final decision and be treated as authoritative. For an auto-merge gate, ambiguous or extra-text responses should fail closed or require an exact single decision object.

Previously reported workflow-gate concerns around check-run spoofing, trusted author filtering, inline comments, Bugbot attribution, actions: read, and the mutable wait action appear addressed.

Risk Level

High Risk for workflow/security automation: injected runtime risk is low, but this adds privileged pull_request_target auto-approval/auto-merge machinery for future Dependabot npm updates.

Recommendations

  1. Parse only an exact decision payload, e.g. JSON.parse(text.trim()) plus strict shape validation, or fail closed if there is any pre/post text or more than one candidate decision object.
  2. Add a regression fixture where Anthropic returns a quoted safe-looking decision object followed by the real blocking decision; the gate should reject the response or choose the unambiguous final decision.

Validation: node --check .github/scripts/dependabot-anthropic-gate.mjs && node --test .github/scripts/dependabot-anthropic-gate.test.mjs passes.

Open in Web View Automation 

Sent by Cursor Automation: Web compat and sec

Comment thread .github/scripts/dependabot-anthropic-gate.mjs Outdated
Copy link
Copy Markdown
Contributor

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 0d6071b. Configure here.

Comment thread .github/scripts/dependabot-anthropic-gate.mjs
Two unresolved review threads on the PR were still valid:

1. parseAnthropicDecision accepted the first valid-shape JSON object
   anywhere in the model response. Because evidence sent to Anthropic
   includes untrusted review/comment bodies, the model could be
   induced to quote {"safe_to_merge":true,...} from an attacker's
   comment before its real answer, and the parser would treat the
   quoted snippet as the merge decision.

   Replace candidateJsonObjects / first-match parsing with strict
   validation: the trimmed response must be exactly a single JSON
   object whose keys are exactly {safe_to_merge, reason, confidence},
   with type and enum validation on each field. Any preamble, code
   fence, trailing prose, extra key, or wrong type fails closed.
   candidateJsonObjects is no longer needed and is removed.

2. pendingExpectedCheckRuns previously returned every matching
   pending run without deduplicating by name, while the rest of the
   wait loop reasons about the latest run per name. A stale
   in_progress Cursor check sitting alongside a newer completed one
   for the same name would keep pendingCursor non-empty even when
   otherIdle was true, deadlocking waitForChecksToSettle() until the
   30-minute timeout fired.

   Route through latestCheckRunsByName() so only the latest matching
   run per name is considered, then filter for non-completed runs.

Test coverage updated: 11 stricter parseAnthropicDecision cases
(bare object, whitespace, preamble, trailing prose, code fence,
embedded snippet, extra keys, missing/invalid required fields,
invalid confidence enum, non-object JSON, empty input, garbage) and
a new stale-run dedup case for pendingExpectedCheckRuns. 46/46 pass
on node --test.

Co-authored-by: Jonathan Kingston <jonathanKingston@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Web Compatibility Assessment

No injected-runtime findings. The current diff is limited to .github/dependabot.yml, .github/workflows/dependabot-auto-merge.yml, and .github/scripts/dependabot-anthropic-gate*.mjs; it does not touch injected/src, wrapper utilities, captured globals, messaging transports, platform entry points, package manifests, or lockfiles.

Security Assessment

No new findings in the current head. I rechecked the previously reported automation-gate concerns and they appear addressed: trusted Cursor check runs are validated by name + app slug + details_url host, source bodies are restricted to trusted cursor[bot] authors, inline PR review comments are fetched, Bugbot evidence is matched by trusted check head_sha, actions: read is granted for the Jobs API call, the mutable third-party wait action is removed, Anthropic responses now require an exact strict JSON decision object, and stale pending Cursor runs are deduped through the latest trusted run per expected check name.

Risk Level

High Risk for workflow/security automation: injected runtime risk is low, but this adds privileged pull_request_target auto-approval/auto-merge machinery for future Dependabot npm updates.

Recommendations

Keep the gate script covered by the new fixtures when changing it; the important regression cases are spoofed check names from another app, untrusted comments containing agent IDs, inline-only Bugbot findings, exact-only Anthropic JSON parsing, and stale in-progress Cursor checks alongside newer completed runs.

Validation: node --check .github/scripts/dependabot-anthropic-gate.mjs && node --test .github/scripts/dependabot-anthropic-gate.test.mjs passes.

Open in Web View Automation 

Sent by Cursor Automation: Web compat and sec

The repo's tsconfig.json includes .github/scripts with checkJs:true and
strictNullChecks:true, so the gate test file participates in 'npm run
tsc'. Three categories of errors:

  - matchExpectedCheck() returns the entry or null; the
    EXPECTED_CHECKS.includes(matched) assertion was passing
    {entry}|null where {entry} was expected. Add an assert.ok(matched)
    null guard before the includes() call.
  - The local externalRun() helper inferred its conclusion parameter
    type from its default of null, so callers passing 'success' /
    'skipped' / 'failure' tripped TS2345. Annotate the helper with
    JSDoc so conclusion is typed as string | null.
  - sourceFromInlineReviewComment() returns the source object or
    null. The captures-path/line/in_reply_to_id test asserted on
    properties of the result without first narrowing it. Add
    assert.ok(src) before the property accesses.

No behavior change. tsc still reports three pre-existing errors in
injected/src/features/* about generated build files
(build/locales/*-locales.js, surrogates-generated.js); those are
unrelated to this PR and only appear when 'npm run build' has not
been run.

Co-authored-by: Jonathan Kingston <jonathanKingston@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Web Compatibility Assessment

No injected-runtime findings. The current diff is limited to Dependabot configuration plus GitHub workflow/script/test files under .github/; it does not touch injected/src, wrapper utilities, captured globals, messaging transports, platform entry points, package manifests, or lockfiles.

Security Assessment

  1. .github/scripts/dependabot-anthropic-gate.mjs lines 185-203, severity: warning
    latestOtherCheckRunsByName() collapses all non-current check runs by display name only before checkRunState() checks for failures. Unlike matchExpectedCheck(), this path does not include the authoring app in the identity key, so a different installed GitHub App with checks:write could publish a newer successful check run named like a failing CI job and cause the gate to ask Anthropic with that failure omitted. Branch protection should still stop required checks, but the gate no longer enforces its stated “all other check runs passed” invariant for non-required or ambiguously protected checks.

Previously reported issues around inline review comments, actions: read, mutable third-party wait actions, trusted Cursor check identity, trusted comment authors, stale pending Cursor runs, and strict Anthropic output parsing appear addressed in this revision.

Risk Level

High Risk for workflow/security automation. Injected runtime risk is low, but this PR adds privileged pull_request_target auto-approval/auto-merge machinery for future Dependabot npm updates.

Recommendations

  1. Key “other” check deduplication by both display name and authoring app identity (run.app?.slug or run.app?.id), or otherwise ensure a success from one app cannot supersede a failure from another app.
  2. Add a unit test where github-actions reports lint as failed and another app reports a newer successful lint; checkRunState() should still surface the failure.
  3. Keep relying on branch protection as the final merge guard, but keep this gate fail-closed when its own evidence is contradictory.

Validation: node --check .github/scripts/dependabot-anthropic-gate.mjs and node --test .github/scripts/dependabot-anthropic-gate.test.mjs pass.

Open in Web View Automation 

Sent by Cursor Automation: Web compat and sec

Comment thread .github/scripts/dependabot-anthropic-gate.mjs Outdated
…can't mask failures

latestOtherCheckRunsByName() previously deduped non-Cursor check runs
by display name only, so an installed GitHub App with checks:write
could publish a newer 'lint: success' check run and have it supersede
a failing 'lint' check run from github-actions before checkRunState()
saw the failure. With the gate auto-asking Anthropic on success, that
would silently drop a real CI failure from the gate's evidence.

Add a checkRunIdentityKey() that combines run.app.slug (falling back
to run.app.id, then null) with run.name, and dedupe on that composite
key. Each app's runs for a given name are tracked independently, so a
failure from any app still surfaces while reruns within the same app
still collapse to the latest run.

Tests:
  - new regression: github-actions reports 'lint: failure', another
    app posts a newer 'lint: success'; the failure must still appear
    in checkRunState().failed.
  - new regression: two reruns of 'lint' from the same app collapse
    to the latest run (status preserved).
  - existing tests updated to attach an explicit app.slug to the
    fixture so the new key works (default 'github-actions').

48/48 node --test cases pass; node --check, prettier, and eslint
clean. Pre-existing tsc errors in injected/src/features/* about
generated build files are unrelated and unchanged.

Co-authored-by: Jonathan Kingston <jonathanKingston@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Web Compatibility Assessment

No injected-runtime findings. The current diff is limited to Dependabot configuration plus GitHub workflow/script/test files under .github/; it does not touch injected/src, wrapper utilities, captured globals, messaging transports, platform entry points, package manifests, or lockfiles.

Security Assessment

No new findings in the current head. The previously reported cross-app check-run spoofing gap appears addressed: non-Cursor check runs are now deduped by (app, name), so a newer success from another app cannot mask a failing same-named check from github-actions, and the regression is covered in .github/scripts/dependabot-anthropic-gate.test.mjs.

Previously reviewed gate concerns around inline review comments, actions: read, mutable third-party wait actions, trusted Cursor check identity, trusted comment authors, Bugbot attribution, stale pending Cursor runs, and strict Anthropic output parsing still appear addressed.

Risk Level

High Risk for workflow/security automation. Injected runtime risk is low, but this PR adds privileged pull_request_target auto-approval/auto-merge machinery for future Dependabot npm updates.

Recommendations

No blocking changes. Keep the gate fixtures covering spoofed check identities, trusted-source filtering, inline-only Bugbot findings, exact Anthropic JSON parsing, stale Cursor runs, and cross-app same-name check collisions when modifying this workflow.

Validation: node --check .github/scripts/dependabot-anthropic-gate.mjs && node --test .github/scripts/dependabot-anthropic-gate.test.mjs passes (48/48).

Open in Web View Automation 

Sent by Cursor Automation: Web compat and sec

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

semver-patch Bug fix / internal — no release needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants