Skip to content

feat!: reduce the action to a generic prompt runner#55

Open
Sayt-0 wants to merge 2 commits into
mainfrom
feat/generic-prompt-runner
Open

feat!: reduce the action to a generic prompt runner#55
Sayt-0 wants to merge 2 commits into
mainfrom
feat/generic-prompt-runner

Conversation

@Sayt-0

@Sayt-0 Sayt-0 commented Jul 9, 2026

Copy link
Copy Markdown
Member

Implements docker/gordon#1258 (part of epic docker/gordon#1253): the public Action is reduced to a generic prompt runner. The reviewer orchestration and all Docker-internal auth are removed; the core runner (pinned docker-agent binary download, prompt execution with retries, output filtering, secret-leak scanning, job summary, verbose-log artifact, optional mcp-gateway) is unchanged. A minimal, documented PR-reviewer example built on the generic runner replaces the full-featured pipeline.

Issue acceptance criteria

Criterion Implementation
Generic prompt runner, no Docker-specific logic Only the runner core remains (src/main, src/security library, src/signed-commit release CLI)
Docker-specific orchestration removed review-pr/, .github/actions/, 6 reviewer workflows, 16 reviewer src/ modules, .agents/ skill deleted
No Docker auth; users supply their own API keys setup-credentials/ (AWS OIDC / GitHub App), the 4-tier authorization waterfall and the org-membership-token, auth-org, skip-auth inputs removed; explicit per-provider API key inputs unchanged
Reviewer example included and documented examples/reviewer/: minimal read-only agent YAML, copy-pasteable workflow, README with setup and security notes

Changes

Area Change
action.yml Inputs org-membership-token, auth-org, skip-auth and output cagent-version removed; everything else unchanged
src/ 16 reviewer modules deleted; src/main/auth.ts and the src/security CLI (index.ts, check-auth.ts) removed; sanitize-input/output kept as library
Build tsup entries reduced to main and signed-commit; @aws-sdk/* and @octokit/auth-app dependencies dropped
release.yml 604 to 178 lines: single-pass dist staging via signed-commit on a throwaway release-staging/<version> branch, tag, gh release create --generate-notes; publish-agent, update-self-refs and notify jobs removed
test-e2e.yml Reviewer and shell-test jobs removed; remaining jobs use plain repo secrets instead of AWS SSM
update-docker-agent-version.yml, test.yml setup-credentials and OIDC permissions removed
Credentials in CI `${{ secrets.RELEASE_TOKEN
Docs README.md, AGENTS.md, SECURITY.md, CONTRIBUTING.md rewritten to match the slimmed repo

Diffstat: 120 files changed, +422 / -22037.

Breaking changes

  • review-pr and setup-credentials sub-actions, the reusable review-pr.yml workflow and the mention-reply internal action are removed.
  • org-membership-token, auth-org, skip-auth inputs and cagent-version output are removed.
  • The action no longer performs any authorization check; gating who can trigger a workflow is the caller's responsibility.
  • Consumers pinned to existing tags are unaffected; this should ship as a major release, ideally once the self-hosted reviewer service (docker/gordon#1284, docker/gordon#1256) is available.

Validation

Check Result
pnpm build pass (dist/ contains exactly main.js, signed-commit.js)
pnpm test 140/140 unit tests pass
pnpm test:integration 13/13 pass
pnpm lint (biome + tsc + actionlint) pass
actionlint on examples/reviewer/review-pr.yml pass (run explicitly; examples/ is outside the default scan path)

Notes for reviewers

  • Repository secrets to configure after merge: OPENAI_API_KEY / ANTHROPIC_API_KEY for e2e tests (replace AWS SSM), optional RELEASE_TOKEN PAT (without it, PRs created by automation do not trigger CI).
  • test-e2e.yml no longer has a manual workflow_dispatch trigger; if a manual knob is wanted for the remaining jobs, it can be added as a follow-up.
  • The example agent is intentionally capability-minimal (read-only filesystem tools, no shell) because PR diffs are untrusted input and yolo defaults to true; if a more capable example seems preferable, it can be extended with an explicit warning instead.

Implements the public-action part of the reviewer revamp
(docker/gordon#1258, epic docker/gordon#1253).

- remove the review-pr composite action, its agents, evals and workflows
- remove Docker-internal auth: setup-credentials (AWS OIDC / GitHub App),
  the authorization waterfall and the org-membership-token, auth-org and
  skip-auth inputs; users supply their own API keys
- remove reviewer-only src modules; tsup entries reduced to main and
  signed-commit
- drop the deprecated cagent-version output
- rewrite release, e2e and version-bump workflows (single-pass dist
  staging, plain github.token with optional RELEASE_TOKEN)
- add a documented reviewer example under examples/reviewer/
- rewrite README, AGENTS.md, SECURITY.md and CONTRIBUTING.md to match

BREAKING CHANGE: the review-pr and setup-credentials sub-actions, the
reusable review-pr.yml workflow, the org-membership-token, auth-org and
skip-auth inputs and the cagent-version output are removed. Consumers
pinned to earlier tags are unaffected.

@derekmisler derekmisler left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated PR Reviewthis comment was posted by the pr-reviewer bot, not by a human or the implementer agent

Assessment: 🟡 NEEDS ATTENTION

This PR is a well-structured, internally consistent large deletion. The removal of Docker-internal auth, the review-pr orchestration, and the 16 reviewer modules is clean — no dangling references to removed inputs remain. The new generic runner core looks solid.

Two medium-severity issues were found in new code, both in the added examples/ files and CI workflows:

Issues requiring attention:

  1. Shell injection in example workflow${{ steps.review.outputs.output-file }} is interpolated directly into the run: shell script rather than passed via env:. This is the canonical GitHub Actions injection vector (GitHub Security Lab advisory). The example is copy-pasteable by users, so it sets a bad pattern.

  2. Silent CI skip on version-bump PRs — When RELEASE_TOKEN is not configured, gh pr create falls back to github.token. GitHub's security model prevents pull_request CI workflows from firing on events triggered by GITHUB_TOKEN-authenticated calls. Version-bump PRs will be created but no tests will run, and the PR can be merged without validation.

Additional notes:

  • gh pr comment (without --edit-last) posts a new comment per push. On multiple pushes the reviewer example accumulates stale comments — behavior that contradicts the README's description.
  • examples/reviewer/agent.yaml uses model claude-sonnet-4-6, which does not match any publicly documented Anthropic API model identifier (expected pattern: claude-3-5-sonnet-20241022 or short alias like claude-sonnet-4-5). Users copying this file will get an immediate API error.
  • The new release.yml no longer has the explicit gh api -X DELETE that previously ran on normal exit (the old code had both a trap for errors and an explicit call on normal exit). Now only the EXIT trap handles cleanup. Bash EXIT traps do not fire on SIGKILL (GitHub Actions job cancellation), so a cancelled release run will leak the release-staging/vX.Y.Z branch requiring manual deletion.

Comment thread examples/reviewer/review-pr.yml Outdated
env:
GH_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
run: gh pr comment "$PR_NUMBER" --body-file "${{ steps.review.outputs.output-file }}"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated PR Reviewthis comment was posted by the pr-reviewer bot, not by a human or the implementer agent

[MEDIUM] Shell injection via direct ${{ }} expression interpolation in run: block

${{ steps.review.outputs.output-file }} is resolved by the GitHub Actions runner and substituted literally into the shell script before bash sees it. A crafted value containing shell metacharacters (;, backticks, $(...)) would be executed as arbitrary shell code — this is the canonical injection vector documented by GitHub Security Lab.

The example is copy-pasteable by users; shipping it with this pattern establishes a bad baseline.

Fix: move the value into an env: variable and reference the variable in the shell:

      - name: Post review comment
        env:
          GH_TOKEN: ${{ github.token }}
          PR_NUMBER: ${{ github.event.pull_request.number }}
          OUTPUT_FILE: ${{ steps.review.outputs.output-file }}
        run: gh pr comment "$PR_NUMBER" --body-file "$OUTPUT_FILE"

env:
GH_TOKEN: ${{ env.GITHUB_APP_TOKEN }}
# optional PAT so created PRs trigger CI; falls back to github.token
GH_TOKEN: ${{ secrets.RELEASE_TOKEN || github.token }}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated PR Reviewthis comment was posted by the pr-reviewer bot, not by a human or the implementer agent

[MEDIUM] github.token fallback silently disables CI on auto-generated version-bump PRs

When RELEASE_TOKEN is not configured, gh pr create runs with GH_TOKEN: ${{ secrets.RELEASE_TOKEN || github.token }}. GitHub's security model explicitly prevents pull_request workflow runs from being triggered by events caused by a GITHUB_TOKEN-authenticated API call (to avoid recursive loops). As a result, version-bump PRs created without RELEASE_TOKEN will be created successfully but will skip all CI checks (test.yml, etc.), and can be merged without validation.

The comment on the preceding line acknowledges this ("optional PAT so created PRs trigger CI; falls back to github.token") but provides no warning or failure signal when the fallback is active, so maintainers who don't configure RELEASE_TOKEN will not notice the regression until they look for missing check statuses.

Suggestion: emit a warning in the step output when falling back, e.g.:

if [ -z "${{ secrets.RELEASE_TOKEN }}" ]; then
  echo "::warning::RELEASE_TOKEN not set — created PR will not trigger pull_request CI workflows"
fi

anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
skip-summary: false

- name: Post review comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated PR Reviewthis comment was posted by the pr-reviewer bot, not by a human or the implementer agent

[LOW] gh pr comment always creates a new comment — stale reviews accumulate on repeated pushes

The workflow triggers on [opened, synchronize, reopened]. gh pr comment (without --edit-last) unconditionally creates a new comment each time. On a PR with five pushes, five separate review comments accumulate and the earlier ones become stale. The example README describes the action as posting "the result as a PR comment" (singular), which users will interpret as update-in-place.

Suggestion: use gh pr comment --edit-last to update the most recent bot comment, or look up and delete the previous bot comment before posting. If --edit-last is used, an initial comment creation can be handled with || gh pr comment "$PR_NUMBER" --body-file "$OUTPUT_FILE" as a fallback.

models:
sonnet:
provider: anthropic
model: claude-sonnet-4-6

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated PR Reviewthis comment was posted by the pr-reviewer bot, not by a human or the implementer agent

[LOW] Model ID claude-sonnet-4-6 does not match any known Anthropic API identifier

Anthropic model IDs follow documented patterns such as claude-3-5-sonnet-20241022, claude-3-7-sonnet-20250219, or the newer short aliases like claude-sonnet-4-5. The string claude-sonnet-4-6 is not publicly documented. If Docker Agent passes this string verbatim to the Anthropic API, the request will be rejected with a model-not-found error at runtime, giving users a confusing failure on a copy-pasted example.

Suggestion: verify the intended model and update to a currently valid identifier (e.g. claude-sonnet-4-5 if that is what was intended), or clearly mark it as a placeholder that must be replaced before use.

@Sayt-0 Sayt-0 requested a review from derekmisler July 10, 2026 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants