-
Notifications
You must be signed in to change notification settings - Fork 90
release: v0.3.0 #230
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
release: v0.3.0 #230
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| # Check out all text files with LF on every platform. Tests compare bytes | ||
| # from checked-out files (skill templates, snapshots); a CRLF working tree | ||
| # (core.autocrlf on Windows) broke those comparisons. | ||
| * text=auto eol=lf | ||
|
|
||
| *.png binary |
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,129 @@ | ||||||||||||||||||||||||||||
| # "Fix this and we merge" nudge (InsForge `agent-zhang-beihai` pattern, part 3). | ||||||||||||||||||||||||||||
| # When a PR's CI finishes red, posts ONE sticky comment listing exactly which | ||||||||||||||||||||||||||||
| # jobs failed and the local one-liner that reproduces/fixes each (the automated | ||||||||||||||||||||||||||||
| # version of the hand-written "run `npm run format` and you're green" review | ||||||||||||||||||||||||||||
| # comments). The comment flips to a green confirmation once all checks pass. | ||||||||||||||||||||||||||||
| # | ||||||||||||||||||||||||||||
| # Runs in base-repo context via workflow_run — no PR code is checked out, so | ||||||||||||||||||||||||||||
| # fork PRs are safe. State is recomputed from check-runs each time, so the two | ||||||||||||||||||||||||||||
| # CI workflows (CI + Test Coverage) can complete in any order. | ||||||||||||||||||||||||||||
| # | ||||||||||||||||||||||||||||
| # Bot identity: same App-token-first / GITHUB_TOKEN-fallback pattern as | ||||||||||||||||||||||||||||
| # pr-triage.yml (the App needs the "Pull requests: Read & write" permission to | ||||||||||||||||||||||||||||
| # post as testsprite-hob[bot]; until then comments come from github-actions[bot]). | ||||||||||||||||||||||||||||
| name: CI failure nudge | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| on: | ||||||||||||||||||||||||||||
| workflow_run: | ||||||||||||||||||||||||||||
| workflows: ['CI', 'Test Coverage'] | ||||||||||||||||||||||||||||
| types: [completed] | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| permissions: | ||||||||||||||||||||||||||||
| checks: read # read the head SHA's check-run state | ||||||||||||||||||||||||||||
| pull-requests: write | ||||||||||||||||||||||||||||
| issues: write # PR comments ride the issues API | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| env: | ||||||||||||||||||||||||||||
| MARKER: '<!-- hob:ci-nudge -->' | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| jobs: | ||||||||||||||||||||||||||||
| nudge: | ||||||||||||||||||||||||||||
| # Public repo only; PR-triggered runs only (pushes to main have no PR to nudge). | ||||||||||||||||||||||||||||
| if: >- | ||||||||||||||||||||||||||||
| github.repository == 'TestSprite/testsprite-cli' && | ||||||||||||||||||||||||||||
| github.event.workflow_run.event == 'pull_request' | ||||||||||||||||||||||||||||
| runs-on: ubuntu-latest | ||||||||||||||||||||||||||||
| steps: | ||||||||||||||||||||||||||||
| - id: app-token | ||||||||||||||||||||||||||||
| continue-on-error: true | ||||||||||||||||||||||||||||
| uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 | ||||||||||||||||||||||||||||
| with: | ||||||||||||||||||||||||||||
| app-id: ${{ secrets.TESTSPRITE_HOB_APP_ID || secrets.ALFHEIM_AGENT_APP_ID }} | ||||||||||||||||||||||||||||
| private-key: ${{ secrets.TESTSPRITE_HOB_PRIVATE_KEY || secrets.ALFHEIM_AGENT_PRIVATE_KEY }} | ||||||||||||||||||||||||||||
|
Comment on lines
+37
to
+42
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '\n## File list\n'
git ls-files .github/workflows/ci-nudge.yml
printf '\n## Relevant lines from workflow\n'
nl -ba .github/workflows/ci-nudge.yml | sed -n '1,140p'
printf '\n## Search for app-token usage and concurrency in workflow\n'
rg -n "create-github-app-token|concurrency|permission-" .github/workflows/ci-nudge.ymlRepository: TestSprite/testsprite-cli Length of output: 281 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '\n## Workflow lines\n'
sed -n '1,160p' .github/workflows/ci-nudge.yml | cat -n
printf '\n## Search for concurrency / app-token inputs\n'
rg -n "concurrency|create-github-app-token|permission-" .github/workflows/ci-nudge.ymlRepository: TestSprite/testsprite-cli Length of output: 7548 🌐 Web query:
💡 Result: When using the Citations:
🌐 Web query:
💡 Result: To comment on pull requests (PRs) using a GitHub App, you typically only need the Issues: write permission, not the Pull Requests: write permission [1][2]. This confusion arises because, while every pull request is an issue in the GitHub data model, they are managed under distinct permission scopes [1][3]. Standard comments on the main conversation thread of a pull request are considered "issue comments" and are handled by the Issues API endpoints [4][2][3]. Therefore, the Issues: write permission is sufficient to create, edit, or delete these comments [1][2]. The Pull Requests: write permission is generally reserved for actions that are specific to the pull request lifecycle, such as: - Submitting pull request reviews (inline code comments) [2][5]. - Requesting reviewers [5]. - Updating the pull request branch or managing merge status. If you are receiving a 403 "Resource not accessible by integration" error when posting a comment on a pull request, ensure your GitHub App has been granted Issues: write access [1]. If your app is performing other operations (like submitting reviews), you may need Pull Requests: write, but for general commenting, Issues: write is the correct scope [4][1][2]. Citations:
Scope the App token to
Proposed fix - id: app-token
continue-on-error: true
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: ${{ secrets.TESTSPRITE_HOB_APP_ID || secrets.ALFHEIM_AGENT_APP_ID }}
private-key: ${{ secrets.TESTSPRITE_HOB_PRIVATE_KEY || secrets.ALFHEIM_AGENT_PRIVATE_KEY }}
+ permission-issues: write📝 Committable suggestion
Suggested change
🧰 Tools🪛 zizmor (1.26.1)[error] 39-39: dangerous use of GitHub App tokens (github-app): app token inherits blanket installation permissions (github-app) 🤖 Prompt for AI AgentsSources: Path instructions, Linters/SAST tools |
||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| - uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7.0.1 | ||||||||||||||||||||||||||||
| env: | ||||||||||||||||||||||||||||
| APP_TOKEN: ${{ steps.app-token.outputs.token }} | ||||||||||||||||||||||||||||
| with: | ||||||||||||||||||||||||||||
| script: | | ||||||||||||||||||||||||||||
| const { owner, repo } = context.repo; | ||||||||||||||||||||||||||||
| const run = context.payload.workflow_run; | ||||||||||||||||||||||||||||
| const marker = process.env.MARKER; | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| const appClient = process.env.APP_TOKEN | ||||||||||||||||||||||||||||
| ? require('@actions/github').getOctokit(process.env.APP_TOKEN) | ||||||||||||||||||||||||||||
| : null; | ||||||||||||||||||||||||||||
| async function write(fn) { | ||||||||||||||||||||||||||||
| if (appClient) { | ||||||||||||||||||||||||||||
| try { return await fn(appClient.rest); } | ||||||||||||||||||||||||||||
| catch (e) { if (e.status !== 403 && e.status !== 404) throw e; } | ||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||
| return await fn(github.rest); | ||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| // Resolve the PR for this run. `workflow_run.pull_requests` is empty | ||||||||||||||||||||||||||||
| // for fork PRs, so fall back to the commit→PRs lookup. | ||||||||||||||||||||||||||||
| let pr = (run.pull_requests || [])[0]; | ||||||||||||||||||||||||||||
| if (!pr) { | ||||||||||||||||||||||||||||
| const { data } = await github.rest.repos.listPullRequestsAssociatedWithCommit( | ||||||||||||||||||||||||||||
| { owner, repo, commit_sha: run.head_sha }); | ||||||||||||||||||||||||||||
| pr = data.find(p => p.state === 'open'); | ||||||||||||||||||||||||||||
| } else { | ||||||||||||||||||||||||||||
| pr = (await github.rest.pulls.get({ owner, repo, pull_number: pr.number })).data; | ||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||
| if (!pr || pr.state !== 'open' || pr.user.type === 'Bot') return; | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| // Job name → the local command that reproduces/fixes it. Also the | ||||||||||||||||||||||||||||
| // allowlist of CI jobs this nudge watches — keep in sync with ci.yml. | ||||||||||||||||||||||||||||
| const FIX = { | ||||||||||||||||||||||||||||
| 'Lint & Format': 'run `npm run lint:fix && npm run format`, then commit', | ||||||||||||||||||||||||||||
| 'Typecheck': 'run `npm run typecheck` and fix the reported type errors', | ||||||||||||||||||||||||||||
| 'Unit Tests': 'run `npm test` and fix the failing tests', | ||||||||||||||||||||||||||||
| 'Build': 'run `npm run build` and fix the compile errors', | ||||||||||||||||||||||||||||
| 'Local E2E Tests': 'run `npm run test:e2e` (it builds first)', | ||||||||||||||||||||||||||||
| 'Coverage (>= 80%)': 'run `npm run test:coverage` — new code needs tests until every metric is back at 80%', | ||||||||||||||||||||||||||||
| }; | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| // Recompute full CI state from this SHA's check runs, narrowed to the | ||||||||||||||||||||||||||||
| // CI jobs above — other workflows (e.g. pr-triage) also attach | ||||||||||||||||||||||||||||
| // github-actions check runs to the PR head and must not count here. | ||||||||||||||||||||||||||||
| const checks = await github.paginate(github.rest.checks.listForRef, | ||||||||||||||||||||||||||||
| { owner, repo, ref: run.head_sha, per_page: 100 }); | ||||||||||||||||||||||||||||
| const ours = checks.filter(c => c.app && c.app.slug === 'github-actions' && FIX[c.name]); | ||||||||||||||||||||||||||||
| const failing = ours.filter(c => ['failure', 'timed_out'].includes(c.conclusion)); | ||||||||||||||||||||||||||||
| const pending = ours.filter(c => c.status !== 'completed'); | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| const comments = await github.paginate(github.rest.issues.listComments, | ||||||||||||||||||||||||||||
| { owner, repo, issue_number: pr.number, per_page: 100 }); | ||||||||||||||||||||||||||||
| const sticky = comments.find(c => (c.body || '').includes(marker)); | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| if (failing.length === 0) { | ||||||||||||||||||||||||||||
| // Only speak up on success if we previously flagged a failure, and | ||||||||||||||||||||||||||||
| // only once everything has actually finished. | ||||||||||||||||||||||||||||
| if (sticky && pending.length === 0 && !sticky.body.includes('all green')) { | ||||||||||||||||||||||||||||
| await write(rest => rest.issues.updateComment({ owner, repo, comment_id: sticky.id, | ||||||||||||||||||||||||||||
| body: `${marker}\n✅ CI is **all green** now — thanks, @${pr.user.login}!` })); | ||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||
| return; | ||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| const lines = failing | ||||||||||||||||||||||||||||
| .sort((a, b) => a.name.localeCompare(b.name)) | ||||||||||||||||||||||||||||
| .map(c => { | ||||||||||||||||||||||||||||
| const fix = FIX[c.name] || 'see the logs for details'; | ||||||||||||||||||||||||||||
| return `- **${c.name}** — ${fix} ([logs](${c.html_url}))`; | ||||||||||||||||||||||||||||
| }); | ||||||||||||||||||||||||||||
| const body = `${marker}\nThanks, @${pr.user.login}! CI is red on this PR — ` | ||||||||||||||||||||||||||||
| + `here's what failed and how to reproduce it locally:\n\n${lines.join('\n')}\n\n` | ||||||||||||||||||||||||||||
| + `Everything runs on Node 22 after \`npm ci\`. Push a fix and this comment ` | ||||||||||||||||||||||||||||
| + `flips green automatically once all checks pass.`; | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| if (sticky) { | ||||||||||||||||||||||||||||
| if (sticky.body !== body) { | ||||||||||||||||||||||||||||
| await write(rest => rest.issues.updateComment( | ||||||||||||||||||||||||||||
| { owner, repo, comment_id: sticky.id, body })); | ||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||
| } else { | ||||||||||||||||||||||||||||
| await write(rest => rest.issues.createComment( | ||||||||||||||||||||||||||||
| { owner, repo, issue_number: pr.number, body })); | ||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
Repository: TestSprite/testsprite-cli
Length of output: 7468
Add a
concurrencygroup to serialize nudge runs.Both
workflow_runtriggers can finish close together, and the second run can still see no sticky comment before the first one writes it, creating duplicate nudge comments. Use a per-SHA group likeci-nudge-${{ github.event.workflow_run.head_sha }}.🧰 Tools
🪛 zizmor (1.26.1)
[error] 16-19: use of fundamentally insecure workflow trigger (dangerous-triggers): workflow_run is almost always used insecurely
(dangerous-triggers)
🤖 Prompt for AI Agents