Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Check out all text files with LF on every platform. Tests compare bytes
# from checked-out files (skill templates, snapshots); a CRLF working tree
# (core.autocrlf on Windows) broke those comparisons.
* text=auto eol=lf

*.png binary
4 changes: 3 additions & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,9 @@ agree on the approach before you invest time — see CONTRIBUTING.md.

## Related issue

<!-- e.g. "Fixes #123" or "Refs #123". Link an issue for non-trivial changes. -->
<!-- e.g. "Closes #123". Required for features and behavior changes — open an
issue first and get it assigned (comment `/assign` there); see
CONTRIBUTING.md → Contribution model. Docs and small fixes don't need one. -->

## Type of change

Expand Down
129 changes: 129 additions & 0 deletions .github/workflows/ci-nudge.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
# "Fix this and we merge" nudge (InsForge `agent-zhang-beihai` pattern, part 3).
# When a PR's CI finishes red, posts ONE sticky comment listing exactly which
# jobs failed and the local one-liner that reproduces/fixes each (the automated
# version of the hand-written "run `npm run format` and you're green" review
# comments). The comment flips to a green confirmation once all checks pass.
#
# Runs in base-repo context via workflow_run — no PR code is checked out, so
# fork PRs are safe. State is recomputed from check-runs each time, so the two
# CI workflows (CI + Test Coverage) can complete in any order.
#
# Bot identity: same App-token-first / GITHUB_TOKEN-fallback pattern as
# pr-triage.yml (the App needs the "Pull requests: Read & write" permission to
# post as testsprite-hob[bot]; until then comments come from github-actions[bot]).
name: CI failure nudge

on:
workflow_run:
workflows: ['CI', 'Test Coverage']
types: [completed]
Comment on lines +16 to +19

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

git ls-files .github/workflows/ci-nudge.yml
echo '---'
ast-grep outline .github/workflows/ci-nudge.yml --view expanded
echo '---'
cat -n .github/workflows/ci-nudge.yml | sed -n '1,220p'

Repository: TestSprite/testsprite-cli

Length of output: 7468


Add a concurrency group to serialize nudge runs.
Both workflow_run triggers can finish close together, and the second run can still see no sticky comment before the first one writes it, creating duplicate nudge comments. Use a per-SHA group like ci-nudge-${{ github.event.workflow_run.head_sha }}.

🧰 Tools
🪛 zizmor (1.26.1)

[error] 16-19: use of fundamentally insecure workflow trigger (dangerous-triggers): workflow_run is almost always used insecurely

(dangerous-triggers)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/ci-nudge.yml around lines 16 - 19, Add a top-level
concurrency group to the workflow using the workflow_run commit SHA,
specifically ci-nudge-${{ github.event.workflow_run.head_sha }}, so overlapping
nudge runs for the same commit are serialized.


permissions:
checks: read # read the head SHA's check-run state
pull-requests: write
issues: write # PR comments ride the issues API

env:
MARKER: '<!-- hob:ci-nudge -->'

jobs:
nudge:
# Public repo only; PR-triggered runs only (pushes to main have no PR to nudge).
if: >-
github.repository == 'TestSprite/testsprite-cli' &&
github.event.workflow_run.event == 'pull_request'
runs-on: ubuntu-latest
steps:
- id: app-token
continue-on-error: true
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: ${{ secrets.TESTSPRITE_HOB_APP_ID || secrets.ALFHEIM_AGENT_APP_ID }}
private-key: ${{ secrets.TESTSPRITE_HOB_PRIVATE_KEY || secrets.ALFHEIM_AGENT_PRIVATE_KEY }}
Comment on lines +37 to +42

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n## File list\n'
git ls-files .github/workflows/ci-nudge.yml

printf '\n## Relevant lines from workflow\n'
nl -ba .github/workflows/ci-nudge.yml | sed -n '1,140p'

printf '\n## Search for app-token usage and concurrency in workflow\n'
rg -n "create-github-app-token|concurrency|permission-" .github/workflows/ci-nudge.yml

Repository: TestSprite/testsprite-cli

Length of output: 281


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n## Workflow lines\n'
sed -n '1,160p' .github/workflows/ci-nudge.yml | cat -n

printf '\n## Search for concurrency / app-token inputs\n'
rg -n "concurrency|create-github-app-token|permission-" .github/workflows/ci-nudge.yml

Repository: TestSprite/testsprite-cli

Length of output: 7548


🌐 Web query:

actions/create-github-app-token permission-* inputs inherit all installation permissions recommend explicitly list permissions

💡 Result:

When using the actions/create-github-app-token action, the generated installation access token inherits all permissions granted to the GitHub App installation by default [1][2]. To improve security and adhere to the principle of least privilege, it is strongly recommended to explicitly list only the permissions required for your specific use case [1][3]. You can restrict the token's permissions by using inputs with the permission- prefix followed by the permission name (e.g., permission-pull-requests: read) [1][3][4]. Key considerations include: - Purpose of Explicit Listing: By explicitly defining permissions, you limit the scope of the generated token. This follows GitHub's general best practice for controlling permissions in workflows [1][3]. - Mechanism: The action uses these permission-* inputs to communicate with the GitHub API endpoint POST /app/installations/{installation_id}/access_tokens, which accepts a permissions parameter to generate a scoped token [5][2]. - Important Constraints: - You can only grant permissions that have already been granted to the GitHub App installation itself [1][2]. Attempting to request a permission that the installation does not possess will result in an error [1][2]. - If you encounter issues (such as 403 or 422 errors) after defining specific permissions, ensure that all permissions necessary for your workflow tasks are included in your inputs [6]. If a token with default (all) permissions works but a restricted one fails, you may need to incrementally add permissions to identify which specific one is missing [6]. - Best Practice: Start by identifying the minimal set of permissions required for your job's API calls and configure those explicitly [1][6]. Using the permission-* inputs also benefits from the action runner's built-in type intelligence and input validation [1][3].

Citations:


🌐 Web query:

GitHub App issue comments permission issues write pull requests permission required for PR issue comments

💡 Result:

To comment on pull requests (PRs) using a GitHub App, you typically only need the Issues: write permission, not the Pull Requests: write permission [1][2]. This confusion arises because, while every pull request is an issue in the GitHub data model, they are managed under distinct permission scopes [1][3]. Standard comments on the main conversation thread of a pull request are considered "issue comments" and are handled by the Issues API endpoints [4][2][3]. Therefore, the Issues: write permission is sufficient to create, edit, or delete these comments [1][2]. The Pull Requests: write permission is generally reserved for actions that are specific to the pull request lifecycle, such as: - Submitting pull request reviews (inline code comments) [2][5]. - Requesting reviewers [5]. - Updating the pull request branch or managing merge status. If you are receiving a 403 "Resource not accessible by integration" error when posting a comment on a pull request, ensure your GitHub App has been granted Issues: write access [1]. If your app is performing other operations (like submitting reviews), you may need Pull Requests: write, but for general commenting, Issues: write is the correct scope [4][1][2].

Citations:


Scope the App token to issues:write.

create-github-app-token inherits the installation’s full permissions by default. This step only creates/updates PR thread comments via the issues API, so permission-issues: write is enough; permission-pull-requests: write isn’t needed.

Proposed fix
       - id: app-token
         continue-on-error: true
         uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
         with:
           app-id: ${{ secrets.TESTSPRITE_HOB_APP_ID || secrets.ALFHEIM_AGENT_APP_ID }}
           private-key: ${{ secrets.TESTSPRITE_HOB_PRIVATE_KEY || secrets.ALFHEIM_AGENT_PRIVATE_KEY }}
+          permission-issues: write
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- id: app-token
continue-on-error: true
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: ${{ secrets.TESTSPRITE_HOB_APP_ID || secrets.ALFHEIM_AGENT_APP_ID }}
private-key: ${{ secrets.TESTSPRITE_HOB_PRIVATE_KEY || secrets.ALFHEIM_AGENT_PRIVATE_KEY }}
- id: app-token
continue-on-error: true
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: ${{ secrets.TESTSPRITE_HOB_APP_ID || secrets.ALFHEIM_AGENT_APP_ID }}
private-key: ${{ secrets.TESTSPRITE_HOB_PRIVATE_KEY || secrets.ALFHEIM_AGENT_PRIVATE_KEY }}
permission-issues: write
🧰 Tools
🪛 zizmor (1.26.1)

[error] 39-39: dangerous use of GitHub App tokens (github-app): app token inherits blanket installation permissions

(github-app)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/ci-nudge.yml around lines 37 - 42, Restrict the token
generated by the `app-token` step to only the required Issues permission by
adding `permission-issues: write` in its `with` configuration; do not grant
pull-request or other unnecessary permissions.

Sources: Path instructions, Linters/SAST tools


- uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7.0.1
env:
APP_TOKEN: ${{ steps.app-token.outputs.token }}
with:
script: |
const { owner, repo } = context.repo;
const run = context.payload.workflow_run;
const marker = process.env.MARKER;

const appClient = process.env.APP_TOKEN
? require('@actions/github').getOctokit(process.env.APP_TOKEN)
: null;
async function write(fn) {
if (appClient) {
try { return await fn(appClient.rest); }
catch (e) { if (e.status !== 403 && e.status !== 404) throw e; }
}
return await fn(github.rest);
}

// Resolve the PR for this run. `workflow_run.pull_requests` is empty
// for fork PRs, so fall back to the commit→PRs lookup.
let pr = (run.pull_requests || [])[0];
if (!pr) {
const { data } = await github.rest.repos.listPullRequestsAssociatedWithCommit(
{ owner, repo, commit_sha: run.head_sha });
pr = data.find(p => p.state === 'open');
} else {
pr = (await github.rest.pulls.get({ owner, repo, pull_number: pr.number })).data;
}
if (!pr || pr.state !== 'open' || pr.user.type === 'Bot') return;

// Job name → the local command that reproduces/fixes it. Also the
// allowlist of CI jobs this nudge watches — keep in sync with ci.yml.
const FIX = {
'Lint & Format': 'run `npm run lint:fix && npm run format`, then commit',
'Typecheck': 'run `npm run typecheck` and fix the reported type errors',
'Unit Tests': 'run `npm test` and fix the failing tests',
'Build': 'run `npm run build` and fix the compile errors',
'Local E2E Tests': 'run `npm run test:e2e` (it builds first)',
'Coverage (>= 80%)': 'run `npm run test:coverage` — new code needs tests until every metric is back at 80%',
};

// Recompute full CI state from this SHA's check runs, narrowed to the
// CI jobs above — other workflows (e.g. pr-triage) also attach
// github-actions check runs to the PR head and must not count here.
const checks = await github.paginate(github.rest.checks.listForRef,
{ owner, repo, ref: run.head_sha, per_page: 100 });
const ours = checks.filter(c => c.app && c.app.slug === 'github-actions' && FIX[c.name]);
const failing = ours.filter(c => ['failure', 'timed_out'].includes(c.conclusion));
const pending = ours.filter(c => c.status !== 'completed');

const comments = await github.paginate(github.rest.issues.listComments,
{ owner, repo, issue_number: pr.number, per_page: 100 });
const sticky = comments.find(c => (c.body || '').includes(marker));

if (failing.length === 0) {
// Only speak up on success if we previously flagged a failure, and
// only once everything has actually finished.
if (sticky && pending.length === 0 && !sticky.body.includes('all green')) {
await write(rest => rest.issues.updateComment({ owner, repo, comment_id: sticky.id,
body: `${marker}\n✅ CI is **all green** now — thanks, @${pr.user.login}!` }));
}
return;
}

const lines = failing
.sort((a, b) => a.name.localeCompare(b.name))
.map(c => {
const fix = FIX[c.name] || 'see the logs for details';
return `- **${c.name}** — ${fix} ([logs](${c.html_url}))`;
});
const body = `${marker}\nThanks, @${pr.user.login}! CI is red on this PR — `
+ `here's what failed and how to reproduce it locally:\n\n${lines.join('\n')}\n\n`
+ `Everything runs on Node 22 after \`npm ci\`. Push a fix and this comment `
+ `flips green automatically once all checks pass.`;

if (sticky) {
if (sticky.body !== body) {
await write(rest => rest.issues.updateComment(
{ owner, repo, comment_id: sticky.id, body }));
}
} else {
await write(rest => rest.issues.createComment(
{ owner, repo, issue_number: pr.number, body }));
}
23 changes: 14 additions & 9 deletions .github/workflows/issue-triage.yml
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
# P1-2 issue auto-assign + 3-slot-cap bot (InsForge `agent-zhang-beihai` pattern).
# Posts as the `alfheim-agent` GitHub App ⇒ `alfheim-agent[bot]`.
# Posts as the `testsprite-hob` GitHub App ⇒ `testsprite-hob[bot]`.
#
# Setup (one-time, by an org admin):
# 1. Create a GitHub App named `alfheim-agent` (org Settings → Developer settings →
# 1. Create a GitHub App named `testsprite-hob` (org Settings → Developer settings →
# GitHub Apps → New). Permissions: Issues = Read & write, Metadata = Read. No webhook.
# Generate a private key; install the App on the public testsprite-cli repo.
# 2. Add two repo (or org) secrets:
# ALFHEIM_AGENT_APP_ID = the App's numeric App ID
# ALFHEIM_AGENT_PRIVATE_KEY = the App's .pem private key (full contents)
# TESTSPRITE_HOB_APP_ID = the App's numeric App ID
# TESTSPRITE_HOB_PRIVATE_KEY = the App's .pem private key (full contents)
# (The ALFHEIM_AGENT_* fallbacks are this App's original secret names from
# before it was renamed — same App ID + key; either naming works.)
# Until those exist, the bot gracefully falls back to github-actions[bot] (still works).
#
# Fires on each new issue comment; assigns the commenter when they claim an issue
Expand All @@ -30,23 +32,26 @@ env:
jobs:
triage:
# issues only (issue_comment also fires on PRs), and never react to a bot's own
# comment — incl. our own alfheim-agent[bot], which (unlike GITHUB_TOKEN) would
# comment — incl. our own testsprite-hob[bot], which (unlike GITHUB_TOKEN) would
# otherwise re-trigger this workflow. `type == 'Bot'` covers both bot identities.
if: ${{ !github.event.issue.pull_request && github.event.comment.user.type != 'Bot' }}
runs-on: ubuntu-latest
steps:
# Mint a token for the alfheim-agent App so comments post as alfheim-agent[bot].
# Mint a token for the testsprite-hob App so comments post as testsprite-hob[bot].
# continue-on-error: until the App + secrets exist this no-ops and we fall back below.
- id: app-token
continue-on-error: true
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: ${{ secrets.ALFHEIM_AGENT_APP_ID }}
private-key: ${{ secrets.ALFHEIM_AGENT_PRIVATE_KEY }}
app-id: ${{ secrets.TESTSPRITE_HOB_APP_ID || secrets.ALFHEIM_AGENT_APP_ID }}
private-key: ${{ secrets.TESTSPRITE_HOB_PRIVATE_KEY || secrets.ALFHEIM_AGENT_PRIVATE_KEY }}
Comment thread
coderabbitai[bot] marked this conversation as resolved.
# Scope the minted token to what this job uses (issues API only) instead
# of inheriting the App's full installation permissions.
permission-issues: write

- uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7.0.1
with:
# App token when available (→ alfheim-agent[bot]); else the default
# App token when available (→ testsprite-hob[bot]); else the default
# GITHUB_TOKEN (→ github-actions[bot]). Either way the logic is identical.
github-token: ${{ steps.app-token.outputs.token || github.token }}
script: |
Expand Down
Loading