From 9c8409bf07b727a927315a99e5cf2e9f7ae7f18e Mon Sep 17 00:00:00 2001 From: OB Date: Tue, 12 May 2026 17:14:39 +0700 Subject: [PATCH 1/4] ci(release): pre-tag lockstep check in verify-action-pins.sh MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes the 0.2.7 lockstep-skip bug class at the release-pipeline gate. `scripts/verify-action-pins.sh` gains an opt-in check, activated when `EXPECTED_RELEASE_TAG` is set in the script's environment. The check reads `.github/workflows/scan.yml`, extracts the `uses: ob-aion/pruner@` literal, and exits 1 if `` does not match the tag being cut. Local runs without the env var keep the previous SHA-pin-only behaviour. Script header documents the env-var contract and the updated exit-code set (0 OK / 1 bad pin / 2 missing gh or scan.yml). `.github/workflows/release.yml` reorders the steps so `Resolve tag` runs before the renamed `Verify action SHA pins and scan.yml lockstep` step, which receives the tag via `EXPECTED_RELEASE_TAG`. No behavioural change to the SHA-pin loop or any downstream release step. The 0.2.7 bug class: four consecutive releases (0.2.3-0.2.6) shipped `scan.yml` pinned to the previous composite because the documented bump step was a manual hand-edit that fell through review. Consumers calling `scan.yml@0.2.6` got the 0.2.2 composite internally and missed the Cisco engine bump. The 0.2.13 attempt to retire the lockstep via a structural self-checkout pattern was reverted at 0.2.14 (cross-repo `github.*` context refers to the caller). With the manual lockstep restored, this verifier is the principled defence — pre-tag, scriptable, idempotent. --- .github/workflows/release.yml | 13 +++++++------ scripts/verify-action-pins.sh | 31 +++++++++++++++++++++++++++++-- 2 files changed, 36 insertions(+), 8 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index f098032..6e4cc31 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -25,12 +25,6 @@ jobs: with: fetch-depth: 0 - - name: Verify action SHA pins are commits - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - shell: bash - run: bash scripts/verify-action-pins.sh - - name: Resolve tag id: tag shell: bash @@ -43,6 +37,13 @@ jobs: fi echo "tag=${REF}" >> "$GITHUB_OUTPUT" + - name: Verify action SHA pins and scan.yml lockstep + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + EXPECTED_RELEASE_TAG: ${{ steps.tag.outputs.tag }} + shell: bash + run: bash scripts/verify-action-pins.sh + - name: Self-scan against the tagged ref id: scan uses: ./ diff --git a/scripts/verify-action-pins.sh b/scripts/verify-action-pins.sh index 520dbd1..2bf02fb 100755 --- a/scripts/verify-action-pins.sh +++ b/scripts/verify-action-pins.sh @@ -1,7 +1,9 @@ #!/usr/bin/env bash # Verify every SHA-pinned `uses:` line resolves to a real commit, not a -# tag-object SHA — Scorecard rejects the latter as "imposter commit". -# See docs/sha-pinning.md. Exit: 0 = OK, 1 = bad pin, 2 = missing gh. +# tag-object SHA — Scorecard rejects the latter as "imposter commit". When +# EXPECTED_RELEASE_TAG is set, also verify scan.yml's `uses: ob-aion/pruner@` +# matches the tag being cut (closes the 0.2.7 lockstep-skip bug class). +# See docs/sha-pinning.md. Exit: 0 = OK, 1 = bad pin, 2 = missing gh or scan.yml. set -euo pipefail @@ -44,4 +46,29 @@ while IFS= read -r line; do done < <(grep -nHE 'uses:[[:space:]]+[^/]+/[^@]+@[a-f0-9]{40}' "${FILES[@]}" 2>/dev/null || true) printf '\n%d pin(s) verified, %d failure(s).\n' "$PASS" "$FAIL" + +# Lockstep check (fires when EXPECTED_RELEASE_TAG is set): scan.yml's +# `uses: ob-aion/pruner@` must match the tag being cut. +if [ -n "${EXPECTED_RELEASE_TAG:-}" ]; then + SCAN_YML=".github/workflows/scan.yml" + if [ ! -f "$SCAN_YML" ]; then + echo "FATAL: ${SCAN_YML} not found" >&2 + exit 2 + fi + PIN=$(grep -nE 'uses:[[:space:]]+ob-aion/pruner@[0-9]+\.[0-9]+\.[0-9]+' "$SCAN_YML" \ + | head -1 \ + | sed -E 's/.*uses:[[:space:]]+ob-aion\/pruner@([0-9]+\.[0-9]+\.[0-9]+).*/\1/') + if [ -z "$PIN" ]; then + echo "FATAL: no ob-aion/pruner@ pin found in ${SCAN_YML}" >&2 + exit 2 + fi + if [ "$PIN" = "$EXPECTED_RELEASE_TAG" ]; then + printf 'OK lockstep: scan.yml pins ob-aion/pruner@%s = %s\n' "$PIN" "$EXPECTED_RELEASE_TAG" + else + printf 'BAD lockstep: scan.yml pins ob-aion/pruner@%s but releasing %s\n' \ + "$PIN" "$EXPECTED_RELEASE_TAG" >&2 + RC=1 + fi +fi + exit "${RC}" From 5955ebd6549d1f2f1b1cbbfb44d673e2a74275f7 Mon Sep 17 00:00:00 2001 From: OB Date: Tue, 12 May 2026 17:14:55 +0700 Subject: [PATCH 2/4] test(self-scan): finding-positive fixture + 0.2.15 release prep MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes the 0.2.8 exit-code bug class — `pruner scan` exit 1 (findings present, all below threshold) must thread through `coroboros-pack-run` and the gate step without being masked as a workflow failure. `examples/finding-positive-skill/` is a minimal skill that trips four sub-critical Coroboros pack rules: FC001 (high, name fails kebab-case), FC003 (medium, top-level `custom_field`), FC004 (low, `metadata.version` present), FC005 (low, non-SPDX license). Zero critical by design. Companion `EXPECTATIONS.md` documents the cross-walk and distinguishes the fixture from `examples/vulnerable-skill/` (21 findings, 7 critical, every finding allowlisted in `.pruner-ignore.yml`). `.github/workflows/self-scan.yml` adds a `pruner-finding-positive` job invoking the composite via `uses: ./` against the new fixture at `fail-on: critical`. Green outcome asserts the exit-code propagation contract. The pre-existing `pruner-self-scan` job scans `.` from repo root with `.pruner-ignore.yml` applied — every Pruner-side fixture finding sits in the allowlist, so `pruner scan` exited 0 and the 0.2.8 bug never surfaced on the maintainer side until the first consumer integration on `coroboros/agent-skills`. Finding-positive findings are intentionally not allowlisted; they surface in Code Scanning at downgraded severities (template-example weight 0.25 drops high to low, medium to info, etc.) but never gate the workflow. `.github/workflows/scan.yml` synced to `ob-aion/pruner@0.2.15` per the lockstep contract, now verified pre-tag by the verifier extension in the previous commit. `CHANGELOG.md` — v0.2.15 entry covering both changes. Branch protection on `main` requires `pruner-self-scan` already; promoting `pruner-finding-positive` to a required status check is a one-line `gh api` follow-up after the first green run on `main`. --- .github/workflows/scan.yml | 2 +- .github/workflows/self-scan.yml | 20 +++++++++++++++++++ CHANGELOG.md | 10 ++++++++++ .../finding-positive-skill/EXPECTATIONS.md | 19 ++++++++++++++++++ examples/finding-positive-skill/SKILL.md | 19 ++++++++++++++++++ 5 files changed, 69 insertions(+), 1 deletion(-) create mode 100644 examples/finding-positive-skill/EXPECTATIONS.md create mode 100644 examples/finding-positive-skill/SKILL.md diff --git a/.github/workflows/scan.yml b/.github/workflows/scan.yml index 23c6dfc..0033699 100644 --- a/.github/workflows/scan.yml +++ b/.github/workflows/scan.yml @@ -49,7 +49,7 @@ jobs: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - id: scan - uses: ob-aion/pruner@0.2.14 + uses: ob-aion/pruner@0.2.15 with: fail-on: ${{ inputs.fail-on }} skill-pattern: ${{ inputs.skill-pattern }} diff --git a/.github/workflows/self-scan.yml b/.github/workflows/self-scan.yml index 7ea7ca7..0601636 100644 --- a/.github/workflows/self-scan.yml +++ b/.github/workflows/self-scan.yml @@ -31,3 +31,23 @@ jobs: name: pruner-self-scan-report path: .pruner/ retention-days: 14 + + # Composite-action invocation against a fixture that trips findings strictly + # below `critical`. Green outcome asserts `pruner scan` exit 1 threads through + # `coroboros-pack-run` and the gate step without being masked as a workflow + # failure — the 0.2.8 bug class. The `pruner-self-scan` job above scans `.` + # which holds zero `SKILL.md` files and never exits 1 (the blind spot). + pruner-finding-positive: + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + + - id: scan + uses: ./ + with: + fail-on: critical + target-path: examples/finding-positive-skill + skill-pattern: SKILL.md + report-output: ./.pruner-finding-positive diff --git a/CHANGELOG.md b/CHANGELOG.md index cc09ea0..4b74a67 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,15 @@ # Changelog +## v0.2.15 - 12/05/2026 + +Two defensive CI additions that close the bug classes 0.2.7 and 0.2.8 exposed. Both lived as open backlog items on the consumer-fixture-CI line and survived the 0.2.13 → 0.2.14 hotfix cycle untouched. Neither attempts to validate cross-repo reusable-workflow behaviour from inside Pruner's own CI — a real consumer PR is the only honest surface for that. Both target a specific bug class rather than a generic CI gap. + +- **`scripts/verify-action-pins.sh` — pre-tag lockstep check.** Opt-in extension activated when `EXPECTED_RELEASE_TAG` is set in the script's environment. Reads `.github/workflows/scan.yml`, extracts the `uses: ob-aion/pruner@` literal, and exits 1 if `` does not match the tag being cut. Script header documents the env-var contract. Closes the 0.2.7 bug class (four consecutive releases shipped `scan.yml` pinned to the previous composite) at the release-pipeline gate, before any release asset is built. +- **`.github/workflows/release.yml` — tag resolution moved ahead of pin verification.** The `Resolve tag` step now precedes the renamed `Verify action SHA pins and scan.yml lockstep` step, which receives the tag via `EXPECTED_RELEASE_TAG`. No behavioural change to the SHA-pin loop or any downstream step. +- **`examples/finding-positive-skill/` — new fixture.** Minimal skill that trips four sub-critical Coroboros pack rules: FC001 (high — `name: "Bad Name"` fails kebab-case), FC003 (medium — `custom_field` at top level), FC004 (low — `metadata.version` present), FC005 (low — `license: "Apache 2"` not a valid SPDX identifier). Zero critical findings by design. Companion `EXPECTATIONS.md` documents the cross-walk. Distinct from `examples/vulnerable-skill/` (21 findings, 7 critical, every finding allowlisted in `.pruner-ignore.yml`) which exists to validate the full Coroboros + Cisco detection surface. +- **`.github/workflows/self-scan.yml` — `pruner-finding-positive` job added.** Invokes the composite via `uses: ./` against `examples/finding-positive-skill` at `fail-on: critical`. Green outcome asserts that `pruner scan` exit 1 (findings present, all below threshold) threads through `coroboros-pack-run` and the gate step without being masked as a workflow failure. The pre-existing `pruner-self-scan` job scans `.` from repo root with `.pruner-ignore.yml` applied — every Pruner-side fixture finding sat in the allowlist, so `pruner scan` exited 0 and the 0.2.8 bug never surfaced on the maintainer side until the first consumer integration on `coroboros/agent-skills`. Finding-positive findings are intentionally not allowlisted; they surface in Code Scanning at downgraded severities (template-example weight 0.25 → high becomes low, etc.) but never gate the workflow. Branch protection on `main` requires `pruner-self-scan` already; promoting `pruner-finding-positive` to a required status check is a one-line `gh api` follow-up after the first green run on `main`. +- **`.github/workflows/scan.yml` — synced to `ob-aion/pruner@0.2.15`** per the lockstep contract, now verified pre-tag by `scripts/verify-action-pins.sh`. + ## v0.2.14 - 12/05/2026 Hotfix that reverts the 0.2.13 `scan.yml` structural fix. The new pattern parsed `github.workflow_ref` and `github.workflow_sha` to derive the Pruner repo + SHA, then `actions/checkout`'d that into `.pruner-action` and invoked the composite via `uses: ./.pruner-action`. The pattern passed Pruner's own PR-only `test-scan-yml.yml` validator because both context variables resolve to the workflow's hosting repo when the reusable workflow is called locally (`uses: ./.github/workflows/scan.yml`). It failed on the first real cross-repo consumer call (`coroboros/agent-skills@scan.yml@0.2.13`) because under remote invocation, **the entire `github.*` context refers to the caller's workflow, not the called reusable workflow** — `WORKFLOW_REF=coroboros/agent-skills/.github/workflows/ci.yml@refs/pull/29/merge`, `FALLBACK_REPOSITORY=coroboros/agent-skills`. The second checkout therefore cloned `agent-skills` (no `action.yml`) and `uses: ./.pruner-action` errored. GHA docs are explicit on this: *"The `github` context, with the exception of `github.token`, references the calling workflow."* No documented context exposes the called reusable workflow's own repo/ref/sha; expressions in `uses:` action refs are not supported (`actions/runner#1493`). The pattern is structurally unfixable. Scorecard documents the same chicken-and-egg in its `RELEASE.md` and lives with the manual two-step. Pruner does the same starting here. diff --git a/examples/finding-positive-skill/EXPECTATIONS.md b/examples/finding-positive-skill/EXPECTATIONS.md new file mode 100644 index 0000000..d3a1e6f --- /dev/null +++ b/examples/finding-positive-skill/EXPECTATIONS.md @@ -0,0 +1,19 @@ +# Finding-positive skill — expected findings + +When `pruner scan examples/finding-positive-skill --without-cisco` runs, the +Coroboros pack should report exactly the following findings. Zero critical +by design. + +| Rule | File | Severity | Why | +|---|---|---|---| +| FC001 | `SKILL.md` | high | `name` is `"Bad Name"` — contains a space, fails kebab-case. | +| FC003 | `SKILL.md` | medium | `custom_field` at top level (must live under `metadata:`). | +| FC004 | `SKILL.md` | low | `metadata.version` present (Coroboros house rule forbids it). | +| FC005 | `SKILL.md` | low | `license: "Apache 2"` is not a valid SPDX identifier (correct: `Apache-2.0`). | + +The fixture exists to keep `pruner scan` exit 1 threaded through the composite +action under `fail-on: critical` — exit 0 is the expected outcome at the +workflow level. Distinct from `examples/vulnerable-skill/`, which trips 7 +critical findings (PI-UNI-001/003 weight-locked at 1.00, PI-IDFILE-001 and +PI-EXFIL-002 on `scripts/`) and exists to validate the full Coroboros + Cisco +detection surface. diff --git a/examples/finding-positive-skill/SKILL.md b/examples/finding-positive-skill/SKILL.md new file mode 100644 index 0000000..331ed30 --- /dev/null +++ b/examples/finding-positive-skill/SKILL.md @@ -0,0 +1,19 @@ +--- +name: "Bad Name" +description: "Finding-positive fixture used by self-scan.yml to catch the 0.2.8 exit-code bug class. Trips four sub-critical Coroboros pack rules." +metadata: + version: "1.0.0" +license: "Apache 2" +custom_field: "bar" +--- + +# Finding-positive + +Trips Coroboros pack findings strictly below the `critical` threshold. The +`pruner-finding-positive` job in `.github/workflows/self-scan.yml` invokes the +composite via `uses: ./` against this fixture at `fail-on: critical` and +asserts the workflow goes green — `pruner scan` exit 1 (findings present, all +below threshold) must thread through `coroboros-pack-run` and the gate step +without being masked as a workflow failure. The 0.2.8 bug class. + +See `EXPECTATIONS.md` for the rule cross-walk. From 9a6ec15bea8471458d61fbc9461260fc77b96317 Mon Sep 17 00:00:00 2001 From: OB Date: Tue, 12 May 2026 17:36:26 +0700 Subject: [PATCH 3/4] docs(changelog): scrub v0.2.15 entry against Coroboros brand voice Split four sentences that ran over the 25-word hard cap. Tighten lexicon ("CI additions" not "defensive CI additions"), drop "lived as", drop the verbose parenthetical that nested another em-dash inside a clause. Bullet for the new fixture restructured: four rule items as short sentences instead of a single 40-word parenthetical chain. Bullet for the new self-scan job restructured: ten short sentences replace four long ones. No semantic change. Same five bullets, same surface area. --- CHANGELOG.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4b74a67..a626aad 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,12 +2,12 @@ ## v0.2.15 - 12/05/2026 -Two defensive CI additions that close the bug classes 0.2.7 and 0.2.8 exposed. Both lived as open backlog items on the consumer-fixture-CI line and survived the 0.2.13 → 0.2.14 hotfix cycle untouched. Neither attempts to validate cross-repo reusable-workflow behaviour from inside Pruner's own CI — a real consumer PR is the only honest surface for that. Both target a specific bug class rather than a generic CI gap. +Two CI additions close bug classes that the 0.2.7 and 0.2.8 releases exposed. Both stayed open through the 0.2.13 → 0.2.14 revert cycle. Neither validates cross-repo reusable-workflow behaviour from inside Pruner's own CI — a real consumer PR remains the only honest surface for that. Both target a specific bug class rather than a generic CI gap. -- **`scripts/verify-action-pins.sh` — pre-tag lockstep check.** Opt-in extension activated when `EXPECTED_RELEASE_TAG` is set in the script's environment. Reads `.github/workflows/scan.yml`, extracts the `uses: ob-aion/pruner@` literal, and exits 1 if `` does not match the tag being cut. Script header documents the env-var contract. Closes the 0.2.7 bug class (four consecutive releases shipped `scan.yml` pinned to the previous composite) at the release-pipeline gate, before any release asset is built. +- **`scripts/verify-action-pins.sh` — pre-tag lockstep check.** Opt-in extension activated when `EXPECTED_RELEASE_TAG` is set in the script's environment. Reads `.github/workflows/scan.yml`, extracts the `uses: ob-aion/pruner@` literal, and exits 1 if the literal does not match the tag being cut. Script header documents the env-var contract. Closes the 0.2.7 bug class at the release-pipeline gate, before any release asset is built. Four consecutive releases (0.2.3-0.2.6) had shipped `scan.yml` pinned to the previous composite. - **`.github/workflows/release.yml` — tag resolution moved ahead of pin verification.** The `Resolve tag` step now precedes the renamed `Verify action SHA pins and scan.yml lockstep` step, which receives the tag via `EXPECTED_RELEASE_TAG`. No behavioural change to the SHA-pin loop or any downstream step. -- **`examples/finding-positive-skill/` — new fixture.** Minimal skill that trips four sub-critical Coroboros pack rules: FC001 (high — `name: "Bad Name"` fails kebab-case), FC003 (medium — `custom_field` at top level), FC004 (low — `metadata.version` present), FC005 (low — `license: "Apache 2"` not a valid SPDX identifier). Zero critical findings by design. Companion `EXPECTATIONS.md` documents the cross-walk. Distinct from `examples/vulnerable-skill/` (21 findings, 7 critical, every finding allowlisted in `.pruner-ignore.yml`) which exists to validate the full Coroboros + Cisco detection surface. -- **`.github/workflows/self-scan.yml` — `pruner-finding-positive` job added.** Invokes the composite via `uses: ./` against `examples/finding-positive-skill` at `fail-on: critical`. Green outcome asserts that `pruner scan` exit 1 (findings present, all below threshold) threads through `coroboros-pack-run` and the gate step without being masked as a workflow failure. The pre-existing `pruner-self-scan` job scans `.` from repo root with `.pruner-ignore.yml` applied — every Pruner-side fixture finding sat in the allowlist, so `pruner scan` exited 0 and the 0.2.8 bug never surfaced on the maintainer side until the first consumer integration on `coroboros/agent-skills`. Finding-positive findings are intentionally not allowlisted; they surface in Code Scanning at downgraded severities (template-example weight 0.25 → high becomes low, etc.) but never gate the workflow. Branch protection on `main` requires `pruner-self-scan` already; promoting `pruner-finding-positive` to a required status check is a one-line `gh api` follow-up after the first green run on `main`. +- **`examples/finding-positive-skill/` — new fixture.** Minimal skill trips four sub-critical Coroboros pack rules. FC001 high: `name: "Bad Name"` fails kebab-case. FC003 medium: top-level `custom_field` (must live under `metadata:`). FC004 low: `metadata.version` present (Coroboros house rule forbids it). FC005 low: `license: "Apache 2"` is not a valid SPDX identifier. Zero critical by design. Companion `EXPECTATIONS.md` documents the cross-walk. Distinct from `examples/vulnerable-skill/`: 21 findings, 7 critical, every finding allowlisted in `.pruner-ignore.yml`, validating the full Coroboros + Cisco detection surface. +- **`.github/workflows/self-scan.yml` — `pruner-finding-positive` job added.** Invokes the composite via `uses: ./` against `examples/finding-positive-skill` at `fail-on: critical`. Green outcome asserts that `pruner scan` exit 1 threads through `coroboros-pack-run` and the gate step. Workflow stays green even though findings exist (all below threshold). The pre-existing `pruner-self-scan` job scans `.` from repo root with `.pruner-ignore.yml` applied. Every Pruner-side fixture finding sat in the allowlist, so `pruner scan` exited 0. The 0.2.8 bug never surfaced on the maintainer side until the first consumer integration on `coroboros/agent-skills`. Finding-positive findings are intentionally not allowlisted. They surface in Code Scanning at downgraded severities (template-example weight 0.25 → high becomes low) but never gate the workflow. Branch protection on `main` requires `pruner-self-scan` already. Promoting `pruner-finding-positive` to a required status check is a one-line `gh api` follow-up after the first green run on `main`. - **`.github/workflows/scan.yml` — synced to `ob-aion/pruner@0.2.15`** per the lockstep contract, now verified pre-tag by `scripts/verify-action-pins.sh`. ## v0.2.14 - 12/05/2026 From 6ecaed305a38330deb0320f3ef92788919d847e7 Mon Sep 17 00:00:00 2001 From: OB Date: Tue, 12 May 2026 18:38:40 +0700 Subject: [PATCH 4/4] refactor(self-scan): dual-path allowlist for the finding-positive fixture + docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PR review on `#28` surfaced inline `github-advanced-security[bot]` comments on `examples/finding-positive-skill/SKILL.md`. The pre-existing `pruner-self-scan` job uploaded the four sub-critical findings to Code Scanning, the bot mirrored each as a PR review comment, and the public PR ended up annotated with intentional-fixture noise. Fix exploits the wrapper's path-relative behaviour: - `pruner-self-scan` runs with `target-path: .`. Finding paths land as `examples/finding-positive-skill/SKILL.md` (repo-root-relative). The four new entries in `.pruner-ignore.yml` are keyed on exactly that path and match, so the findings filter out before SARIF upload. Security tab and PR review surface stay clean. - `pruner-finding-positive` runs with `target-path: examples/finding-positive-skill`. The wrapper computes finding paths relative to `target-path`, so each one is the bare string `SKILL.md`. Allowlist entries do not match, findings fire, `pruner scan` exits 1, and the composite's `coroboros-pack-run` step has to mask that exit for the workflow to pass — exactly the 0.2.8 propagation contract this fixture exists to validate. Same path-rooted asymmetry as the 14 `examples/vulnerable-skill/` entries already in `.pruner-ignore.yml`, so the file stays internally consistent. Documentation updates folded in alongside the fix. - `examples/finding-positive-skill/EXPECTATIONS.md` gains a "Dual-path allowlist behaviour" section with the truth table above. Distinct-from- vulnerable-skill note tightened. - `examples/finding-positive-skill/SKILL.md` body sentence split across the brand-voice 25-word cap. - `.github/workflows/self-scan.yml` comment block reworded — earlier "zero `SKILL.md` files" was inaccurate; the real reason `pruner scan` exits 0 on the pre-existing job is that every Pruner-side fixture finding sits in `.pruner-ignore.yml`. - `docs/sha-pinning.md` gains a "Pre-tag lockstep check" section documenting the `EXPECTED_RELEASE_TAG` env-var contract on `verify-action-pins.sh`. - `docs/threat-model.md` self-scan bullet now names both jobs; release- integrity bullet now mentions the verifier's two invariants (SHA-pin validity + scan.yml lockstep). - `CHANGELOG.md` v0.2.15 entry gains a fifth bullet describing the allowlist additions. --- .github/workflows/self-scan.yml | 11 +++---- .pruner-ignore.yml | 22 ++++++++++++++ CHANGELOG.md | 3 +- docs/sha-pinning.md | 30 +++++++++++++++++++ docs/threat-model.md | 4 +-- .../finding-positive-skill/EXPECTATIONS.md | 22 ++++++++++---- examples/finding-positive-skill/SKILL.md | 9 ++---- 7 files changed, 81 insertions(+), 20 deletions(-) diff --git a/.github/workflows/self-scan.yml b/.github/workflows/self-scan.yml index 0601636..d502ebe 100644 --- a/.github/workflows/self-scan.yml +++ b/.github/workflows/self-scan.yml @@ -32,11 +32,12 @@ jobs: path: .pruner/ retention-days: 14 - # Composite-action invocation against a fixture that trips findings strictly - # below `critical`. Green outcome asserts `pruner scan` exit 1 threads through - # `coroboros-pack-run` and the gate step without being masked as a workflow - # failure — the 0.2.8 bug class. The `pruner-self-scan` job above scans `.` - # which holds zero `SKILL.md` files and never exits 1 (the blind spot). + # Fixture job. Trips sub-critical findings; green outcome asserts + # `pruner scan` exit 1 threads through `coroboros-pack-run` and the gate + # step without being masked as a workflow failure (0.2.8 bug class). + # The `pruner-self-scan` job above scans `.` with `.pruner-ignore.yml` + # applied — every Pruner-side fixture finding sits in the allowlist, so + # `pruner scan` exits 0 and the propagation contract goes untested there. pruner-finding-positive: runs-on: ubuntu-latest permissions: diff --git a/.pruner-ignore.yml b/.pruner-ignore.yml index 6fec9aa..85c8326 100644 --- a/.pruner-ignore.yml +++ b/.pruner-ignore.yml @@ -58,3 +58,25 @@ ignores: - rule: PI-PERM-001 path: examples/vulnerable-skill/SKILL.md justification: "Intentional vulnerable-skill fixture — allowed-tools mismatch with sibling scripts demonstrates PI-PERM-001." + + # finding-positive-skill — intentional fixture. The path-rooted entries below + # suppress the findings in `pruner-self-scan` (target-path: `.`, paths land as + # `examples/finding-positive-skill/SKILL.md`). The `pruner-finding-positive` + # job scans `examples/finding-positive-skill` directly, so finding paths are + # bare `SKILL.md` and the entries do not match — findings fire there, exit-code + # propagation gets exercised. + - rule: FC001 + path: examples/finding-positive-skill/SKILL.md + justification: "Intentional finding-positive-skill fixture — non-kebab-case name trips FC001 to keep `pruner scan` exit 1 wired through `coroboros-pack-run`." + + - rule: FC003 + path: examples/finding-positive-skill/SKILL.md + justification: "Intentional finding-positive-skill fixture — top-level `custom_field` trips FC003 for the same exit-code-propagation purpose." + + - rule: FC004 + path: examples/finding-positive-skill/SKILL.md + justification: "Intentional finding-positive-skill fixture — `metadata.version` trips FC004 for the same exit-code-propagation purpose." + + - rule: FC005 + path: examples/finding-positive-skill/SKILL.md + justification: "Intentional finding-positive-skill fixture — non-SPDX license string trips FC005 for the same exit-code-propagation purpose." diff --git a/CHANGELOG.md b/CHANGELOG.md index a626aad..b97f72d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,7 +7,8 @@ Two CI additions close bug classes that the 0.2.7 and 0.2.8 releases exposed. Bo - **`scripts/verify-action-pins.sh` — pre-tag lockstep check.** Opt-in extension activated when `EXPECTED_RELEASE_TAG` is set in the script's environment. Reads `.github/workflows/scan.yml`, extracts the `uses: ob-aion/pruner@` literal, and exits 1 if the literal does not match the tag being cut. Script header documents the env-var contract. Closes the 0.2.7 bug class at the release-pipeline gate, before any release asset is built. Four consecutive releases (0.2.3-0.2.6) had shipped `scan.yml` pinned to the previous composite. - **`.github/workflows/release.yml` — tag resolution moved ahead of pin verification.** The `Resolve tag` step now precedes the renamed `Verify action SHA pins and scan.yml lockstep` step, which receives the tag via `EXPECTED_RELEASE_TAG`. No behavioural change to the SHA-pin loop or any downstream step. - **`examples/finding-positive-skill/` — new fixture.** Minimal skill trips four sub-critical Coroboros pack rules. FC001 high: `name: "Bad Name"` fails kebab-case. FC003 medium: top-level `custom_field` (must live under `metadata:`). FC004 low: `metadata.version` present (Coroboros house rule forbids it). FC005 low: `license: "Apache 2"` is not a valid SPDX identifier. Zero critical by design. Companion `EXPECTATIONS.md` documents the cross-walk. Distinct from `examples/vulnerable-skill/`: 21 findings, 7 critical, every finding allowlisted in `.pruner-ignore.yml`, validating the full Coroboros + Cisco detection surface. -- **`.github/workflows/self-scan.yml` — `pruner-finding-positive` job added.** Invokes the composite via `uses: ./` against `examples/finding-positive-skill` at `fail-on: critical`. Green outcome asserts that `pruner scan` exit 1 threads through `coroboros-pack-run` and the gate step. Workflow stays green even though findings exist (all below threshold). The pre-existing `pruner-self-scan` job scans `.` from repo root with `.pruner-ignore.yml` applied. Every Pruner-side fixture finding sat in the allowlist, so `pruner scan` exited 0. The 0.2.8 bug never surfaced on the maintainer side until the first consumer integration on `coroboros/agent-skills`. Finding-positive findings are intentionally not allowlisted. They surface in Code Scanning at downgraded severities (template-example weight 0.25 → high becomes low) but never gate the workflow. Branch protection on `main` requires `pruner-self-scan` already. Promoting `pruner-finding-positive` to a required status check is a one-line `gh api` follow-up after the first green run on `main`. +- **`.github/workflows/self-scan.yml` — `pruner-finding-positive` job added.** Invokes the composite via `uses: ./` against `examples/finding-positive-skill` at `fail-on: critical`. Green outcome asserts that `pruner scan` exit 1 threads through `coroboros-pack-run` and the gate step. Workflow stays green even though findings exist (all below threshold). The pre-existing `pruner-self-scan` job scans `.` from repo root with `.pruner-ignore.yml` applied. Every Pruner-side fixture finding sat in the allowlist, so `pruner scan` exited 0. The 0.2.8 bug never surfaced on the maintainer side until the first consumer integration on `coroboros/agent-skills`. Branch protection on `main` requires `pruner-self-scan` already. Promoting `pruner-finding-positive` to a required status check is a one-line `gh api` follow-up after the first green run on `main`. +- **`.pruner-ignore.yml` — four entries added for the new fixture.** Path-rooted entries (`examples/finding-positive-skill/SKILL.md`) suppress the findings inside `pruner-self-scan` so the Security tab and PR review surface stay clean. The new `pruner-finding-positive` job scans the fixture directly, so finding paths are bare `SKILL.md` and the entries do not match there — findings fire, exit-code propagation gets exercised. Same pattern as the 14 vulnerable-skill entries already in the file. - **`.github/workflows/scan.yml` — synced to `ob-aion/pruner@0.2.15`** per the lockstep contract, now verified pre-tag by `scripts/verify-action-pins.sh`. ## v0.2.14 - 12/05/2026 diff --git a/docs/sha-pinning.md b/docs/sha-pinning.md index fca222f..51b9b61 100644 --- a/docs/sha-pinning.md +++ b/docs/sha-pinning.md @@ -38,3 +38,33 @@ bash scripts/verify-action-pins.sh ``` Dependabot itself resolves to commit SHAs correctly when it opens a bump PR, so the bug class is mostly introduced by manual pins. The pre-flight catches both paths. + +## Pre-tag lockstep check + +The same script gates one more invariant when `EXPECTED_RELEASE_TAG` is set in its environment. `.github/workflows/scan.yml` self-references the composite via `uses: ob-aion/pruner@`. That literal must match the tag being cut, or consumers pinning `scan.yml@` end up running an earlier composite internally (the 0.2.7 bug class — four consecutive releases shipped out of lockstep). + +`release.yml` resolves the tag first and exports it to the verifier: + +```yaml +- name: Resolve tag + id: tag + shell: bash + run: | + REF="${GITHUB_REF#refs/tags/}" + echo "tag=${REF}" >> "$GITHUB_OUTPUT" + +- name: Verify action SHA pins and scan.yml lockstep + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + EXPECTED_RELEASE_TAG: ${{ steps.tag.outputs.tag }} + shell: bash + run: bash scripts/verify-action-pins.sh +``` + +Local invocation before tagging: + +```bash +EXPECTED_RELEASE_TAG=0.2.15 bash scripts/verify-action-pins.sh +``` + +Exit codes: `0` OK, `1` bad SHA pin or lockstep mismatch, `2` missing `gh` CLI or missing `scan.yml`. Running the script without the env var keeps the SHA-only behaviour — useful for routine Dependabot review. diff --git a/docs/threat-model.md b/docs/threat-model.md index 1712896..535f9dd 100644 --- a/docs/threat-model.md +++ b/docs/threat-model.md @@ -80,5 +80,5 @@ Pruner's own attack surface is small but real: - **Wrapper package (`pruner-wrapper`).** Runtime deps are `pyyaml` and `cisco-ai-skill-scanner` (the pinned engine). Stdlib otherwise. The dependency surface is reviewable in `wrapper/pyproject.toml`. Dependabot tracks bumps; CODEOWNERS-required review. - **Cisco engine.** Apache-2.0, multi-thousand-LOC scanner. Pinned at `2.0.9`. License-drift check runs at install ([`scripts/setup-cisco.sh`](../scripts/setup-cisco.sh)) and halts the action if the upstream license marker changes. Monthly cron probe at [`.github/workflows/cisco-upstream-check.yml`](../.github/workflows/cisco-upstream-check.yml) opens an `upstream-drift` issue on archival, license change, or 90-day staleness. - **Composite action.** Every `uses:` line is SHA-pinned with a `# v1.2.3` comment for human-readable diff review. No JS / no compiled `dist/`. -- **Self-scan.** Pruner scans Pruner on every push ([`.github/workflows/self-scan.yml`](../.github/workflows/self-scan.yml)). A failing self-scan blocks merge. -- **Release integrity.** [`release.yml`](../.github/workflows/release.yml) re-runs the full scan against the tagged ref. Drift between `main` and the tag fails the release. +- **Self-scan.** Pruner scans Pruner on every push ([`.github/workflows/self-scan.yml`](../.github/workflows/self-scan.yml)). Two jobs: `pruner-self-scan` against the repo root (broad coverage), and `pruner-finding-positive` against [`examples/finding-positive-skill/`](../examples/finding-positive-skill/) (exit-code propagation under `fail-on: critical`). A failing job blocks merge. +- **Release integrity.** [`release.yml`](../.github/workflows/release.yml) re-runs the full scan against the tagged ref. Drift between `main` and the tag fails the release. [`scripts/verify-action-pins.sh`](../scripts/verify-action-pins.sh) gates the release on two invariants: every SHA-pinned `uses:` line resolves to a real commit (not a tag-object SHA), and `scan.yml`'s `uses: ob-aion/pruner@` matches the tag being cut. diff --git a/examples/finding-positive-skill/EXPECTATIONS.md b/examples/finding-positive-skill/EXPECTATIONS.md index d3a1e6f..691a8f9 100644 --- a/examples/finding-positive-skill/EXPECTATIONS.md +++ b/examples/finding-positive-skill/EXPECTATIONS.md @@ -11,9 +11,19 @@ by design. | FC004 | `SKILL.md` | low | `metadata.version` present (Coroboros house rule forbids it). | | FC005 | `SKILL.md` | low | `license: "Apache 2"` is not a valid SPDX identifier (correct: `Apache-2.0`). | -The fixture exists to keep `pruner scan` exit 1 threaded through the composite -action under `fail-on: critical` — exit 0 is the expected outcome at the -workflow level. Distinct from `examples/vulnerable-skill/`, which trips 7 -critical findings (PI-UNI-001/003 weight-locked at 1.00, PI-IDFILE-001 and -PI-EXFIL-002 on `scripts/`) and exists to validate the full Coroboros + Cisco -detection surface. +The fixture keeps `pruner scan` exit 1 threaded through the composite action under `fail-on: critical`. Workflow exit 0 is the expected outcome. + +## Dual-path allowlist behaviour + +The repo-root `.pruner-ignore.yml` lists each of the four findings keyed by the path `examples/finding-positive-skill/SKILL.md`. Two scan contexts read that file: + +| Job | `target-path` | Finding path | Allowlist match | Findings | +|---|---|---|---|---| +| `pruner-self-scan` | `.` | `examples/finding-positive-skill/SKILL.md` | yes | suppressed (Security tab + PR review stay clean) | +| `pruner-finding-positive` | `examples/finding-positive-skill` | `SKILL.md` | no | fire (exit-code propagation gets exercised) | + +The path-based asymmetry is intentional and mirrors the existing 14 `examples/vulnerable-skill/` entries. + +## Distinct from vulnerable-skill + +`examples/vulnerable-skill/` trips 7 critical findings (PI-UNI-001 / PI-UNI-003 weight-locked at 1.00; PI-IDFILE-001 and PI-EXFIL-002 on `scripts/`) and validates the full Coroboros + Cisco detection surface. This fixture trips four sub-critical findings and validates a single composite-action contract. diff --git a/examples/finding-positive-skill/SKILL.md b/examples/finding-positive-skill/SKILL.md index 331ed30..cc8c486 100644 --- a/examples/finding-positive-skill/SKILL.md +++ b/examples/finding-positive-skill/SKILL.md @@ -9,11 +9,8 @@ custom_field: "bar" # Finding-positive -Trips Coroboros pack findings strictly below the `critical` threshold. The -`pruner-finding-positive` job in `.github/workflows/self-scan.yml` invokes the -composite via `uses: ./` against this fixture at `fail-on: critical` and -asserts the workflow goes green — `pruner scan` exit 1 (findings present, all -below threshold) must thread through `coroboros-pack-run` and the gate step -without being masked as a workflow failure. The 0.2.8 bug class. +Trips Coroboros pack findings strictly below the `critical` threshold. + +The `pruner-finding-positive` job in `.github/workflows/self-scan.yml` invokes the composite via `uses: ./` against this fixture at `fail-on: critical`. Green outcome asserts the workflow stays green. `pruner scan` exit 1 (findings present, below threshold) must thread through `coroboros-pack-run` and the gate step. The 0.2.8 bug class. See `EXPECTATIONS.md` for the rule cross-walk.