Harden the base image and add a gated weekly base refresh#99
Merged
Conversation
…icate curl The base build fetched wolfi-signing.rsa.pub over the network at every build - trust-on-first-use of whatever the CDN served, and a second install of curl solely to do it. The key is now vendored at core/wolfi-signing.rsa.pub (COPY'd into /etc/apk/keys), acquired from two independent channels (packages.wolfi.dev and the wolfi-base image's own /etc/apk/keys copy) and committed only on byte-identity. Its sha256 is pinned in verify-ci-supplychain-unit.bats, so a swapped file fails the unit lane, and a rotated upstream key fails the image build closed until the copy is deliberately refreshed. curl now installs once, in the tooling layer.
shadow exists in the base solely so the Dockerfile can groupadd/useradd the sluice and squid users at build; at runtime nothing touches it (the inventory path's su resolves to busybox, verified in-image), yet its setuid tool set is the reason the strip sweep exists. apk del shadow now chains onto the squid-user RUN - the last build-time use - so the shipped filesystem carries no user-management tools at all. The setuid sweep and its structure invariant stay as defense in depth against other packages' bits. New structure tests pin the absence (busybox ships adduser, not useradd, so the check cannot pass vacuously) and a positive control proves busybox su still drops to uid 1000 for the lock/sbom inventory path. A setup needing useradd re-adds it with SLUICE_EXTRA_PKGS=shadow (documented).
The Dockerfile's append-not-prepend PATH rule is a security claim (a uid-1000-writable ~/.npm-global/bin placed before /usr/bin would let a planted binary shadow system tools for a root-context exec) and the /home/node symlink is a compat contract - neither had a structure test. Both now do; each was proven able to fail against a deliberately broken scratch build (PATH prepended, symlink removed: 2 failures) before the real image went green.
…lost digest pin
verify_base treated a FAILED cosign verification of the official base
as a warning unless SLUICE_REQUIRE_SIGNED=1 - a fail-open default on
the exact signal the feature exists to catch: cosign present, official
ref, signature does not verify. That case now dies unconditionally.
Missing cosign stays a warning (absence is an environment fact, not a
tamper signal), mirror refs still skip verification, and =1 keeps its
stricter meaning (cosign, attestation, digest, official ref all
mandatory).
resolve_base_digest's silent fallback to the mutable tag quietly
reopened the verify/build TOCTOU window it exists to close; it now
warns loudly, and dies under SLUICE_REQUIRE_SIGNED=1 where the pin is
part of the guarantee.
THREAT_MODEL previously read stronger than the code ('verifies ...
before building on it'); code and doc now state the same contract.
Suite: the warn-path test is inverted to pin the new die, with a
good-signature positive control proving the gate does not overblock,
an absent-cosign control, and loud/fatal fallback coverage. The docker
stub now answers image inspect with a RepoDigest so the digest gate
cannot mask the verify-layer contracts under test.
…ion tags The cosign identity regexp pinned publish-base.yml@refs/tags/v, which rejects a signature minted by the SAME workflow running at refs/heads/main - the ref every workflow_dispatch run signs at today (a latent bug: a manually republished base verified by no launcher) and the ref the scheduled weekly refresh will sign at. The regexp now accepts exactly two refs of the one workflow file: refs/tags/v* and refs/heads/main. No new trust anchor: an attacker who can push to main can already edit the signing workflow itself, and a main-ref sign only happens after the structure tests and scan gate pass in the same job. All 7 occurrences move together (launcher x2 via bin/sluice, workflow x2, supply-chain doc x2, SECURITY.md x1) and a new sync test keeps them byte-identical forever - the policy-signing identity is a separate feature and deliberately outside the set. Rollout: a pre-upgrade launcher meeting a main-signed :latest warns (its default was fail-open) and dies only under SLUICE_REQUIRE_SIGNED=1; the fix is upgrading or pinning a version tag. This lands BEFORE the scheduled refresh ships.
…it suite The weekly base refresh needs a deterministic push/skip/refuse decision between the candidate rebuild and the published :latest. The gate compares sorted purl SETS for inventory identity (the deterministic SBOM embeds the image ref in metadata.component.name, so raw bytes never match across refs) and applies cumulative severity dominance over the grype results: at every tier, the candidate's count of findings at-or-above must not exceed the published image's - same-tier CVE swaps and severity downgrades pass (rolling-repo reality), any net worsening refuses. Exit contract mirrors lock --scan: 0 push, 2 skip (identical), 3 refuse, 4 gate-could-not-evaluate - with 4 covering missing jq, missing files, malformed JSON, and hollow (zero-purl) SBOMs, so a broken checker can never read as a push. Eleven fixture tests were written first and all failed before the script existed.
The published base was rebuilt only on v* release tags, so it fossilized between releases while rolling Wolfi shipped CVE fixes: the published :latest scanned worse than a fresh local build (a High in gh's vendored grpc, an nghttp2 medium), inverting the point of publishing a curated base. publish-base.yml now runs in two modes under one signing identity. Release (v* tag): unchanged semantics - push :<tag> + :latest, no gate, the human override. Refresh (weekly cron + workflow_dispatch): rebuild from rolling Wolfi and push :latest only when base-scan-gate rules the candidate inventory-different and no worse at every severity tier; identical skips, worse or unevaluable fails closed. A concurrency group serializes release and refresh. Also in the rewrite: the arm64 image - pushed since day one but never tested - now gets the same structure-test pass emulated under qemu; every build/pull step carries the acceptance.yml 3-attempt retry (the publish path had none); the two build-push-action steps become plain retried buildx invocations (one fewer pinned action, digest read from the metadata file and required to be non-empty before anything signs); and the sanity verify targets the exact pushed digest instead of a mutable tag. Structural unit guards pin the refresh-only-:latest tag set, the gate invocation, the schedule, and the concurrency group.
org.opencontainers.image.source links the GHCR package back to the repo (GHCR renders it and it feeds provenance tooling); revision is stamped with the building commit by the publish workflow and empty on local builds, so the structure test asserts only the static source label. Proven red against the unlabeled image first.
Rehearsing the gate against the real published base exposed a fail-open design: scanning the six-ecosystem SBOM returned zero findings on BOTH sides, while a direct image scan of the same published base returned six - including the High (GHSA-hrxh-6v49-42gf) in the grpc vendored inside gh's Go binary. The SBOM lists gh as one apk package, so an sbom: scan is structurally blind to exactly the CVE class that motivated the gate; grype's own image catalog sees it. Severity now comes from grype docker: scans of the candidate and the published image; the sluice SBOMs keep their two roles (inventory identity for the skip decision, the attest predicate). A structural guard pins images-not-sbom. Re-rehearsed with real data: candidate Medium:3/Unknown:2 vs published High:1/Medium:3/Unknown:2 - PUSH, with the High visible and eliminated.
…e purl sets
attack-changes found three fail-open holes in the refresh gate, all
confirmed against the branch and absent on main:
- HIGH: a clean grype scan (empty matches) counted a PHANTOM 'unknown'
finding, because jq's // treats an empty stream as absent and
substitutes the fallback. That phantom cancelled a real new
Unknown-severity CVE on the other side, so a worse candidate PUSHed.
The fallback is now bound per-element, so an empty stream stays zero.
- MEDIUM: a present-but-unrecognized severity ('', 'Moderate', a future
grype vocabulary bump) matched no tier and was invisible to the whole
comparison, so a strictly-worse candidate PUSHed. Off-list strings now
fold to 'unknown' and count at the last tier.
- LOW: purl identity was a raw-line multiset, so a duplicate purl flipped
a no-change SKIP into a weekly no-op PUSH, and a forged embedded-newline
purl could split into lines matching a real inventory and SKIP a genuine
change. Identity is now a jq unique array; the hollow-SBOM guard checks
for the empty array.
Four fixtures + cases, each proven red against the pre-fix script.
Two HIGH defects attack-changes confirmed in the refresh workflow, both new to this branch: - The pushed image was a guaranteed from-scratch rebuild the gates never saw: the flow built a :smoke image, structure-tested and scanned THAT, then ran a SECOND buildx build from rolling Wolfi to push - so a package the gate rejected could reappear in the bytes that shipped, and the 'scans no worse' promise in THREAT_MODEL was attached to nothing. Now the multi-arch base is built and pushed ONCE to a throwaway :candidate digest; the structure tests, SBOM, and scan gate all run against that exact digest; and promotion to :latest/:vX is a re-tag (docker buildx imagetools create) that rebuilds nothing. A structural guard pins that exactly one multi-arch build exists. - A workflow_dispatch launched from a tag ref ran ungated RELEASE mode and would force-repush the frozen version tag plus :latest, because the mode keyed on ref_type alone. Release now requires an actual tag PUSH event; a tag-ref dispatch is refused outright. actionlint clean; the refresh-mode structural guards updated to the promote-by-digest shape.
verify_base discarded cosign's stderr (>/dev/null 2>&1) and, on the new fail-closed path, died with 'use a mirror ref to skip verification'. cosign exits nonzero for BOTH a signature mismatch and a transient failure (offline dev box, Rekor/Fulcio/TUF outage), so the swallowed output turned every outage into an undiagnosable 'tamper' that no longer builds even with the base already pulled - and the remediation text actively steered the user to disable verification to escape what might be a network blip. The verify now captures cosign's stderr (declare-then-assign so the assignment carries cosign's status, not local's) and prints its tail before dying, and the die advises retry-when-online vs do-not-build instead of a bypass. Regression: a cosign stub emitting a Rekor-outage line, asserted to reach the user on the die path (red against the old swallowing code).
… can't smuggle it The base stage strips every setuid/setgid bit and the base structure test asserts none remain, but SLUICE_EXTRA_PKGS, SLUICE_SETUP_ROOT_CMDS, and the SLUICE_PIN replay legs all install into the PROJECT stage, after that sweep. So the documented SLUICE_EXTRA_PKGS="shadow" escape hatch (or a pin carrying shadow) reinstalls chfn/chsh/passwd/... with their setuid-root bits into the shipped image, silently voiding the THREAT_MODEL no-setuid-primitive guarantee while every base-stage gate stays green. The project stage now mirrors the strip as its last root layer, so anything a project-stage layer introduced is stripped too. New engine suite builds a box with SLUICE_EXTRA_PKGS="shadow", proves shadow's tools really installed (non-vacuous), and asserts zero setuid/setgid binaries survive - red against the pre-fix Dockerfile, which shipped shadow's setuid set.
…olate the tests The remaining four LOW findings from attack-changes, all new to this branch: - The new REQUIRE_SIGNED die in resolve_base_digest, and the fail-closed verify_base die, fired inside build()'s command substitution AFTER mktemp, so 'rm -rf $tmp' was never reached and a build-context dir (a copy of the user's sluice.config.sh) leaked on every fatal run. Both the resolve and the verify are hoisted above the mktemp; net temp-dir leak on a fatal run is now zero (was one per run, ~2000 accumulated). - resolve_base_digest's fallback warning is stderr-only, but the suite only asserted on merged output - a regression sending it to stdout would corrupt the captured --build-arg BASE_IMAGE and stay green. A new test captures stdout ALONE and requires it be exactly the ref. - the 'digest fallback FATAL under REQUIRE_SIGNED=1' test accepted any nonzero exit and matched 'digest' in any text, so a failed sed extraction (127, error text) passed it vacuously; now pinned to the die's exit 9 and its specific message. - a leaked SLUICE_REQUIRE_SIGNED=1 in the invoking shell flipped the default-posture tests; a file-level setup() exports it empty. Also tightens a handful of inline test comments to the two-line rule - the defect-class rationale stays in the file headers (repo convention), inline notes keep only the non-obvious attack mechanism.
…cate at 2^31 Every awk emitter that prints a byte count used %d. mawk casts %d to a 32-bit int, so a byte total past 2^31 truncated to 2147483647 (~2.0 GB) on Debian/CI/Alpine mawk, while macOS one-true-awk (doubles) and the CI runner's awk rendered it correctly - which is why the GB-ladder test passed everywhere except a truncating-mawk host and this shipped undetected. The truncation was not cosmetic: the same row byte total feeds the high-volume flag comparison and the JSON, so a multi-GB exfil with a SLUICE_EGRESS_FLAG_BYTES threshold above 2 GiB slid under the flag and was never tagged. All byte-carrying emitters (reached/blocked host rows, host-budget tx, the allow-ips iptables byte counter, the firewall-drop total) now use %.0f, which formats the double exactly to 2^53; request and packet counts stay %d. Adds a flag-dodge regression alongside the existing render test - both only fail on truncating mawk, so the Linux VM lane is the one that catches them (CI's awk masks this class).
Round-1's hoist of resolve+verify to the top of build() ran them on the LIVE tag even in pin mode - but the build FROMs the pinned digest (its --build-arg BASE_IMAGE is appended last and wins). Two consequences, both new this branch: under SLUICE_REQUIRE_SIGNED=1 a pull hiccup or an air-gapped machine (the live tag yields no digest) hard-aborted a build that would have used the pinned digest, defeating the deterministic replay pin mode exists for; and verify_base verified the live tag, not the pinned bytes actually built. Pin detection and the pinned-base extraction/repo-check are hoisted above the resolve+verify. When pin mode is active with a signed base, the pinned @sha256 becomes the value resolved (it passes through resolve_base_digest with no pull, so a signed pin replays air-gapped), verified, and built - one digest for all three. The later pin block no longer re-appends BASE_IMAGE (no double build-arg) and its dies, now before the mktemp, need no context cleanup. Regression: pin + REQUIRE_SIGNED=1 with pull failing reaches the build FROMing the pin (marker-file assert), red against the pre-fix abort.
…g them past the guards
A zero-purl SBOM written as a concatenated multi-document stream
('{"components":[]}\n{"components":[]}') bypassed the hollow-SBOM
guard: purls() runs jq -c per document, so it emitted the two-line
'[]\n[]', which is not equal to the single-line literal '[]' the guard
compares against, and the shape check jq -e reflects only the last
document's status. The zero-purl inventory then reached the severity
comparison and PUSHed. (The grype side was already caught by count_sev's
digit check, but only incidentally.)
The gate now requires each input to be EXACTLY ONE JSON document
(jq -s 'length' == 1) up front - which also rejects invalid JSON - so a
multi-doc stream fails closed (4) before any guard can be fooled.
Regression fixtures + cases for a multi-doc SBOM (red against the old
script) and a multi-doc grype.
… project-stage sweep The 'Host privilege escalation' bullet still described the no-setuid guarantee as 'the shadow package's passwd/chsh de-setuid at build', which two commits on this branch made stale: shadow is now removed from the base after its build-time useradd (not merely de-setuid), and the setuid sweep is mirrored in the project stage so a re-added package (SLUICE_EXTRA_PKGS=shadow) or a pin replay cannot reintroduce a setuid binary. Restates the guarantee to match; it should have shipped with the shadow/setuid commits.
…security why The Dockerfile ran 53 comment lines against ~60 instructions - many were decorative dividers or SLUICE_* knob banners that restated the self-evident RUN and duplicated docs/configuration.md. Cut those and tightened the paragraph blocks to a single line of intent (53 -> 35 comment lines). Every security rationale is kept verbatim: the PATH-append privesc note, no-in-box-sudo, both setuid sweeps, the vendored-key fail-closed, the Docker 29.5.x COPY --chmod workaround, and the pin-legs-at-the-end cache ordering. Comment-only: instruction lines are byte-identical (verified) and the structure test still passes.
gitleaks' generic-api-key rule flags WOLFI_KEY_SHA256 (a 64-hex value next to a KEY-suffixed name), but it is the sha256 of the PUBLIC Wolfi signing key, not a secret. Inline gitleaks:allow silences the advisory finding so it stops re-firing on every PR that touches this file.
The inline gitleaks:allow from the prior commit could not work: gitleaks scans the whole PR commit range, and the WOLFI_KEY_SHA256 value was introduced (without the comment) in an earlier commit, which the range scan still flags. A repo-root .gitleaks.toml allowlist applies to every scanned commit regardless of which introduced the finding. It extends the default rule set and allowlists only the WOLFI_KEY_SHA256=<64hex> line pattern - the sha256 of the PUBLIC Wolfi signing key, not a secret. Validated locally with gitleaks 8.30.1: default rules reproduce the finding, this config suppresses it, and an injected real secret in the same file is still caught (allowlist is not too broad). Reverts the non-functional inline comment.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
The published
sluice-basewas rebuilt only on release tags, so it aged while rolling Wolfi shipped fixes: it carried GHSA-hrxh-6v49-42gf (High, grpc vendored ingh, fixed in grpc 1.82.1) and CVE-2026-58055 (nghttp2) while upstreamwolfi-basescanned clean. This hardens the base and its build/verify/publish path, and keeps:latestfresh between releases.Each commit is a single revertable change; review per-commit.
Base image. Vendor the checksum-pinned Wolfi signing key instead of fetching at build; delete
shadowafter build and re-strip setuid in the project stage so a re-added package cannot reintroduce it; structure-test the PATH-append and/home/nodeinvariants; add OCI labels.Verification. Fail closed on a failed cosign verification of the official base (was warn-and-continue), surfacing cosign's reason so an outage is not mistaken for tamper; accept signatures from the workflow on
mainas well asvtags; in pin mode verify and build the same pinned digest.Weekly base refresh. New scheduled job rebuilds
:latestfrom rolling Wolfi, structure-tests both arches (arm64 under qemu), and republishes only ifscripts/base-scan-gate.shrules the candidate no worse than the published image at every severity tier. Built once and promoted by digest, so the published bytes are the evaluated bytes; version tags stay frozen.Egress. Render byte totals with awk
%.0fnot%d, which mawk truncates at 2^31 (mis-rendered multi-GB transfers and could slip a >2 GiB transfer under the high-volume flag).THREAT_MODEL.md,docs/supply-chain.md,docs/configuration.md,README.md, andSECURITY.mdare updated for the changed guarantees.Verification
macOS unit lane and the Linux VM (real Docker: unit, structure, engine, security) pass; every behavior change ships a regression test.
publish-base.ymlis checked with actionlint and structural guards and a rehearsal of the scan-gate logic against the real published base; the full workflow first runs on the next tag or schedule.