diff --git a/.claude/agents/platform-security-surveyor.md b/.claude/agents/platform-security-surveyor.md index d1fd2218..52a80d3e 100644 --- a/.claude/agents/platform-security-surveyor.md +++ b/.claude/agents/platform-security-surveyor.md @@ -27,6 +27,17 @@ acts on it, not a human); return the digest and nothing else. all-zero severities and empty VEX/matches even when the real objects are full — you MUST `kubectl get -n -o json` **by name** (sample 2–3 objects per surface) before concluding anything about data quality. An all-zero LIST is a *display artifact*, not a finding. +Use **LIST metadata for coverage and freshness** only: reconcile result names, identity labels, +manifest/summary pairs, and timestamps against the **current workload/container inventory** with +bounded metadata LISTs. A missing or stale pair is a coverage gap, not zero findings. Sampling proves +**liveness only**. If the contributing set fits the ~dozen-read bound, +directly GET every object whose payload contributes; otherwise use a trusted aggregate endpoint already +verified against direct samples. When neither a payload-complete source nor complete inventory coverage fits the bound, +**report the cluster-wide result as unavailable or partial** and name the missing proof; never +extrapolate from the liveness sample. Bounded proof failure is per-surface: +**posture, CVE, and runtime each report unavailable or partial** instead of inventing a numeric value. +Every confirmed partial coverage gap must also appear in `deltas_needing_action` so the orchestrator +cannot miss a scanner blind spot. And the standing trap this agent exists to catch: **an empty/zero reading almost always means the scanner is BROKEN, not that the cluster is clean** — a broken scanner and a compliant cluster read identically, so every surface check is **liveness first, values second**. @@ -38,9 +49,19 @@ identically, so every surface check is **liveness first, values second**. --tail=50` for scan aborts. Then: framework scores + the top failed controls (id, name, failing count) vs baseline. 2. **CVE (kubevuln)** — `vulnerabilitymanifestsummaries` / `vulnerabilitymanifests` (+ `openvulnerabilityexchangecontainers` - for VEX). Liveness: GET 2–3 summaries by name — real grype `matches`/`tool.name` present, `.all` - and `.relevant` populated; on suspicion, kubevuln logs for `ScanCP … partial`. Then: critical/high - counts (all vs relevant), notable new reachable CVEs vs baseline, VEX doc count. + for VEX). Liveness: directly GET both named `vulnerabilitymanifests` and their corresponding + `vulnerabilitymanifestsummaries` for 2–3 workloads. + Accept **(Grype matches or scanner version metadata) with coherent paired summaries** + (`spec.metadata.tool.version` or `kubescape.io/tool-version`) across the sample. A genuinely zero- + match image is healthy when version evidence, summary refs, and zero counters agree. Positive + manifest matches with an empty or disagreeing summary are partial, not healthy — and *partial has a + place to go*: report `scanners_alive: cve=PARTIAL:`, `coverage: cve=PARTIAL:`, and a + `deltas_needing_action` line naming the incoherent pair. Without that routing the only states are + `yes` and `BROKEN`, so an incoherent pair reads as a healthy scanner. Do not require + `spec.metadata.tool.name`; healthy current objects leave it blank. + On suspicion, check kubevuln logs for `ScanCP … partial`. Then: critical/high counts (all vs relevant), + notable new reachable CVEs vs baseline, VEX doc count, collected only from the direct-object/verified- + aggregate rule above. 3. **Runtime (node-agent)** — `applicationprofiles`, `networkneighborhoods`, alert routing. Liveness: profiles present and not all `partial`; routing **visible, not stdout-only** (exporter config / `alertManagerExporterUrls` / Prometheus exporter state, and whether alerts reach the routed sink — @@ -51,10 +72,11 @@ identically, so every surface check is **liveness first, values second**. ## Return — one compact digest (target < ~600 tokens), this exact shape ``` ## Security digest — (baseline: ) -scanners_alive: posture= cve= runtime= -posture: score vs baseline — top failed: , … -cve: crit/high all=/ relevant=/ — new reachable: , … ; vex_docs= -runtime: new detections — routing= +scanners_alive: posture= cve= runtime= +posture: score vs baseline — top failed: , … +coverage: posture= cve= runtime= +cve: crit/high all=/| relevant=/| — new reachable: , … ; vex_docs= +runtime: new detections — routing= ci_gate: threshold= (in-cluster agrees|DIVERGES: ) deltas_needing_action: - @@ -62,4 +84,9 @@ deltas_needing_action: Digest rules: **classify, don't decide** — the orchestrator turns confirmed deltas into `security` issues under the epic (platform#2447) or a hotfix; you never file, comment, or fix. A broken/invisible scanner is itself a `deltas_needing_action` line (worst class of finding). If the context/credentials -are unavailable, say so in one line and stop — never guess from stale data. +are unavailable, say so in one line and stop — never guess from stale data. A partial coverage set or +missing payload-complete aggregate is also explicit `unavailable`/`partial` evidence, never a zero, +and every confirmed partial coverage gap is repeated in `deltas_needing_action`. A surface whose +evidence is incoherent rather than absent — a positive manifest paired with an empty or disagreeing +summary — is `PARTIAL`, never `yes`: `yes` asserts the surface is proven, and there is no state +between `yes` and `BROKEN` unless this one is used. diff --git a/.claude/scripts/portfolio-surveyor.test.sh b/.claude/scripts/portfolio-surveyor.test.sh index 018af10f..30a2f687 100755 --- a/.claude/scripts/portfolio-surveyor.test.sh +++ b/.claude/scripts/portfolio-surveyor.test.sh @@ -11,12 +11,57 @@ monorepo_skill="${repo_root}/.claude/skills/products/monorepo/SKILL.md" product_engineering_skill="${repo_root}/.claude/skills/product-engineering/SKILL.md" agent_skills_card="${repo_root}/.claude/skills/products/agent-skills/SKILL.md" ksail_card="${repo_root}/.claude/skills/products/ksail/SKILL.md" +platform_card="${repo_root}/.claude/skills/products/platform/SKILL.md" +platform_security_surveyor="${repo_root}/.claude/agents/platform-security-surveyor.md" fail() { echo "portfolio surveyor contract: FAIL — $*" >&2 exit 1 } +for security_definition in "${platform_card}" "${platform_security_surveyor}"; do + grep -Fq 'Kubescape CR LISTs return spec-stripped skeletons.' "${security_definition}" || + fail "${security_definition#"${repo_root}/"} can mistake a Kubescape LIST skeleton for empty scanner output" + grep -Fq 'sample 2–3 objects per surface' "${security_definition}" || + fail "${security_definition#"${repo_root}/"} does not require direct per-object Kubescape payload checks" + grep -Fq 'directly GET every object whose payload contributes' "${security_definition}" || + fail "${security_definition#"${repo_root}/"} can extrapolate cluster-wide findings from a liveness sample" + # Literal Markdown code spans; command substitution is intentionally disabled. + # shellcheck disable=SC2016 + grep -Fq 'both named `vulnerabilitymanifests` and their corresponding' "${security_definition}" || + fail "${security_definition#"${repo_root}/"} does not require direct reads of both CVE object types" + grep -Fq '(Grype matches or scanner version metadata) with coherent paired summaries' "${security_definition}" || + fail "${security_definition#"${repo_root}/"} can accept manifest matches paired with an incoherent CVE summary" + grep -Fq 'LIST metadata for coverage and freshness' "${security_definition}" || + fail "${security_definition#"${repo_root}/"} does not define a bounded use for Kubescape LISTs" + grep -Fq 'current workload/container inventory' "${security_definition}" || + fail "${security_definition#"${repo_root}/"} can miss workloads with no vulnerability result object" + grep -Fq 'report the cluster-wide result as unavailable or partial' "${security_definition}" || + fail "${security_definition#"${repo_root}/"} can overclaim cluster-wide findings when bounded proof is unavailable" + grep -Fq 'posture, CVE, and runtime each report unavailable or partial' "${security_definition}" || + fail "${security_definition#"${repo_root}/"} does not represent bounded proof failures for every Kubescape surface" + # Literal Markdown code spans; command substitution is intentionally disabled. + # shellcheck disable=SC2016 + grep -Fq 'Every confirmed partial coverage gap must also appear in `deltas_needing_action`' "${security_definition}" || + fail "${security_definition#"${repo_root}/"} can report a scanner blind spot without making it actionable" +done + +grep -Fq 'posture: score ' "${platform_security_surveyor}" || + fail "platform security digest cannot represent unavailable posture findings" +grep -Fq 'coverage: posture=' "${platform_security_surveyor}" || + fail "platform security digest does not expose per-surface coverage" +grep -Fq 'cve: crit/high all=/|' "${platform_security_surveyor}" || + fail "platform security digest cannot represent unavailable CVE findings" +grep -Fq 'runtime: new detections' "${platform_security_surveyor}" || + fail "platform security digest cannot represent unavailable runtime findings" +# An incoherent CVE pair is neither proven nor absent. Without a PARTIAL state the +# only choices are `yes` and `BROKEN`, so the digest reports a healthy scanner over +# evidence the prose has already called not-healthy. +grep -Fq 'cve=' "${platform_security_surveyor}" || + fail "platform security digest has no PARTIAL state, so incoherent CVE evidence reads as a live scanner" +grep -Fq 'report `scanners_alive: cve=PARTIAL:`' "${platform_security_surveyor}" || + fail "platform security surveyor does not route an incoherent CVE pair to the PARTIAL state" + [[ -x "${classifier}" ]] || fail "release-bot exemption classifier is missing or not executable" grep -Fq '.claude/scripts/release-bot-exemption.sh' "${surveyor}" || fail "surveyor does not delegate exemption decisions to the exact classifier" diff --git a/.claude/skills/products/platform/SKILL.md b/.claude/skills/products/platform/SKILL.md index 963d6a26..7c6be26e 100644 --- a/.claude/skills/products/platform/SKILL.md +++ b/.claude/skills/products/platform/SKILL.md @@ -71,17 +71,42 @@ The recurring trap, learned the hard way (2026-07-04, all three surfaces were si empty/zero reading almost always means the scanner is broken, not that the cluster is clean** — a broken scanner and a compliant cluster look identical, so **check liveness first, every time**: +**Probe rule that overrides reflex:** **Kubescape CR LISTs return spec-stripped skeletons.** Use a +cluster-wide LIST only as **LIST metadata for coverage and freshness**: select object names, timestamps, +and identity labels; all-zero severities, empty matches, or empty VEX payloads in that response are +display artifacts, not findings. For CVE coverage, reconcile the result identities and manifest/summary +pairs against the **current workload/container inventory** before making any cluster-wide claim; an +absent or stale result is a coverage gap, not zero findings. Before judging payload quality, +`kubectl get -n -o json` by name and **sample 2–3 objects per surface**. For CVE +liveness, directly GET both named `vulnerabilitymanifests` and their corresponding +`vulnerabilitymanifestsummaries`, then cross-check scanner freshness and logs. Never declare the +scanner broken or the cluster clean from a LIST projection. Sampling proves **liveness only**. If the +contributing set fits the read bound, directly GET every object whose payload contributes; otherwise +use a trusted aggregate endpoint already verified against direct samples. When neither a payload- +complete source nor complete inventory coverage can be established safely, +**report the cluster-wide result as unavailable or partial** and name the missing proof. Never +extrapolate cluster-wide findings from the liveness sample. Bounded proof failure is per-surface: +**posture, CVE, and runtime each report unavailable or partial** instead of inventing a numeric value. +Every confirmed partial coverage gap must also appear in `deltas_needing_action` so the orchestrator +turns the blind spot into tracked remediation. + - **Posture** (config scan) — `configurationscansummaries` / `workloadconfigurationscansummaries` (per-namespace scores + failed controls). **Broken if** scores are `0.00` across frameworks, `controls: null` en masse, or objects are days stale — check the `kubescape` scanner pod logs for scan aborts. Note: the CI gate (`ksail workload scan --framework nsa --compliance-threshold N`) is a **separate** static scan in a healthy CLI context — **a green CI gate does NOT prove the in-cluster scan works.** -- **CVE** (kubevuln) — `vulnerabilitymanifestsummaries` / `vulnerabilitymanifests`. **Broken if** - manifests carry no grype `matches` / no `tool.name` and summaries are empty (both `.all` and - `.relevant`) — check kubevuln logs for `ScanCP … partial` (the relevancy path aborting on partial - ApplicationProfiles). Prioritise by relevancy × severity × fixability; emit VEX to suppress - non-reachable CVEs. +- **CVE** (kubevuln) — `vulnerabilitymanifestsummaries` / `vulnerabilitymanifests`. Liveness requires + **(Grype matches or scanner version metadata) with coherent paired summaries** + (`spec.metadata.tool.version` or the `kubescape.io/tool-version` annotation) across the direct + samples. A genuinely zero-match image is healthy when version evidence, summary refs, and zero + counters agree; positive manifest matches with an empty or disagreeing summary are partial, not + healthy. A blank + `spec.metadata.tool.name` alone is an upstream representation quirk, not an outage. + **Broken if** direct manifests carry neither matches nor scanner-version evidence and summaries are + empty (both `.all` and `.relevant`) — check kubevuln logs for `ScanCP … partial` (the relevancy path + aborting on partial ApplicationProfiles). Prioritise by relevancy × severity × fixability; emit VEX + to suppress non-reachable CVEs. - **Runtime** (node-agent) — `applicationprofiles`, `networkneighborhoods`, and the `node_agent_alert_counter` metric. **Invisible if** the exporters are stdout-only (`alertManagerExporterUrls: []`, `prometheusExporterEnabled: false`). Route **natively to Coroot** diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index a0b93cda..5e6d9694 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -93,11 +93,13 @@ jobs: - 'AGENTS.md' - '.claude/agents/daily-maintainer.md' - '.claude/agents/portfolio-surveyor.md' + - '.claude/agents/platform-security-surveyor.md' - '.claude/skills/portfolio-maintenance/SKILL.md' - '.claude/skills/product-engineering/SKILL.md' - '.claude/skills/products/agent-skills/SKILL.md' - '.claude/skills/products/ksail/SKILL.md' - '.claude/skills/products/monorepo/SKILL.md' + - '.claude/skills/products/platform/SKILL.md' - '.claude/scripts/portfolio-surveyor.test.sh' - '.claude/scripts/release-bot-exemption.sh' - '.github/workflows/ci.yaml'