Skip to content
Merged
43 changes: 35 additions & 8 deletions .claude/agents/platform-security-surveyor.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,17 @@ acts on it, not a human); return the digest and nothing else.
all-zero severities and empty VEX/matches even when the real objects are full — you MUST `kubectl get
<crd> <name> -n <ns> -o json` **by name** (sample 2–3 objects per surface) before concluding anything
about data quality. An all-zero LIST is a *display artifact*, not a finding.
Use **LIST metadata for coverage and freshness** only: reconcile result names, identity labels,
manifest/summary pairs, and timestamps against the **current workload/container inventory** with
bounded metadata LISTs. A missing or stale pair is a coverage gap, not zero findings. Sampling proves
**liveness only**. If the contributing set fits the ~dozen-read bound,
directly GET every object whose payload contributes; otherwise use a trusted aggregate endpoint already
verified against direct samples. When neither a payload-complete source nor complete inventory coverage fits the bound,
**report the cluster-wide result as unavailable or partial** and name the missing proof; never
extrapolate from the liveness sample. Bounded proof failure is per-surface:
**posture, CVE, and runtime each report unavailable or partial** instead of inventing a numeric value.
Every confirmed partial coverage gap must also appear in `deltas_needing_action` so the orchestrator
cannot miss a scanner blind spot.
And the standing trap this agent exists to catch: **an empty/zero reading almost always means the
scanner is BROKEN, not that the cluster is clean** — a broken scanner and a compliant cluster read
identically, so every surface check is **liveness first, values second**.
Expand All @@ -38,9 +49,19 @@ identically, so every surface check is **liveness first, values second**.
--tail=50` for scan aborts. Then: framework scores + the top failed controls (id, name, failing
count) vs baseline.
2. **CVE (kubevuln)** — `vulnerabilitymanifestsummaries` / `vulnerabilitymanifests` (+ `openvulnerabilityexchangecontainers`
for VEX). Liveness: GET 2–3 summaries by name — real grype `matches`/`tool.name` present, `.all`
and `.relevant` populated; on suspicion, kubevuln logs for `ScanCP … partial`. Then: critical/high
counts (all vs relevant), notable new reachable CVEs vs baseline, VEX doc count.
for VEX). Liveness: directly GET both named `vulnerabilitymanifests` and their corresponding
`vulnerabilitymanifestsummaries` for 2–3 workloads.
Accept **(Grype matches or scanner version metadata) with coherent paired summaries**
(`spec.metadata.tool.version` or `kubescape.io/tool-version`) across the sample. A genuinely zero-
match image is healthy when version evidence, summary refs, and zero counters agree. Positive
manifest matches with an empty or disagreeing summary are partial, not healthy — and *partial has a
place to go*: report `scanners_alive: cve=PARTIAL:<why>`, `coverage: cve=PARTIAL:<why>`, and a
`deltas_needing_action` line naming the incoherent pair. Without that routing the only states are
`yes` and `BROKEN`, so an incoherent pair reads as a healthy scanner. Do not require
`spec.metadata.tool.name`; healthy current objects leave it blank.
On suspicion, check kubevuln logs for `ScanCP … partial`. Then: critical/high counts (all vs relevant),
notable new reachable CVEs vs baseline, VEX doc count, collected only from the direct-object/verified-
aggregate rule above.
3. **Runtime (node-agent)** — `applicationprofiles`, `networkneighborhoods`, alert routing. Liveness:
profiles present and not all `partial`; routing **visible, not stdout-only** (exporter config /
`alertManagerExporterUrls` / Prometheus exporter state, and whether alerts reach the routed sink —
Expand All @@ -51,15 +72,21 @@ identically, so every surface check is **liveness first, values second**.
## Return — one compact digest (target < ~600 tokens), this exact shape
```
## Security digest — <UTC date> (baseline: <the baseline date/state the orchestrator gave you>)
scanners_alive: posture=<yes|BROKEN:why> cve=<yes|BROKEN:why> runtime=<yes|INVISIBLE:why>
posture: score <x> vs baseline <y> — top failed: <C-xxxx name (n)>, …
cve: crit/high all=<a>/<b> relevant=<c>/<d> — new reachable: <cve id → workload>, … ; vex_docs=<n>
runtime: <n> new detections — routing=<routed-to-X|stdout-only>
scanners_alive: posture=<yes|PARTIAL:why|BROKEN:why> cve=<yes|PARTIAL:why|BROKEN:why> runtime=<yes|PARTIAL:why|INVISIBLE:why>
posture: score <x|unavailable:why> vs baseline <y> — top failed: <C-xxxx name (n)>, …
coverage: posture=<complete|PARTIAL:why> cve=<complete|PARTIAL: n current containers lack fresh paired results> runtime=<complete|PARTIAL:why>
cve: crit/high all=<a>/<b>|<unavailable:why> relevant=<c>/<d>|<unavailable:why> — new reachable: <cve id → workload>, … ; vex_docs=<n>
runtime: <n|unavailable:why> new detections — routing=<routed-to-X|stdout-only>
ci_gate: threshold=<n> (in-cluster agrees|DIVERGES: <how>)
deltas_needing_action:
- <one line per confirmed off-baseline finding or broken scanner, worst first>
```
Digest rules: **classify, don't decide** — the orchestrator turns confirmed deltas into `security`
issues under the epic (platform#2447) or a hotfix; you never file, comment, or fix. A broken/invisible
scanner is itself a `deltas_needing_action` line (worst class of finding). If the context/credentials
are unavailable, say so in one line and stop — never guess from stale data.
Comment thread
cursor[bot] marked this conversation as resolved.
are unavailable, say so in one line and stop — never guess from stale data. A partial coverage set or
missing payload-complete aggregate is also explicit `unavailable`/`partial` evidence, never a zero,
and every confirmed partial coverage gap is repeated in `deltas_needing_action`. A surface whose
evidence is incoherent rather than absent — a positive manifest paired with an empty or disagreeing
summary — is `PARTIAL`, never `yes`: `yes` asserts the surface is proven, and there is no state
between `yes` and `BROKEN` unless this one is used.
45 changes: 45 additions & 0 deletions .claude/scripts/portfolio-surveyor.test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,57 @@ monorepo_skill="${repo_root}/.claude/skills/products/monorepo/SKILL.md"
product_engineering_skill="${repo_root}/.claude/skills/product-engineering/SKILL.md"
agent_skills_card="${repo_root}/.claude/skills/products/agent-skills/SKILL.md"
ksail_card="${repo_root}/.claude/skills/products/ksail/SKILL.md"
platform_card="${repo_root}/.claude/skills/products/platform/SKILL.md"
platform_security_surveyor="${repo_root}/.claude/agents/platform-security-surveyor.md"

fail() {
echo "portfolio surveyor contract: FAIL — $*" >&2
exit 1
}

for security_definition in "${platform_card}" "${platform_security_surveyor}"; do
grep -Fq 'Kubescape CR LISTs return spec-stripped skeletons.' "${security_definition}" ||
fail "${security_definition#"${repo_root}/"} can mistake a Kubescape LIST skeleton for empty scanner output"
grep -Fq 'sample 2–3 objects per surface' "${security_definition}" ||
fail "${security_definition#"${repo_root}/"} does not require direct per-object Kubescape payload checks"
Comment thread
devantler marked this conversation as resolved.
grep -Fq 'directly GET every object whose payload contributes' "${security_definition}" ||
fail "${security_definition#"${repo_root}/"} can extrapolate cluster-wide findings from a liveness sample"
# Literal Markdown code spans; command substitution is intentionally disabled.
# shellcheck disable=SC2016
grep -Fq 'both named `vulnerabilitymanifests` and their corresponding' "${security_definition}" ||
fail "${security_definition#"${repo_root}/"} does not require direct reads of both CVE object types"
grep -Fq '(Grype matches or scanner version metadata) with coherent paired summaries' "${security_definition}" ||
fail "${security_definition#"${repo_root}/"} can accept manifest matches paired with an incoherent CVE summary"
grep -Fq 'LIST metadata for coverage and freshness' "${security_definition}" ||
fail "${security_definition#"${repo_root}/"} does not define a bounded use for Kubescape LISTs"
grep -Fq 'current workload/container inventory' "${security_definition}" ||
fail "${security_definition#"${repo_root}/"} can miss workloads with no vulnerability result object"
grep -Fq 'report the cluster-wide result as unavailable or partial' "${security_definition}" ||
fail "${security_definition#"${repo_root}/"} can overclaim cluster-wide findings when bounded proof is unavailable"
grep -Fq 'posture, CVE, and runtime each report unavailable or partial' "${security_definition}" ||
fail "${security_definition#"${repo_root}/"} does not represent bounded proof failures for every Kubescape surface"
# Literal Markdown code spans; command substitution is intentionally disabled.
# shellcheck disable=SC2016
grep -Fq 'Every confirmed partial coverage gap must also appear in `deltas_needing_action`' "${security_definition}" ||
fail "${security_definition#"${repo_root}/"} can report a scanner blind spot without making it actionable"
done

grep -Fq 'posture: score <x|unavailable:why>' "${platform_security_surveyor}" ||
fail "platform security digest cannot represent unavailable posture findings"
grep -Fq 'coverage: posture=<complete|PARTIAL:why>' "${platform_security_surveyor}" ||
fail "platform security digest does not expose per-surface coverage"
grep -Fq 'cve: crit/high all=<a>/<b>|<unavailable:why>' "${platform_security_surveyor}" ||
fail "platform security digest cannot represent unavailable CVE findings"
grep -Fq 'runtime: <n|unavailable:why> new detections' "${platform_security_surveyor}" ||
fail "platform security digest cannot represent unavailable runtime findings"
Comment thread
cursor[bot] marked this conversation as resolved.
# An incoherent CVE pair is neither proven nor absent. Without a PARTIAL state the
# only choices are `yes` and `BROKEN`, so the digest reports a healthy scanner over
# evidence the prose has already called not-healthy.
grep -Fq 'cve=<yes|PARTIAL:why|BROKEN:why>' "${platform_security_surveyor}" ||
fail "platform security digest has no PARTIAL state, so incoherent CVE evidence reads as a live scanner"
grep -Fq 'report `scanners_alive: cve=PARTIAL:<why>`' "${platform_security_surveyor}" ||
fail "platform security surveyor does not route an incoherent CVE pair to the PARTIAL state"

[[ -x "${classifier}" ]] || fail "release-bot exemption classifier is missing or not executable"
grep -Fq '.claude/scripts/release-bot-exemption.sh' "${surveyor}" ||
fail "surveyor does not delegate exemption decisions to the exact classifier"
Expand Down
35 changes: 30 additions & 5 deletions .claude/skills/products/platform/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,17 +71,42 @@ The recurring trap, learned the hard way (2026-07-04, all three surfaces were si
empty/zero reading almost always means the scanner is broken, not that the cluster is clean** — a broken
scanner and a compliant cluster look identical, so **check liveness first, every time**:

**Probe rule that overrides reflex:** **Kubescape CR LISTs return spec-stripped skeletons.** Use a
cluster-wide LIST only as **LIST metadata for coverage and freshness**: select object names, timestamps,
and identity labels; all-zero severities, empty matches, or empty VEX payloads in that response are
display artifacts, not findings. For CVE coverage, reconcile the result identities and manifest/summary
pairs against the **current workload/container inventory** before making any cluster-wide claim; an
absent or stale result is a coverage gap, not zero findings. Before judging payload quality,
`kubectl get <crd> <name> -n <ns> -o json` by name and **sample 2–3 objects per surface**. For CVE
liveness, directly GET both named `vulnerabilitymanifests` and their corresponding
`vulnerabilitymanifestsummaries`, then cross-check scanner freshness and logs. Never declare the
scanner broken or the cluster clean from a LIST projection. Sampling proves **liveness only**. If the
contributing set fits the read bound, directly GET every object whose payload contributes; otherwise
use a trusted aggregate endpoint already verified against direct samples. When neither a payload-
complete source nor complete inventory coverage can be established safely,
**report the cluster-wide result as unavailable or partial** and name the missing proof. Never
extrapolate cluster-wide findings from the liveness sample. Bounded proof failure is per-surface:
**posture, CVE, and runtime each report unavailable or partial** instead of inventing a numeric value.
Every confirmed partial coverage gap must also appear in `deltas_needing_action` so the orchestrator
turns the blind spot into tracked remediation.

- **Posture** (config scan) — `configurationscansummaries` / `workloadconfigurationscansummaries`
(per-namespace scores + failed controls). **Broken if** scores are `0.00` across frameworks,
`controls: null` en masse, or objects are days stale — check the `kubescape` scanner pod logs for scan
aborts. Note: the CI gate (`ksail workload scan --framework nsa --compliance-threshold N`) is a
**separate** static scan in a healthy CLI context — **a green CI gate does NOT prove the in-cluster
scan works.**
- **CVE** (kubevuln) — `vulnerabilitymanifestsummaries` / `vulnerabilitymanifests`. **Broken if**
manifests carry no grype `matches` / no `tool.name` and summaries are empty (both `.all` and
`.relevant`) — check kubevuln logs for `ScanCP … partial` (the relevancy path aborting on partial
ApplicationProfiles). Prioritise by relevancy × severity × fixability; emit VEX to suppress
non-reachable CVEs.
- **CVE** (kubevuln) — `vulnerabilitymanifestsummaries` / `vulnerabilitymanifests`. Liveness requires
**(Grype matches or scanner version metadata) with coherent paired summaries**
(`spec.metadata.tool.version` or the `kubescape.io/tool-version` annotation) across the direct
samples. A genuinely zero-match image is healthy when version evidence, summary refs, and zero
counters agree; positive manifest matches with an empty or disagreeing summary are partial, not
healthy. A blank
`spec.metadata.tool.name` alone is an upstream representation quirk, not an outage.
**Broken if** direct manifests carry neither matches nor scanner-version evidence and summaries are
empty (both `.all` and `.relevant`) — check kubevuln logs for `ScanCP … partial` (the relevancy path
aborting on partial ApplicationProfiles). Prioritise by relevancy × severity × fixability; emit VEX
to suppress non-reachable CVEs.
- **Runtime** (node-agent) — `applicationprofiles`, `networkneighborhoods`, and the
`node_agent_alert_counter` metric. **Invisible if** the exporters are stdout-only
(`alertManagerExporterUrls: []`, `prometheusExporterEnabled: false`). Route **natively to Coroot**
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -93,11 +93,13 @@ jobs:
- 'AGENTS.md'
- '.claude/agents/daily-maintainer.md'
- '.claude/agents/portfolio-surveyor.md'
- '.claude/agents/platform-security-surveyor.md'
- '.claude/skills/portfolio-maintenance/SKILL.md'
- '.claude/skills/product-engineering/SKILL.md'
- '.claude/skills/products/agent-skills/SKILL.md'
- '.claude/skills/products/ksail/SKILL.md'
- '.claude/skills/products/monorepo/SKILL.md'
- '.claude/skills/products/platform/SKILL.md'
- '.claude/scripts/portfolio-surveyor.test.sh'
- '.claude/scripts/release-bot-exemption.sh'
- '.github/workflows/ci.yaml'
Expand Down
Loading