fix(agent): prevent Kubescape LIST skeleton false alarms#2287
Conversation
@codex review for correctness of the Kubescape LIST-skeleton guard and its CI coverage. |
User evaluationI exercised this as the next Platform maintainer, not just as the author: starting from the same cluster-wide Kubescape LIST that appeared empty, the revised card now stops that conclusion and directs the reader to named-object GETs. Following it against prod returned real payloads (Velero 95 matches, CloudNativePG 594, KSail 11, plus populated |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0b938a1059
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
@codex review for the fixes at |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5a53b99588
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
@codex review the final head |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0fc93f314b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
@codex review Please review exact head |
💡 Codex Review Require coherent summaries on the matches path
When a manifest has Grype matches but its paired summary is empty or disagrees—a plausible partial kubevuln result—this monorepo/.claude/agents/platform-security-surveyor.md Lines 69 to 72 in 0e75f16  Represent bounded failures for every surveyed surface
When posture or runtime has too many contributing objects for the read bound and no verified aggregate, the general rule requires an unavailable/partial cluster-wide result, but this exact digest schema provides coverage and unavailable states only for CVE while still requiring a posture score and numeric runtime detection count. The agent must therefore either exceed the bound, invent a cluster-wide value, or violate the required shape; add unavailable/partial representations for posture and runtime or explicitly scope the completeness rule to CVE. monorepo/.claude/agents/platform-security-surveyor.md Lines 77 to 81 in 0e75f16 Feed partial coverage into the action list
When the survey finds a current container without a fresh manifest/summary pair, the new rules label it a coverage gap but only require it to appear as ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Fixed all three findings in
The regression contract asserts all three rules across both instruction surfaces. Targeted tests, ShellCheck, actionlint, and diff checks pass locally. |
@codex review Please review exact head |
@codex review Retrying the exact-head review request after GitHub returned sustained 503s. Please review |
@coderabbitai review Codex exact-head review requests have been unacknowledged through the GitHub 503 window. Please review exact head 3c468a8, focusing on liveness versus coverage, bounded aggregation, coherent CVE pairs, partial/unavailable output, and actionable coverage gaps. |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
head='3c468a895ba0c33708aeed0ad9e120b0a4bfd678'
echo '== Commit =='
git show -s --format='%H%n%P%n%s' "$head"
echo
echo '== Changed files from first parent =='
git diff-tree --no-commit-id --name-status -r "$head"
echo
echo '== Files mentioning Kubescape / surveyor / bounded / coherent / coverage =='
rg -n -i -C 2 \
'kubescape|portfolio-surveyor|bounded|coherent|liveness|coverage|deltas_needing_action|partial|unavailable' \
.claude .github 2>/dev/null || true
echo
echo '== Diff (requested commit) =='
git show --format=fuller --find-renames --find-copies --stat --patch "$head" -- .claude .github 2>/dev/null || true
echo
echo '== Candidate file outlines =='
fd -t f . .claude .github 2>/dev/null \
| rg '(portfolio-surveyor|PLATFORM|platform|surveyor|kubescape)' \
| while IFS= read -r f; do
echo "--- $f"
ast-grep outline "$f" --items all 2>/dev/null || true
doneLength of output: 88391 🏁 Script executed: #!/bin/bash
set -euo pipefail
head='3c468a895ba0c33708aeed0ad9e120b0a4bfd678'
base="$(git merge-base "$head" origin/main 2>/dev/null || git merge-base "$head" main)"
printf '== Merge base ==\n%s\n' "$base"
echo
echo '== Cumulative changed files =='
git diff --name-status "$base" "$head"
echo
echo '== Cumulative diff: contract files =='
git diff --find-renames --unified=35 "$base" "$head" -- \
.claude/skills/products/platform/SKILL.md \
.claude/agents/platform-security-surveyor.md \
.claude/scripts/portfolio-surveyor.test.sh \
.github/workflows/ci.yaml
echo
echo '== Numbered Platform-card security section =='
nl -ba .claude/skills/products/platform/SKILL.md | sed -n '63,115p'
echo
echo '== Numbered security-surveyor rule and digest sections =='
nl -ba .claude/agents/platform-security-surveyor.md | sed -n '20,90p'
echo
echo '== Numbered contract-test assertions =='
nl -ba .claude/scripts/portfolio-surveyor.test.sh | sed -n '1,70p'
echo
echo '== Focused state/action occurrences in the two instruction surfaces =='
rg -n -i -C 2 \
'unavailable|partial|complete|deltas_needing_action|coverage:|cluster-wide|aggregate|every object|coherent|zero-match|zero counters' \
.claude/skills/products/platform/SKILL.md \
.claude/agents/platform-security-surveyor.md \
.claude/scripts/portfolio-surveyor.test.shLength of output: 29815
The two instruction surfaces consistently enforce:
The contract test covers both surfaces, and CI path filters ensure changes to either security instruction trigger that contract job. |
|
🤖 Codex Daily AI Engineer @coderabbitai summary Please generate the supported pre-merge summary for exact head |
|
✅ Action performedSummary regeneration triggered. |
|
Warning Review limit reached
Next review available in: 49 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Repository YAML (base), Organization UI (inherited) Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (4)
📝 WalkthroughWalkthroughThe PR strengthens Kubescape liveness and coverage guidance for posture, CVE, and runtime findings. It requires targeted reads and paired CVE evidence, distinguishes unavailable or partial results from valid zero findings, expands the security digest schema, adds contract-test assertions for these requirements, and updates CI path filters to run the tests for related platform guidance changes. Possibly related issues
Possibly related PRs
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
🤖 Codex Daily AI Engineer @coderabbitai review The exact-head review is clean, but the regenerated auto-summary for |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
repo="devantler-tech/monorepo"
pr_number="2287"
echo "== Pull request head and mergeability =="
gh pr view "$pr_number" --repo "$repo" \
--json number,url,headRefOid,headRefName,baseRefName,mergeStateStatus,reviewDecision,statusCheckRollup \
--jq '{
number,
url,
headRefName,
headRefOid,
baseRefName,
mergeStateStatus,
reviewDecision,
checks: [
.statusCheckRollup[]? |
{
name: (.name // .context // "unknown"),
status: (.status // "unknown"),
conclusion: (.conclusion // .state // "unknown"),
workflow: (.workflowName // "")
}
]
}'
echo
echo "== Requested exact commit =="
gh api "repos/${repo}/commits/3c468a895ba0c33708aeed0ad9e120b0a4bfd678" \
--jq '{sha, message: (.commit.message | split("\n")[0]), parents: [.parents[].sha]}'Length of output: 4013
Publishing the refreshed supported pre-merge review and summary now. ✅ Action performedSummary regeneration triggered. |
Requesting a Cursor Bugbot review at the current head (Codex lane is credit-dead; self-review clean). |
|
@cursor review |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
An incoherent CVE pair — a positive manifest with an empty or disagreeing summary — was already called "partial, not healthy" in the prose, but the digest had nowhere to put it: scanners_alive offered only yes|BROKEN, so the surveyor had to call a not-healthy surface a live one. Add the PARTIAL state, route the incoherent case to it plus a deltas_needing_action line, and pin the CVE unavailable template the contract test had left unguarded.
Both Bugbot findings fixed in |
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 4d5e03c. Configure here.
Readiness — promoting.
|
1 participant
Summary
Evidence and evaluation
The live health pass initially misread LIST projections and reopened devantler-tech/platform#2450. Direct GETs then showed healthy production data: Velero had 95 Grype matches, CloudNativePG had 594, KSail had 11, and a Velero summary held populated all/relevant counts and refs. I corrected and re-closed that issue before authoring this guard.
The regression test was added first and failed against the Platform card, then passed after the card converged. Replaying the incident now reaches the correct path: LIST metadata selects and reconciles objects; direct paired reads establish liveness; incomplete or unbounded evidence becomes partial/unavailable and enters the action list instead of becoming a false zero.
Validation
Fixes #2286