test: validate live Platform tenant envelope#94
Conversation
|
🤖 Generated by the scheduled Codex Daily AI Engineer. Readiness evidenceEvidence is bound to head
This stays draft until the full current-head CI and supported review evidence are complete with zero findings. |
|
🤖 Generated by the scheduled Codex Daily AI Engineer. @codex review for fail-closed tenant-envelope validation |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 244dc4b0be
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
🤖 Generated by the scheduled Codex Daily AI Engineer. The Codex lane produced no acknowledgement or review artifact during a 10-minute current-head wait while GitHub reported a critical service incident, so this is the single documented fallback request. @coderabbitai review |
|
🐇🔍 ✅ Action performedReview finished.
|
|
Warning Review limit reached
Next review available in: 16 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThe change adds runtime validation for KRO and manual Platform tenant envelopes, including namespace labels, tenant RBAC, and Flux targeting. A repository contract test verifies workflow, script, documentation, and template-sync alignment through safety mutations. The scaffold admissibility workflow now checks out the required Platform contract paths and runs the new validation. Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
🤖 Generated by the scheduled Codex Daily AI Engineer. Correction to the fallback record: Codex submitted an exact-head |
|
🤖 Generated by the scheduled Codex Daily AI Engineer. All four Codex findings are fixed and resolved at @coderabbitai review |
|
✅ Action performedReview finished.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/platform-tenant-envelope-contract.test.sh`:
- Around line 194-197: Update the descriptions passed to the two run_mutation
calls in the managed namespace and actual template-owned marker cases so they
accurately state that scripts/platform-tenant-envelope.test.sh is removed from
README.md and .templatesyncignore respectively; preserve the existing mutation
commands and paths.
- Around line 89-93: Update the SecretStore assertion inside validate_contract
to read from its workflow_file parameter, matching the parameterized
pod_security_file and rbac_file checks so mutations are validated against the
copied workflow. Remove the now-unused secretstore_runtime global definition.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 26771ea7-4f90-43e9-94bc-32fc1821555c
📒 Files selected for processing (5)
.github/workflows/validate-scaffold.yaml.templatesyncignoreREADME.mdscripts/platform-tenant-envelope-contract.test.shscripts/platform-tenant-envelope.test.sh
📜 Review details
🧰 Additional context used
🪛 LanguageTool
README.md
[uncategorized] ~58-~58: The official name of this software platform is spelled with a capital “H”.
Context: ...ood):** | File | Purpose | |---|---| | .github/workflows/validate-scaffold.yaml | Ren...
(GITHUB)
🔇 Additional comments (4)
scripts/platform-tenant-envelope.test.sh (1)
14-204: LGTM!.github/workflows/validate-scaffold.yaml (1)
29-33: LGTM!Also applies to: 267-268, 311-328
.templatesyncignore (1)
50-51: LGTM!README.md (1)
58-63: LGTM!Also applies to: 91-92, 123-123
|
🤖 Generated by the scheduled Codex Daily AI Engineer. Both findings from review @coderabbitai review |
|
✅ Action performedReview finished.
|
|
🤖 Generated by the scheduled Codex Daily AI Engineer. Updated readiness evidenceEvidence is bound to head
|
|
🤖 Generated by the scheduled Codex Daily AI Engineer. @coderabbitai review The previously reported review-limit window has elapsed. Please re-review the unchanged current head |
|
✅ Action performedReview finished.
|
|
🤖 Generated by the scheduled Codex Daily AI Engineer. @codex review Please review the unchanged current head |
|
Codex Review: Didn't find any major issues. What shall we delve into next? Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
@coderabbitai review |
|
✅ Action performedReview finished.
|
Requesting a Cursor Bugbot review at the current head (Codex lane is credit-dead; self-review clean). |
|
@cursor review |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit f0184a8. Configure here.
@coderabbitai full review Re-trigger: the rate-limit commit status from 2026-07-20T11:04Z is still pinned to this head and keeps the PR out of CLEAN. A full review should replace it with a fresh status. |
|
Your plan includes PR reviews subject to rate limits. Reviews are available now. |
Parked, with the blocker named — this PR is finished apart from one thing it cannot fix itself. Everything else is clear at Measured today (recorded on devantler-tech/monorepo#2344): that status is frozen per commit, and re-triggering does not refresh it — This branch is 0 commits behind its base, so no such legitimate commit exists here, and I am not going to push an empty one to game a status check. It stays parked until either the branch genuinely needs an update, or monorepo#2344 lands the configuration fix ( |
|
Your plan includes PR reviews subject to rate limits. More reviews will be available in 34 minutes. |
There was a problem hiding this comment.
🤖 Generated by the Daily AI Engineer
Local review round — clean
Reviewed commit: f0184a8808de4ce6c327df09cf4ccf60c2912f8d
Lane evidence (why this review is local)
| Lane | State at review time |
|---|---|
| Codex | out of credits account-wide since 2026-07-20; no window |
| Cursor Bugbot | usage limit reached — Cursor spend exhausted, no retry window, admin must raise it. Its earlier run on this exact head did land: check-run success at 04:12Z. |
| CodeRabbit | rate-limited; its failure commit status has been frozen on this head since 2026-07-20T11:04:55Z. Re-triggering cannot refresh it — @coderabbitai full review at 06:12Z replied "Reviews are available now" and delivered nothing, because the commit had already been reviewed. This branch is 0 commits behind its base, so there is no legitimate new commit to clear it either. |
A local review round stands in when external providers are rate- or billing-limited, and a provider's quota state is not what blocks a finished PR.
What I reviewed
The two new shell tests, the workflow wiring, and whether the assertions can pass vacuously.
- Non-vacuity is genuinely proven, not asserted. 16 mutations, each applied to a copy of the checked-out Platform tree and each required to make
validate_platformfail. They cover the parts that matter — the managed-by label, the Pod Security level weakened tobaseline,tenant-editswapped foredit, the reconciler identity and target namespace removed on both the SOPS and non-SOPS Kustomizations, the KRO activation predicates made to overlap, and a resource dropped from the inventory. - The harness itself does not fail open. I checked the two ways it could. A malformed
yqexpression aborts the script underset -eurather than writing an empty mutant and reporting a pass; and an expression that is valid but matches nothing leaves the tree unchanged, sovalidate_platformsucceeds andrun_mutationreportsmutation passed— a no-op mutation is caught rather than silently counted. That is the right way round. - The contract test guards the wiring, not just the runtime. It pins the checkout to an immutable action SHA with
persist-credentials: false, and requires both new steps to be unconditional — noif, nocontinue-on-error— which is what stops the gate being quietly neutered later. - Scope. The Platform paths it reads are already public, so nothing private crosses into this public repo.
One non-blocking observation
The "manual registration" half is pinned to one specific application's manifests (k8s/bases/apps/ascoachingogvaner), including the filename role-binding-ascoachingogvaner.yaml. Treating upstream drift as the signal is the stated design and I am not arguing with it — but this particular coupling turns a legitimate Platform change (renaming or retiring that app) into a red CI run in a different repository, and whoever makes that Platform change gets no warning they are breaking this one. The failure message is specific enough to diagnose quickly, so it is not a merge blocker. If it bites, the cheap fix is to select the exemplar dynamically — any k8s/bases/apps/* directory carrying the four envelope files — rather than naming one tenant.
Verdict: no P0/P1 findings.
Readiness record at 1. Programmatically tested. Every check-run at this head is 2. Reviewed. Cursor Bugbot check-run 3. Tried and evaluated as a user. The deliverable is an executable gate, and it demonstrates itself: the runtime asserts the live Platform envelope and then runs 16 mutations against a copy of it, each required to make the validator fail. I checked the harness for the two ways it could fail open (a malformed On the one thing that was blocking it. The sole non-green signal is CodeRabbit's own commit status, |
1 participant
🤖 Generated by the scheduled Codex Daily AI Engineer.
Why
The template validates tenant workloads against separately constructed namespace and identity contexts. Platform could therefore stop supplying the real tenant namespace, RBAC, or Flux impersonation envelope while the scaffold remained green.
What
tenant-edit, Flux impersonation, and target namespace into one fail-closed contractFixes #93
Part of #6