Background
The scripts/ci/clean-environment.sh cleanup script identifies test-owned resources by positive prefix-match on formae-plugin-sdk-test-* (and a couple of other prefixes for legacy fixtures). Every cleanup block uses this pattern. PR #95 added seven new blocks (CloudFront Functions, KVS, three policy types, OAC, ACM Certificate) following the same convention.
Problem
This works today because:
- The
aws-conformance-tests concurrency group in .github/workflows/ci.yml and .github/workflows/nightly.yml ensures only one CI run is in-flight at a time
- Developers are unlikely to manually create resources with that prefix outside conformance
But the convention isn't load-bearing — a parallel CI job in another repo, a developer experimenting locally with a matching prefix, or a future migration to non-serialized CI would all let the cleanup script delete resources it doesn't own. Codex flagged this in the 2026-06-03 adversarial review of the CloudFront/ACM design spec.
Proposed fix
Every cleanup block should require BOTH the test prefix AND a match against the active FORMAE_TEST_RUN_ID (the env var the conformance harness injects into every fixture).
Approach options:
- Tag everything with
FormaeTestRunID = $RUN_ID at create time, and filter on that tag in cleanup. Most robust but requires tagging support on every resource type (e.g., KVS doesn't carry tags via formae today).
- Encode the run-id into the resource name suffix (already true for most fixtures — e.g.,
formae-plugin-sdk-test-secret-abc123 where abc123 is the run-id). Cleanup grep can then check endswith against the run-id from CI.
- Hybrid: tag where possible, name-suffix elsewhere.
Probably option 2 is the cheapest: extend every cleanup block's JMESPath filter to also require the run-id substring. CI exposes FORMAE_TEST_RUN_ID as an env var; the cleanup script can either consume it directly or accept it as an argument.
Scope
This is a script-wide refactor — touches every cleanup block, not just the new ones from PR #95. Should be a single coherent PR rather than incremental per-block changes. Consider it a follow-up to PR #95 rather than blocking that merge.
References
Background
The
scripts/ci/clean-environment.shcleanup script identifies test-owned resources by positive prefix-match onformae-plugin-sdk-test-*(and a couple of other prefixes for legacy fixtures). Every cleanup block uses this pattern. PR #95 added seven new blocks (CloudFront Functions, KVS, three policy types, OAC, ACM Certificate) following the same convention.Problem
This works today because:
aws-conformance-testsconcurrency group in.github/workflows/ci.ymland.github/workflows/nightly.ymlensures only one CI run is in-flight at a timeBut the convention isn't load-bearing — a parallel CI job in another repo, a developer experimenting locally with a matching prefix, or a future migration to non-serialized CI would all let the cleanup script delete resources it doesn't own. Codex flagged this in the 2026-06-03 adversarial review of the CloudFront/ACM design spec.
Proposed fix
Every cleanup block should require BOTH the test prefix AND a match against the active
FORMAE_TEST_RUN_ID(the env var the conformance harness injects into every fixture).Approach options:
FormaeTestRunID = $RUN_IDat create time, and filter on that tag in cleanup. Most robust but requires tagging support on every resource type (e.g., KVS doesn't carry tags via formae today).formae-plugin-sdk-test-secret-abc123whereabc123is the run-id). Cleanup grep can then checkendswithagainst the run-id from CI.Probably option 2 is the cheapest: extend every cleanup block's JMESPath filter to also require the run-id substring. CI exposes
FORMAE_TEST_RUN_IDas an env var; the cleanup script can either consume it directly or accept it as an argument.Scope
This is a script-wide refactor — touches every cleanup block, not just the new ones from PR #95. Should be a single coherent PR rather than incremental per-block changes. Consider it a follow-up to PR #95 rather than blocking that merge.
References
engineering-notes/formae-plugin-aws/2026-06-03-cloudfront-acm-resources-design.md("Risks and mitigations" table)