fix: preserve Durable Object migration compatibility

[simple-agent-manager]

Summary

• Fixes fresh-install Cloudflare error 10099 without rewriting the immutable v1-v17 Durable Object history in apps/api/wrangler.toml.
• Reads the target API Worker's deployed migration_tag before generating environment configuration. Applied migrations remain structurally unchanged; only pending new_classes creates become new_sqlite_classes.
• A confirmed missing Worker receives an all-SQLite 17-class bootstrap. Unknown, duplicate, missing, or unreadable migration state fails before Wrangler can replay history.
• Canonical staging, production, and self-host deployments share the same generator; Pulumi does not create Durable Object namespaces separately.

Closes #1614

Migration reasoning and compatibility

Cloudflare migration tags are ordered and applied once, and namespace storage cannot be changed in place. A global rewrite of old new_classes entries would therefore be unsafe for existing KV-backed deployments. The resolver preserves the complete applied prefix and converts only pending creates:

• clean target: 17 tags, zero new_classes, 17 SQLite class creates;
• deployed v17: output is structurally identical to the checked-in history;
• partial deployment: applied entries remain exact and only future legacy creates become SQLite;
• unknown or unreadable state: deployment stops.

On a first two-pass deployment, pass one creates all classes as SQLite. Pass two reads v17; Wrangler has no pending migration to replay. Future deployments append and submit only tags after the deployed tag.

Validation

• pnpm lint
• pnpm typecheck
• pnpm test (6,208 API/Worker tests and 2,740 web tests)
• Additional validation: pnpm build; deployment-script TypeScript; Wrangler binding, D1/DO migration safety and ordering, AST, file-size, source-contract, manifest, and specialist-review harness gates
• Focused migration/generator suite: 37 tests; initial TDD suite failed 12/12 against the old verbatim-copy behavior
• N/A: no sweep/cron/alarm candidate-selection change

Staging Verification (REQUIRED for all code changes — merge-blocking)

• Staging deployment green — Deploy Staging run 29830545589 passed for commit 44548e8cb
• Live app verified via Playwright — the secret-backed staging suite ran against app.sammy.party
• Existing workflows confirmed working — all 12 dashboard/navigation/settings/health smoke tests passed
• New feature/fix verified on staging — generator detected sam-api-staging at v17; both API Worker deploys succeeded; tag and namespace backends were unchanged
• Infrastructure verification completed — no real VM was provisioned. The changed file is under scripts/deploy/, but the change is limited to Worker migration preflight/config generation. The existing secret-backed staging workflow does not provide a real-workspace/heartbeat scenario, and the task shell cannot read its smoke credential. This remains a human-review gate.
• N/A: no UI changes; mobile/desktop verification is not applicable

Staging Verification Evidence

Before and after the deployment, read-only Cloudflare namespace listings contained the same 17 classes. AdminLogs, CodexRefreshLock, GitHubUserAccessTokenLock, GitLabUserAccessTokenLock, NodeLifecycle, TaskRunner, and TrialEventBus remained KV-backed; the other ten remained SQLite-backed. The Worker migration tag remained v17. No production resource was mutated.

A live clean-account deployment could not be performed: the only accessible Cloudflare accounts are canonical staging and production, and both already contain legacy KV namespaces. Creating a throwaway Worker in either account would not exercise Cloudflare's account-level “no existing legacy namespace” restriction. Automated generated-TOML proof and Cloudflare's official contract cover that path; a genuinely clean account remains the exact external verification gap.

UI Compliance Checklist (Required for UI changes)

• N/A: no UI changes
• N/A: no accessibility changes
• N/A: no UI components changed
• N/A: no visual audit required

End-to-End Verification (Required for multi-component changes)

• Data flow traced from Cloudflare state to generated environment and Wrangler deploy
• Capability tests exercise the clean and existing happy paths across the mocked Cloudflare boundary and parsed TOML output
• All assumptions were checked against the full migration history, deployment workflows, Pulumi, Miniflare config, official docs, and read-only deployed state
• Remaining live clean-account and real-VM gaps are documented below

Data Flow Trace

1. main() in scripts/deploy/sync-wrangler-config.ts loads Pulumi outputs and derives the API Worker name.
2. getDeployedWorkerMigrationTag() proves the account listing is readable, reads migration_tag for an existing Worker, and uses an exact settings 404 only to confirm a missing Worker.
3. resolveDurableObjectMigrations() locates the applied tag, preserves entries through it, and converts only pending legacy creates.
4. generateApiWorkerEnv() places the resolved list in the generated environment; TOML serialization is exercised by scripts/quality/do-migration-compatibility.test.ts.
5. Both generator calls in .github/workflows/deploy-reusable.yml receive the deploy token and feed the same environment to Wrangler.

Untested Gaps

• No genuinely clean/new Cloudflare account credential was available for a live error-10099 bootstrap.
• No real VM/heartbeat check was run; the change does not alter VM provisioning, but the repository template path rule covers scripts/deploy/.
• The staging and production accounts were inspected read-only; production was never deployed or mutated.

Post-Mortem (Required for bug fix PRs)

What broke

Fresh SAM installations on newer Cloudflare accounts could not deploy the API Worker because historical new_classes migrations attempted to create the account's first KV-backed Durable Object namespace and Cloudflare returned error 10099.

Root cause

SAM used a single static migration list for two incompatible states: immutable existing deployments whose historical namespaces are KV-backed, and clean accounts where new KV namespaces are no longer supported. The generator copied that list verbatim.

Class of bug

Externally versioned infrastructure history was treated as state-independent configuration despite immutable storage backends and account-dependent bootstrap capability.

Why it wasn't caught

Tests verified binding copying but did not distinguish a clean target from an already-migrated Worker. Miniflare config bypasses Cloudflare's remote migration contract.

Process fix included in this PR

• .claude/rules/07-env-and-urls.md now requires immutable applied history, SQLite for new namespaces, fail-closed state detection, and clean/existing tests.
• scripts/quality/do-migration-compatibility.test.ts protects clean, v17, partial, duplicate, unknown, missing, unreadable, and generated-TOML behavior.
• Public self-hosting docs explain the automatic compatibility behavior and fail-closed preflight.

Post-mortem file

tasks/archive/2026-07-21-cloudflare-do-clean-install-migrations.md

Specialist Review Evidence (Required for agent-authored PRs)

• All repository specialist checklists completed locally and findings addressed
• Subagent reviewer attempts were interrupted; needs-human-review is applied and merge is deferred to a human

Reviewer	Status	Outcome
task-completion-validator	PASS	Checks A-F completed locally after three interrupted read-only reviewer attempts
cloudflare-specialist	PASS	Local checklist: immutable/applied history, staging parity, Miniflare, and deployment paths reviewed
test-engineer	ADDRESSED	Duplicate-tag and unreadable-list cases added in 44548e8cb; 37 focused tests pass
constitution-validator	PASS	External Cloudflare URL and bounded protocol page size are not business configuration
doc-sync-validator	PASS	Rule and public self-hosting docs match generator behavior

Exceptions (If any)

• Scope: live clean-account deployment and real-VM heartbeat proof were not available.
• Rationale: available Cloudflare accounts already have legacy namespaces; the existing secret-backed smoke workflow has no workspace-provisioning scenario.
• Expiration: human review on a genuinely clean Cloudflare account and, if deemed applicable to this Worker-only deploy script change, an authorized staging workspace check.

Agent Preflight (Required)

• Preflight completed before code changes

Classification

• external-api-change
• cross-component-change
• business-logic-change
• public-surface-change
• docs-sync-change
• security-sensitive-change
• ui-change
• infra-change

External References

Official documentation from Cloudflare consulted before implementation:

• https://developers.cloudflare.com/changelog/post/2026-07-09-restrict-new-kv-backed-namespaces/
• https://developers.cloudflare.com/durable-objects/reference/durable-object-class-migrations-legacy/
• https://developers.cloudflare.com/durable-objects/reference/durable-objects-migrations/
• https://developers.cloudflare.com/durable-objects/api/sqlite-storage-api/
• https://developers.cloudflare.com/api/resources/workers/subresources/scripts/
• https://developers.cloudflare.com/api/resources/durable_objects/subresources/namespaces/methods/list/

Codebase Impact Analysis

• scripts/deploy/sync-wrangler-config.ts: state probe, resolver, generated environment wiring
• scripts/deploy/types.ts: complete legacy migration directive typing
• .github/workflows/deploy-reusable.yml: inspected; both existing generator passes already provide token/account context
• apps/api/wrangler.toml: inspected complete v1-v17 history; intentionally unchanged
• Pulumi, self-host, staging, production, Tail Worker, and Miniflare paths were audited for parity

Documentation & Specs

• .claude/rules/07-env-and-urls.md
• apps/www/src/content/docs/docs/guides/self-hosting.mdx
• tasks/archive/2026-07-21-cloudflare-do-clean-install-migrations.md

Constitution & Risk Check

Principle XI was checked. The Cloudflare API URL is an external service contract and the page size is a bounded control-plane request, not deployment business configuration. Primary risks are incorrect deployed-state inference and migration replay; list-first readability proof, exact missing-Worker confirmation, unknown/duplicate-tag rejection, applied-prefix preservation, and staging snapshots mitigate them.

[codspeed-hq]

Merging this PR will not alter performance

✅ 6 untouched benchmarks

---

_{Comparing sam/implement-validate-backward-compatible-qsqb28 (421c6e5) with main (0c13372)}

[simple-agent-manager]

Independent deep verification of this PR (quadruple-check requested by Raphaël)

A separate session re-verified this PR from scratch with local tests, wrangler-source inspection, live read-only Cloudflare API checks, and two adversarial subagent reviews (cloudflare-specialist + general logic reviewer). Verdict: the core fix is correct — no correctness-breaking path was found. Findings below are hardening/verification gaps for human judgment, not blockers to the fix's logic.

Independently verified ✅

• Wrangler 4.106.0 source (wrangler-dist/cli.js pending-migration logic): uses the identical /workers/scripts listing + id match as getDeployedWorkerMigrationTag, submits only entries after the deployed tag, and replays the full history when a tag is absent/unknown — the fail-closed throws map exactly onto wrangler's dangerous paths.
• Mutation test: reverting resolveDurableObjectMigrations to verbatim copy fails 6/14 tests — the suite is genuinely discriminating.
• Full generation round-trip (real generateApiWorkerEnv → TOML stringify → parse): clean install = 17 SQLite creates / 0 KV creates; deployed v17 = byte-identical history, 0 pending steps; fork-at-v10 = verbatim applied prefix, all 7 pending creates SQLite. No scenario submits a KV-backed create.
• wrangler deploy --dry-run passes on both generated configs (full config validation + bundling); all 17 migration classes present in the bundle's exports.
• Live CF API (read-only, staging): list returns migration_tag: "v17" for sam-api-staging; nonexistent-worker /settings returns exactly HTTP 404 / code 10007; no pagination metadata on the list endpoint (fail-closed fallback covers hypothetical truncation).
• All 7 legacy-KV DO classes reviewed (NodeLifecycle, AdminLogs, TaskRunner, CodexRefreshLock, TrialEventBus, GitHub/GitLabUserAccessTokenLock): only backend-agnostic ctx.storage KV APIs / alarms / no storage — safe as SQLite-backed on clean installs.
• Probe/deploy worker-name divergence impossible (same DEPLOYMENT_CONFIG.resources.workerName(stack) in one process); no appliedIndex off-by-one; no mutation/TOML round-trip hazards; Pulumi creates no DO namespaces; both workflow sync invocations already pass CF_API_TOKEN; a thrown probe leaves no partial config on disk (throw precedes saveWranglerToml).

Findings (ranked)

1. HIGH (test protection): the new regression suite never runs in CI. No workflow invokes scripts/quality/*.test.ts (pnpm quality:specialist-review:test exists in package.json only). A future revert of the resolver keeps CI green. Suggested one-liner: add npx vitest run --config scripts/quality/vitest.config.ts to the Validate Deploy Scripts job.
2. MEDIUM (reproduced): env-fragile test. do-migration-compatibility.test.ts "requires the deployment token" stubs neither env nor fetch; with CF_API_TOKEN exported (normal in deploy shells) it makes a real network call and fails. Stub both env vars to '' and stub fetch.
3. MEDIUM (disclosed residual risk): the actual clean-account bootstrap was never run live — proof is unit/config-level + wrangler-source semantics. The one way to close it: run Deploy Production from a fork against a throwaway new Cloudflare account.
4. MEDIUM (operator UX): fail-closed states have no documented recovery. A fork whose deployed tag isn't in upstream history (e.g. lost local v18 after an upgrade merge) is permanently blocked; the error text doesn't warn that deleting the Worker to "recover" destroys all DO data.
5. MEDIUM (first-deploy pass 2): the re-sync re-probes the listing seconds after pass 1 created the Worker. Any propagation lag → confusing hard fail at the end of a first install (self-heals on re-run; wrangler independently recomputes, so never wrong migrations). Consider a bounded retry or a "safe to re-run" hint in the error.
6. MEDIUM (hardening): append-only migration ordering is not CI-enforced while the new rule text declares applied history immutable. (Note: a mid-array insertion cannot ship unconverted legacy creates to fresh installs — appliedIndex=-1 converts everything; the actual hazard is wrangler silently skipping the inserted entry for existing deployments, which is wrangler's own index-slicing semantics and pre-existing.)
7. LOW (doc sync): CLAUDE.md line ~148 still says the sync script "copies static bindings (Durable Objects, AI, migrations)". Also two now-false wrangler.toml comments: v6 CodexRefreshLock "Intentionally stateless" (uses ctx.storage for rate-limit + rotated-tokens) and v8 TrialEventBus "In-memory only" (persists a closed flag).
8. LOW (secret-leak landmine): GET /workers/scripts/{name}/settings returns plaintext SETUP_TOKEN in its body (verified live). Current code only reads .status on the exists path — add a comment/test pinning "never read the body" so a future debug-log edit can't leak it into Actions logs.
9. LOW: wrangler resolves migrations via inheritable() (silent top-level fallback) — the "not inherited by environments" grep-guard cannot protect it if env.*.migrations were ever omitted. Currently always emitted and pinned by a test; worth a note in rule 07.
10. LOW: fetchCloudflareMigrationState discards the underlying fetch error (DNS vs TLS vs timeout indistinguishable in deploy logs).
11. INFO (pre-existing, untouched by this PR): the pass-2 re-sync env block omits CF_CONTAINER_*/SANDBOX_* vars present in pass 1 — irrelevant to migrations (all migration-relevant vars are present in both passes) but first-deploy configs silently lose those operator overrides until the next deploy.

Local gates re-run in this verification: scripts/quality vitest suite 129/129 pass, deploy-scripts tsc, quality:wrangler-bindings, wrangler dry-run ×2. Worktree left byte-clean.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[simple-agent-manager]

Verification follow-ups implemented (medium/high findings from the review above)

Four commits (a7a2f8f97, cdd38bfb4, 9b9ad03e5, 421c6e5dc) address every actionable medium/high finding from the independent verification:

Finding	Fix
HIGH 1 — regression suite not in CI	Validate Deploy Scripts now runs the full scripts/quality vitest suite (133 tests incl. the migration-compatibility regressions) via pnpm quality:scripts:test (renamed from quality:specialist-review:test, no npx per SonarCloud). Proven green in CI.
MEDIUM 2 — env-fragile token tests	Both token-requirement tests (migration probe + tail-worker) stub env + fetch and assert zero network calls; suite passes with and without CF_TOKEN/CF_API_TOKEN exported.
MEDIUM 4 — no recovery guidance	Every fail-closed error now says what to do: transient states → "re-running is safe"; tag-not-in-history → restore dropped fork-local entries; all paths warn that deleting the Worker destroys all Durable Object data. Matching recovery section added to the self-hosting guide.
MEDIUM 5 — first-deploy pass-2 listing lag	The read-only probe retries transient failures: DO_MIGRATION_STATE_PROBE_ATTEMPTS (default 3) / DO_MIGRATION_STATE_PROBE_RETRY_DELAY_MS (default 2000), env-configurable and passed through both deploy-reusable.yml sync steps. New tests: retry-then-succeed, bounded exhaustion, invalid-env fallback.
MEDIUM 6 — append-only not enforced	New test pins sequential v1..vN tags on the checked-in history (mutation-proven: fails when two entries are swapped); rule 07 documents the invariant plus the wrangler inheritable() asymmetry for migrations.
MEDIUM 3 — live clean-account bootstrap	Not addressable — requires a brand-new Cloudflare account; remains the disclosed human-review gap.

Also: the migration-state logic moved to scripts/deploy/durable-object-migrations.ts (rule 18 — the file would have crossed 800 lines), and the stale CLAUDE.md "copies … migrations" line was synced (rule 01).

Latent CI bug found and fixed while wiring the suite in

Five quality-checker scripts ran main() unconditionally at import. Under vitest in Actions (where GITHUB_EVENT_NAME/GITHUB_EVENT_PATH are set and CF_TOKEN is not), the specialist-review checker evaluated the real PR event and the observability checker process.exit(2)d inside the test worker — invisible locally where CF_TOKEN exists. All five now use the same direct-execution guard as sync-wrangler-config.ts; every pnpm quality:* direct invocation verified to still execute its check, and the suite passes a CI-faithful local run (tokens unset, GitHub event env set).

Verification evidence

• CI (421c6e5dc): all checks green except Specialist Review Evidence, which is the intentional needs-human-review label gate. The new script-test step passes in CI.
• Staging: Deploy Staging run 29855492925 succeeded for 9b9ad03e5; the sync log shows API worker "sam-api-staging" migration state: v17 (first attempt, no retries needed). Post-deploy read-only checks: migration_tag still v17, namespace backends unchanged, api.sammy.party/health 200.
• Tip coverage: git diff 9b9ad03e5..421c6e5dc touches only the four CI-only checker scripts — deploy-path code is byte-identical to the staging-verified commit.
• Generation invariants re-confirmed at tip: clean install = 17 SQLite / 0 KV creates; deployed v17 = byte-identical history, 0 pending; v10 fork = verbatim applied prefix, all-SQLite pending.

The needs-human-review label and merge decision remain with a human, per policy.