diff --git a/.claude/rules/07-env-and-urls.md b/.claude/rules/07-env-and-urls.md index 37bba3aa9..db51f0f8c 100644 --- a/.claude/rules/07-env-and-urls.md +++ b/.claude/rules/07-env-and-urls.md @@ -57,7 +57,7 @@ Examples: Environment-specific sections (`[env.staging]`, `[env.production]`) are NOT checked into the repository. They are generated dynamically at deploy time by `scripts/deploy/sync-wrangler-config.ts`, which: 1. Reads Pulumi stack outputs for dynamic bindings (D1 IDs, KV IDs, R2 bucket names) -2. Copies static bindings from the top-level config (Durable Objects, AI, migrations) +2. Copies static Durable Object and AI bindings, then resolves Durable Object migrations against the target Worker's deployed tag 3. Derives worker names from `DEPLOYMENT_CONFIG` in `scripts/deploy/config.ts` 4. Conditionally adds `tail_consumers` (only if the tail worker already exists) @@ -65,10 +65,21 @@ Environment-specific sections (`[env.staging]`, `[env.production]`) are NOT chec Add the binding to the **top-level section of `wrangler.toml` only**. The sync script handles the rest. -- **Static bindings** (Durable Objects, AI, migrations): Copied verbatim from top-level to generated env sections. +- **Static bindings** (Durable Objects, AI): Copied verbatim from top-level to generated env sections. +- **Durable Object migrations**: The applied prefix is copied verbatim; only pending legacy `new_classes` creates are emitted as `new_sqlite_classes`. - **Dynamic bindings** (D1, KV, R2): Generated from Pulumi outputs with correct resource IDs per environment. - **Derived bindings** (worker name, routes, tail_consumers): Computed from `DEPLOYMENT_CONFIG` naming conventions. +### Durable Object migration safety + +- Applied migration tags are immutable history. Never rewrite an applied `new_classes`, rename, delete, or transfer entry to change storage or behavior. +- New Durable Object namespaces MUST use `new_sqlite_classes`. +- The sync script MUST confirm whether the target Worker is absent or read its deployed `migration_tag` before generating an environment. +- A missing, unreadable, duplicated, or unknown migration tag MUST fail the deployment preflight. Never assume an ambiguous Worker is a clean install; Wrangler can otherwise submit the full local history. The probe retries transient failures a bounded number of times (`DO_MIGRATION_STATE_PROBE_ATTEMPTS`, `DO_MIGRATION_STATE_PROBE_RETRY_DELAY_MS`) before failing closed. +- The `[[migrations]]` array is append-only with sequential `v1..vN` tags. The resolver (and Wrangler) treat array position relative to the deployed tag as the applied/pending boundary, so inserting or reordering entries silently corrupts that boundary. The compatibility test suite enforces the sequence and runs in the `Validate Deploy Scripts` CI job (`scripts/quality/do-migration-compatibility.test.ts`). +- Wrangler resolves `migrations` via its `inheritable()` config path: if `env.*.migrations` were ever omitted, Wrangler silently falls back to the top-level (legacy) array with NO "not inherited by environments" warning — the deploy grep-guard does not protect this field. The generated-environment test pins that `env.*.migrations` is always emitted. +- Test both a clean bootstrap and an existing deployment at the latest historical tag whenever migration generation changes. Miniflare alone does not exercise the remote migration contract. + The CI quality check (`pnpm quality:wrangler-bindings`) verifies: 1. No `[env.*]` sections exist in checked-in `wrangler.toml` files diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 06e3a2b21..ae22ed141 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -311,6 +311,11 @@ jobs: npx tsx --check scripts/deploy/sync-wrangler-config.ts npx tsx --check scripts/deploy/generate-keys.ts + # Deploy/quality script behavior — includes the Durable Object migration + # compatibility regression suite (clean install vs applied history). + - name: Run deploy and quality script tests + run: pnpm quality:scripts:test + - name: Validate wrangler binding consistency run: pnpm quality:wrangler-bindings diff --git a/.github/workflows/deploy-reusable.yml b/.github/workflows/deploy-reusable.yml index 0a57cd90a..bb6677920 100644 --- a/.github/workflows/deploy-reusable.yml +++ b/.github/workflows/deploy-reusable.yml @@ -357,6 +357,8 @@ jobs: REQUIRE_APPROVAL: ${{ vars.REQUIRE_APPROVAL }} HETZNER_BASE_IMAGE: ${{ vars.HETZNER_BASE_IMAGE }} ARTIFACTS_BINDING_ENABLED: ${{ vars.ARTIFACTS_BINDING_ENABLED }} + DO_MIGRATION_STATE_PROBE_ATTEMPTS: ${{ vars.DO_MIGRATION_STATE_PROBE_ATTEMPTS }} + DO_MIGRATION_STATE_PROBE_RETRY_DELAY_MS: ${{ vars.DO_MIGRATION_STATE_PROBE_RETRY_DELAY_MS }} CF_CONTAINER_ENABLED: ${{ vars.CF_CONTAINER_ENABLED }} CF_CONTAINER_SLEEP_AFTER: ${{ vars.CF_CONTAINER_SLEEP_AFTER }} CF_CONTAINER_PORT_READY_TIMEOUT_MS: ${{ vars.CF_CONTAINER_PORT_READY_TIMEOUT_MS }} @@ -668,6 +670,8 @@ jobs: REQUIRE_APPROVAL: ${{ vars.REQUIRE_APPROVAL }} HETZNER_BASE_IMAGE: ${{ vars.HETZNER_BASE_IMAGE }} ARTIFACTS_BINDING_ENABLED: ${{ vars.ARTIFACTS_BINDING_ENABLED }} + DO_MIGRATION_STATE_PROBE_ATTEMPTS: ${{ vars.DO_MIGRATION_STATE_PROBE_ATTEMPTS }} + DO_MIGRATION_STATE_PROBE_RETRY_DELAY_MS: ${{ vars.DO_MIGRATION_STATE_PROBE_RETRY_DELAY_MS }} - name: Re-deploy API Worker (with tail_consumers) if: ${{ inputs.dry_run != true && steps.first_deploy.outputs.is_first == 'true' }} diff --git a/CLAUDE.md b/CLAUDE.md index 31af9f170..2bd0ac83f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -145,7 +145,7 @@ Full env var reference: use the `env-reference` skill or see `apps/api/.env.exam ## Wrangler Binding Rule (CRITICAL) -Environment-specific `[env.*]` sections are NOT checked into the repository. They are generated at deploy time by `scripts/deploy/sync-wrangler-config.ts` from Pulumi outputs + the top-level config. When adding ANY new binding to `wrangler.toml`, add it to the **top-level section only**. The sync script copies static bindings (Durable Objects, AI, migrations) and generates dynamic bindings (D1, KV, R2, worker name, routes, tail_consumers) automatically. The CI quality check (`pnpm quality:wrangler-bindings`) verifies that no env sections are committed and that required binding types are present at the top level. See `.claude/rules/07-env-and-urls.md` for details. +Environment-specific `[env.*]` sections are NOT checked into the repository. They are generated at deploy time by `scripts/deploy/sync-wrangler-config.ts` from Pulumi outputs + the top-level config. When adding ANY new binding to `wrangler.toml`, add it to the **top-level section only**. The sync script copies static bindings (Durable Objects, AI), resolves Durable Object migrations against the target Worker's deployed migration tag, and generates dynamic bindings (D1, KV, R2, worker name, routes, tail_consumers) automatically. The CI quality check (`pnpm quality:wrangler-bindings`) verifies that no env sections are committed and that required binding types are present at the top level. See `.claude/rules/07-env-and-urls.md` for details. ## Architecture Principles diff --git a/apps/www/src/content/docs/docs/guides/self-hosting.mdx b/apps/www/src/content/docs/docs/guides/self-hosting.mdx index 18a6cb95c..1be7d618f 100644 --- a/apps/www/src/content/docs/docs/guides/self-hosting.mdx +++ b/apps/www/src/content/docs/docs/guides/self-hosting.mdx @@ -215,6 +215,16 @@ The workflow: 5. Builds and uploads VM Agent binaries 6. Runs health check +Before deploying the API Worker, the workflow reads its applied Durable Object migration tag. A fresh installation creates every namespace with SQLite storage; an existing installation retains its already-applied namespace history and storage backends. This is automatic—do not edit historical migration entries or create namespaces manually. + +If migration state cannot be read or does not match the checked-in history, deployment stops before Wrangler runs rather than risking migration replay or namespace replacement. + +If deployment stops with a Durable Object migration-state error: + +- **Errors mentioning listing lag or a Worker "created moments ago" are transient.** The workflow already retries the state probe a few times (tune with the `DO_MIGRATION_STATE_PROBE_ATTEMPTS` and `DO_MIGRATION_STATE_PROBE_RETRY_DELAY_MS` GitHub repository variables); re-running the deployment is safe and resumes cleanly. +- **"tag is not present in the checked-in history"** means `apps/api/wrangler.toml` lost migration entries that were already applied to your Worker — usually an upgrade merge that dropped fork-local `[[migrations]]` entries. Restore the missing entries so the deployed tag appears in the history, then redeploy. +- **Never delete the API Worker to recover.** Deleting a Worker destroys every Durable Object namespace behind it — all chat sessions, messages, and task state. There is no undo. + ## Verification After deployment completes: diff --git a/package.json b/package.json index ad2c2fbea..156727c6c 100644 --- a/package.json +++ b/package.json @@ -43,7 +43,7 @@ "quality:do-migration-safety": "tsx scripts/quality/check-do-migration-safety.ts", "quality:source-contract-tests": "tsx scripts/quality/check-source-contract-tests.ts", "quality:specialist-review": "tsx scripts/quality/check-specialist-review-evidence.ts", - "quality:specialist-review:test": "npx vitest run --config scripts/quality/vitest.config.ts", + "quality:scripts:test": "vitest run --config scripts/quality/vitest.config.ts", "quality:do-wall-time": "tsx scripts/quality/check-do-wall-time.ts", "quality:observability-noise": "tsx scripts/quality/check-observability-noise.ts", "test:infra": "pnpm --filter @simple-agent-manager/infra test", diff --git a/scripts/deploy/durable-object-migrations.ts b/scripts/deploy/durable-object-migrations.ts new file mode 100644 index 000000000..40984f7ab --- /dev/null +++ b/scripts/deploy/durable-object-migrations.ts @@ -0,0 +1,247 @@ +/** + * Durable Object migration compatibility resolution. + * + * Reads the target API Worker's deployed migration state from Cloudflare and + * resolves the checked-in migration history against it: applied entries are + * preserved exactly, and only pending legacy `new_classes` creates are emitted + * as `new_sqlite_classes` (new Cloudflare accounts cannot create KV-backed + * Durable Object namespaces — error 10099, issue #1614). + * + * Every ambiguous state fails closed: Wrangler treats an unknown deployed tag + * as a reason to submit the entire local migration history, which can replay + * immutable namespace history. + */ + +import type { MigrationEntry } from './types.js'; + +const CLOUDFLARE_API_BASE_URL = 'https://api.cloudflare.com/client/v4'; +const WORKERS_LIST_PAGE_SIZE = 1000; + +/** + * The migration-state probe runs immediately after `wrangler deploy` on the + * two-pass first install, where the scripts listing can briefly lag the + * just-created Worker. The probe is read-only and idempotent, so transient + * failures are retried a bounded number of times before failing the deploy. + */ +const DEFAULT_MIGRATION_STATE_PROBE_ATTEMPTS = 3; +const DEFAULT_MIGRATION_STATE_PROBE_RETRY_DELAY_MS = 2000; + +const NEVER_DELETE_WORKER_GUIDANCE = + 'Never delete a Worker that has served SAM traffic to recover from this: deleting it destroys all Durable Object data.'; + +function requireCloudflareApiToken(operation: string): string { + const apiToken = process.env.CF_API_TOKEN || process.env.CLOUDFLARE_API_TOKEN; + if (!apiToken) { + throw new Error(`CF_API_TOKEN or CLOUDFLARE_API_TOKEN is required to ${operation}`); + } + return apiToken; +} + +function readBoundedIntEnv(name: string, fallback: number, minimum: number): number { + const raw = process.env[name]; + if (!raw) { + return fallback; + } + const parsed = Number.parseInt(raw, 10); + if (!Number.isSafeInteger(parsed) || parsed < minimum) { + console.warn(` ${name}="${raw}" is not an integer >= ${minimum}; using default ${fallback}`); + return fallback; + } + return parsed; +} + +async function fetchCloudflareMigrationState( + url: string, + apiToken: string, + operation: string +): Promise { + try { + return await fetch(url, { headers: { Authorization: `Bearer ${apiToken}` } }); + } catch { + throw new Error(`${operation}: Cloudflare API request failed`); + } +} + +/** + * Single read of the deployed migration state. Returns null only when + * Cloudflare's exact Worker settings endpoint confirms that the target Worker + * does not exist. For an existing Worker, reads the latest applied Durable + * Object migration tag from Workers script metadata. + * + * SECURITY: the settings endpoint's 200 response body contains plaintext + * values for every `plain_text` binding, including the live SETUP_TOKEN. + * Only the HTTP status may be inspected here — never read or log the body. + */ +async function probeDeployedWorkerMigrationTag( + accountId: string, + workerName: string, + apiToken: string +): Promise { + const scriptsUrl = + `${CLOUDFLARE_API_BASE_URL}/accounts/${encodeURIComponent(accountId)}` + + `/workers/scripts?per_page=${WORKERS_LIST_PAGE_SIZE}`; + const listOperation = `Failed to list Workers while reading migration state for "${workerName}"`; + const scriptsResponse = await fetchCloudflareMigrationState(scriptsUrl, apiToken, listOperation); + if (!scriptsResponse.ok) { + throw new Error(`${listOperation} (HTTP ${scriptsResponse.status})`); + } + + let payload: unknown; + try { + payload = await scriptsResponse.json(); + } catch { + throw new Error(`${listOperation}: Cloudflare returned an invalid JSON response`); + } + if ( + typeof payload !== 'object' || + payload === null || + !('success' in payload) || + payload.success !== true || + !('result' in payload) || + !Array.isArray(payload.result) + ) { + throw new Error(`${listOperation}: Cloudflare returned an invalid response`); + } + + const worker = payload.result.find( + (candidate): candidate is Record => + typeof candidate === 'object' && + candidate !== null && + 'id' in candidate && + candidate.id === workerName + ); + if (!worker) { + const readOperation = `Failed to read Durable Object migration state for Worker "${workerName}"`; + const settingsUrl = + `${CLOUDFLARE_API_BASE_URL}/accounts/${encodeURIComponent(accountId)}` + + `/workers/scripts/${encodeURIComponent(workerName)}/settings`; + const settingsResponse = await fetchCloudflareMigrationState( + settingsUrl, + apiToken, + readOperation + ); + + if (settingsResponse.status === 404) { + return null; + } + if (!settingsResponse.ok) { + throw new Error(`${readOperation} (HTTP ${settingsResponse.status})`); + } + throw new Error( + `Worker "${workerName}" exists but is absent from the Workers scripts listing; ` + + `refusing to assume clean migration state. This can be transient listing lag ` + + `immediately after the Worker was created — re-running the deployment is safe.` + ); + } + if (typeof worker.migration_tag !== 'string' || worker.migration_tag.length === 0) { + throw new Error( + `Existing Worker "${workerName}" has no migration_tag; refusing to replay Durable Object migrations. ` + + `If the Worker was created moments ago this can be propagation lag — re-running the deployment is safe. ` + + `Otherwise this Worker was not created by SAM's deploy pipeline and must be investigated manually. ` + + NEVER_DELETE_WORKER_GUIDANCE + ); + } + return worker.migration_tag; +} + +/** + * Reads the deployed Durable Object migration tag with bounded retries. + * + * This intentionally fails closed on incomplete or unreadable state. Wrangler + * otherwise treats an unknown deployed tag as a reason to submit every local + * migration, which can replay immutable namespace history. All probe failures + * are retried (the probe is read-only and idempotent); a missing API token is + * a deterministic configuration error and fails immediately. + */ +export async function getDeployedWorkerMigrationTag( + accountId: string, + workerName: string +): Promise { + const apiToken = requireCloudflareApiToken('read Durable Object migration state'); + const attempts = readBoundedIntEnv( + 'DO_MIGRATION_STATE_PROBE_ATTEMPTS', + DEFAULT_MIGRATION_STATE_PROBE_ATTEMPTS, + 1 + ); + const retryDelayMs = readBoundedIntEnv( + 'DO_MIGRATION_STATE_PROBE_RETRY_DELAY_MS', + DEFAULT_MIGRATION_STATE_PROBE_RETRY_DELAY_MS, + 0 + ); + + let lastError: unknown; + for (let attempt = 1; attempt <= attempts; attempt += 1) { + try { + return await probeDeployedWorkerMigrationTag(accountId, workerName, apiToken); + } catch (error) { + lastError = error; + if (attempt < attempts) { + const message = error instanceof Error ? error.message : String(error); + console.warn( + ` Durable Object migration state probe attempt ${attempt}/${attempts} failed: ${message}` + ); + console.warn(` Retrying in ${retryDelayMs}ms...`); + await new Promise((resolveDelay) => setTimeout(resolveDelay, retryDelayMs)); + } + } + } + throw lastError; +} + +/** + * Preserve applied migration entries exactly, while ensuring every pending + * namespace create uses Cloudflare's supported SQLite storage backend. + */ +export function resolveDurableObjectMigrations( + migrations: MigrationEntry[] | undefined, + deployedMigrationTag: string | null +): MigrationEntry[] | undefined { + if (!migrations || migrations.length === 0) { + if (deployedMigrationTag !== null) { + throw new Error( + `Deployed Durable Object migration tag "${deployedMigrationTag}" cannot be reconciled because the checked-in migration history is empty. ` + + `Restore the [[migrations]] entries in apps/api/wrangler.toml so the deployed tag appears in the history, then redeploy. ` + + NEVER_DELETE_WORKER_GUIDANCE + ); + } + return undefined; + } + + const duplicateTag = migrations.find( + (migration, index) => + migrations.findIndex((candidate) => candidate.tag === migration.tag) !== index + )?.tag; + if (duplicateTag) { + throw new Error(`Durable Object migration tag "${duplicateTag}" is duplicated`); + } + + const appliedIndex = + deployedMigrationTag === null + ? -1 + : migrations.findIndex((migration) => migration.tag === deployedMigrationTag); + if (deployedMigrationTag !== null && appliedIndex === -1) { + throw new Error( + `Deployed Durable Object migration tag "${deployedMigrationTag}" is not present in the checked-in history; refusing to replay migrations. ` + + `The checked-in wrangler.toml is missing migration entries that were already applied to this Worker ` + + `(often an upgrade merge that dropped fork-local migrations). Restore the missing entries so the ` + + `deployed tag appears in the history, then redeploy. ` + + NEVER_DELETE_WORKER_GUIDANCE + ); + } + + return migrations.map((migration, index) => { + if (index <= appliedIndex || !migration.new_classes?.length) { + return migration; + } + + const { + new_classes: legacyClasses, + new_sqlite_classes: sqliteClasses, + ...unchanged + } = migration; + return { + ...unchanged, + new_sqlite_classes: [...(sqliteClasses ?? []), ...legacyClasses], + }; + }); +} diff --git a/scripts/deploy/sync-wrangler-config.ts b/scripts/deploy/sync-wrangler-config.ts index d50631ce4..7863b0622 100644 --- a/scripts/deploy/sync-wrangler-config.ts +++ b/scripts/deploy/sync-wrangler-config.ts @@ -6,9 +6,10 @@ * at deploy time. The checked-in wrangler.toml files contain only top-level * config for local dev — all environment sections are generated here. * - * Static bindings (Durable Objects, AI, migrations) are copied from the - * top-level config. Dynamic bindings (D1 IDs, KV IDs, R2 names) come from - * Pulumi stack outputs. Worker names are derived from DEPLOYMENT_CONFIG. + * Static bindings (Durable Objects and AI) are copied from the top-level + * config. Durable Object migrations are resolved against deployed Worker + * state. Dynamic bindings (D1 IDs, KV IDs, R2 names) come from Pulumi stack + * outputs. Worker names are derived from DEPLOYMENT_CONFIG. * * Usage: * PULUMI_STACK=prod pnpm tsx scripts/deploy/sync-wrangler-config.ts @@ -23,6 +24,10 @@ import * as TOML from '@iarna/toml'; import * as v from 'valibot'; import { DEPLOYMENT_CONFIG } from './config.js'; +import { + getDeployedWorkerMigrationTag, + resolveDurableObjectMigrations, +} from './durable-object-migrations.js'; import type { AIBinding, AnalyticsEngineDatasetBinding, @@ -44,6 +49,10 @@ const DEPLOY_STATE_DIR = resolve(import.meta.dirname, '../../.wrangler'); const FIRST_DEPLOY_MARKER = resolve(DEPLOY_STATE_DIR, 'tail-worker-first-deploy'); const SETUP_TOKEN_BYTES = 24; +// Re-exported so tests and callers keep a single import site while the +// migration-state logic lives in its own module (file size rule 18). +export { getDeployedWorkerMigrationTag, resolveDurableObjectMigrations }; + const recordSchema = v.custom>( (value) => typeof value === 'object' && value !== null && !Array.isArray(value), 'Expected an object' @@ -381,13 +390,14 @@ function getAnalyticsEngineDatasets( function getStaticApiWorkerBindings( staticBindings: StaticBindings, analyticsEngineDatasets: AnalyticsEngineDatasetBinding[] | undefined, - includeArtifactsBinding: boolean + includeArtifactsBinding: boolean, + durableObjectMigrations: MigrationEntry[] | undefined ): Partial { return { ...(staticBindings.durable_objects ? { durable_objects: staticBindings.durable_objects } : {}), ...(staticBindings.ai ? { ai: staticBindings.ai } : {}), ...(analyticsEngineDatasets ? { analytics_engine_datasets: analyticsEngineDatasets } : {}), - ...(staticBindings.migrations ? { migrations: staticBindings.migrations } : {}), + ...(durableObjectMigrations ? { migrations: durableObjectMigrations } : {}), ...(staticBindings.containers ? { containers: staticBindings.containers } : {}), ...(includeArtifactsBinding ? { artifacts: staticBindings.artifacts } : {}), }; @@ -405,9 +415,14 @@ export function generateApiWorkerEnv( outputs: PulumiOutputs, stack: string, includeTailConsumers: boolean, - artifactsBindingEnabled: boolean + artifactsBindingEnabled: boolean, + deployedMigrationTag: string | null ): WranglerEnvConfig { const staticBindings = extractStaticBindings(topLevel); + const durableObjectMigrations = resolveDurableObjectMigrations( + staticBindings.migrations, + deployedMigrationTag + ); if (artifactsBindingEnabled && !staticBindings.artifacts) { throw new Error( 'Artifacts is enabled but no top-level [[artifacts]] binding exists in wrangler.toml' @@ -475,7 +490,12 @@ export function generateApiWorkerEnv( r2_buckets: [{ binding: 'R2', bucket_name: outputs.r2Name }], // Static bindings copied from top-level config - ...getStaticApiWorkerBindings(staticBindings, analyticsEngineDatasets, includeArtifactsBinding), + ...getStaticApiWorkerBindings( + staticBindings, + analyticsEngineDatasets, + includeArtifactsBinding, + durableObjectMigrations + ), // Tail consumers (conditional — omitted on first deploy when tail worker doesn't exist) ...getTailConsumers(includeTailConsumers, tailWorkerName), @@ -552,6 +572,15 @@ async function main(): Promise { const config = loadWranglerToml(); const envKey = DEPLOYMENT_CONFIG.getEnvironmentFromStack(stack); + const apiWorkerName = DEPLOYMENT_CONFIG.resources.workerName(stack); + const deployedMigrationTag = await getDeployedWorkerMigrationTag( + outputs.cloudflareAccountId, + apiWorkerName + ); + console.log( + ` API worker "${apiWorkerName}" migration state: ${deployedMigrationTag ?? 'clean install'}` + ); + // Check if tail worker already exists (for conditional tail_consumers) const tailWorkerName = DEPLOYMENT_CONFIG.resources.tailWorkerName(stack); const hasTailWorker = await checkTailWorkerExists(outputs.cloudflareAccountId, tailWorkerName); @@ -584,7 +613,8 @@ async function main(): Promise { outputs, stack, hasTailWorker, - artifactsBindingEnabled + artifactsBindingEnabled, + deployedMigrationTag ); saveWranglerToml(config); console.log(`Updated wrangler.toml [env.${envKey}]`); diff --git a/scripts/deploy/types.ts b/scripts/deploy/types.ts index 215ef9872..9905de6ce 100644 --- a/scripts/deploy/types.ts +++ b/scripts/deploy/types.ts @@ -434,6 +434,16 @@ export interface MigrationEntry { tag: string; new_sqlite_classes?: string[]; new_classes?: string[]; + renamed_classes?: Array<{ + from: string; + to: string; + }>; + deleted_classes?: string[]; + transferred_classes?: Array<{ + from: string; + from_script: string; + to: string; + }>; } export interface AnalyticsEngineDatasetBinding { diff --git a/scripts/quality/check-do-migration-safety.ts b/scripts/quality/check-do-migration-safety.ts index 1b4fd7f63..2f8993912 100644 --- a/scripts/quality/check-do-migration-safety.ts +++ b/scripts/quality/check-do-migration-safety.ts @@ -264,4 +264,8 @@ function main(): void { ); } -main(); +// Only run main when executed directly (not when imported for testing). +const isDirectExecution = process.argv[1]?.endsWith('check-do-migration-safety.ts'); +if (isDirectExecution) { + main(); +} diff --git a/scripts/quality/check-migration-ordering.ts b/scripts/quality/check-migration-ordering.ts index 3fb9f6b8e..6ad1a743d 100644 --- a/scripts/quality/check-migration-ordering.ts +++ b/scripts/quality/check-migration-ordering.ts @@ -116,4 +116,8 @@ function main(): void { ); } -main(); +// Only run main when executed directly (not when imported for testing). +const isDirectExecution = process.argv[1]?.endsWith('check-migration-ordering.ts'); +if (isDirectExecution) { + main(); +} diff --git a/scripts/quality/check-migration-safety.ts b/scripts/quality/check-migration-safety.ts index 3a69232d9..7022ca208 100644 --- a/scripts/quality/check-migration-safety.ts +++ b/scripts/quality/check-migration-safety.ts @@ -396,4 +396,8 @@ function main(): void { ); } -main(); +// Only run main when executed directly (not when imported for testing). +const isDirectExecution = process.argv[1]?.endsWith('check-migration-safety.ts'); +if (isDirectExecution) { + main(); +} diff --git a/scripts/quality/check-observability-noise.ts b/scripts/quality/check-observability-noise.ts index 6d006eaf2..386f1bc44 100644 --- a/scripts/quality/check-observability-noise.ts +++ b/scripts/quality/check-observability-noise.ts @@ -464,7 +464,11 @@ async function main() { process.exit(0); } -main().catch((err) => { - console.error('Fatal error:', err instanceof Error ? err.message : String(err)); - process.exit(2); -}); +// Only run main when executed directly (not when imported for testing). +const isDirectExecution = process.argv[1]?.endsWith('check-observability-noise.ts'); +if (isDirectExecution) { + main().catch((err) => { + console.error('Fatal error:', err instanceof Error ? err.message : String(err)); + process.exit(2); + }); +} diff --git a/scripts/quality/check-specialist-review-evidence.ts b/scripts/quality/check-specialist-review-evidence.ts index 1c2a31b8c..376040ad2 100644 --- a/scripts/quality/check-specialist-review-evidence.ts +++ b/scripts/quality/check-specialist-review-evidence.ts @@ -302,4 +302,11 @@ function main(): void { } } -main(); +// Only run main when executed directly (not when imported for testing). +// Under vitest in CI, GITHUB_EVENT_NAME/GITHUB_EVENT_PATH are set, so an +// import-time main() would evaluate the real PR and process.exit inside the +// test runner. +const isDirectExecution = process.argv[1]?.endsWith('check-specialist-review-evidence.ts'); +if (isDirectExecution) { + main(); +} diff --git a/scripts/quality/check-wrangler-bindings.ts b/scripts/quality/check-wrangler-bindings.ts index 6b8409956..e07d7075e 100644 --- a/scripts/quality/check-wrangler-bindings.ts +++ b/scripts/quality/check-wrangler-bindings.ts @@ -7,9 +7,8 @@ * at deploy time by scripts/deploy/sync-wrangler-config.ts. * * 2. Top-level config has all required binding types — the sync script copies - * static bindings (Durable Objects, AI, migrations) from top-level into - * generated env sections. If they're missing at the top level, they'll be - * missing at runtime. + * static bindings and resolves migrations from the top-level declarations. + * If they're missing at the top level, they'll be missing at runtime. * * This check runs in CI to prevent misconfigurations. */ @@ -18,7 +17,10 @@ import { resolve } from 'node:path'; import * as TOML from '@iarna/toml'; const API_WRANGLER_PATH = resolve(import.meta.dirname, '../../apps/api/wrangler.toml'); -const TAIL_WORKER_WRANGLER_PATH = resolve(import.meta.dirname, '../../apps/tail-worker/wrangler.toml'); +const TAIL_WORKER_WRANGLER_PATH = resolve( + import.meta.dirname, + '../../apps/tail-worker/wrangler.toml' +); interface Binding { name?: string; @@ -69,8 +71,8 @@ function main(): void { const envNames = Object.keys(apiConfig.env).join(', '); errors.push( `apps/api/wrangler.toml contains [env.*] sections (${envNames}). ` + - `These are generated at deploy time by sync-wrangler-config.ts. ` + - `Remove them from the checked-in file.` + `These are generated at deploy time by sync-wrangler-config.ts. ` + + `Remove them from the checked-in file.` ); } @@ -81,7 +83,7 @@ function main(): void { const envNames = Object.keys(tailConfig.env).join(', '); errors.push( `apps/tail-worker/wrangler.toml contains [env.*] sections (${envNames}). ` + - `These are generated at deploy time. Remove them from the checked-in file.` + `These are generated at deploy time. Remove them from the checked-in file.` ); } @@ -90,11 +92,15 @@ function main(): void { // ======================================== if (!apiConfig.durable_objects?.bindings?.length) { - errors.push('apps/api/wrangler.toml: top-level missing durable_objects.bindings (sync script copies these to env sections)'); + errors.push( + 'apps/api/wrangler.toml: top-level missing durable_objects.bindings (sync script copies these to env sections)' + ); } if (!apiConfig.ai?.binding) { - errors.push('apps/api/wrangler.toml: top-level missing [ai] binding (sync script copies this to env sections)'); + errors.push( + 'apps/api/wrangler.toml: top-level missing [ai] binding (sync script copies this to env sections)' + ); } if (!apiConfig.d1_databases?.length) { @@ -110,7 +116,9 @@ function main(): void { } if (!apiConfig.migrations?.length) { - errors.push('apps/api/wrangler.toml: top-level missing [[migrations]] (sync script copies these to env sections)'); + errors.push( + 'apps/api/wrangler.toml: top-level missing [[migrations]] (sync script resolves these into env sections)' + ); } // ======================================== @@ -129,7 +137,9 @@ function main(): void { console.log('Wrangler config check passed.'); console.log(` No [env.*] sections in checked-in files.`); - console.log(` Top-level: ${doCount} DOs, ${d1Count} D1, ${kvCount} KV, ${r2Count} R2, AI, ${migrationCount} migrations`); + console.log( + ` Top-level: ${doCount} DOs, ${d1Count} D1, ${kvCount} KV, ${r2Count} R2, AI, ${migrationCount} migrations` + ); } main(); diff --git a/scripts/quality/do-migration-compatibility.test.ts b/scripts/quality/do-migration-compatibility.test.ts new file mode 100644 index 000000000..30cfddea2 --- /dev/null +++ b/scripts/quality/do-migration-compatibility.test.ts @@ -0,0 +1,346 @@ +import { readFileSync } from 'node:fs'; +import { resolve } from 'node:path'; + +import * as TOML from '@iarna/toml'; +import { afterEach, describe, expect, it, vi } from 'vitest'; + +import { + generateApiWorkerEnv, + getDeployedWorkerMigrationTag, + resolveDurableObjectMigrations, +} from '../deploy/sync-wrangler-config.js'; +import type { MigrationEntry, PulumiOutputs, WranglerToml } from '../deploy/types.js'; + +const WRANGLER_TOML_PATH = resolve(import.meta.dirname, '../../apps/api/wrangler.toml'); + +const outputs: PulumiOutputs = { + d1DatabaseId: 'd1-id', + d1DatabaseName: 'prefix-prod', + observabilityD1DatabaseId: 'obs-d1-id', + observabilityD1DatabaseName: 'prefix-observability-prod', + kvId: 'kv-id', + kvName: 'prefix-prod-sessions', + r2Name: 'prefix-prod-assets', + sessionSnapshotTtlDays: 30, + dnsIds: { + api: 'api-dns-id', + app: 'app-dns-id', + wildcard: 'wildcard-dns-id', + }, + hostnames: { + api: 'api.example.com', + app: 'app.example.com', + }, + stackSummary: { + stack: 'prod', + baseDomain: 'example.com', + resources: { + d1: 'prefix-prod', + kv: 'prefix-prod-sessions', + r2: 'prefix-prod-assets', + }, + }, + cloudflareAccountId: 'account-id', + pagesName: 'prefix-web-prod', +}; + +function loadCheckedInMigrations(): MigrationEntry[] { + const config = TOML.parse(readFileSync(WRANGLER_TOML_PATH, 'utf8')) as WranglerToml; + return config.migrations as MigrationEntry[]; +} + +afterEach(() => { + vi.unstubAllEnvs(); + vi.unstubAllGlobals(); +}); + +describe('resolveDurableObjectMigrations', () => { + it('generates an all-SQLite bootstrap from the complete checked-in history for a clean install', () => { + const history = loadCheckedInMigrations(); + const legacyCreateCount = history.filter((migration) => migration.new_classes).length; + + const resolved = resolveDurableObjectMigrations(history, null); + + expect(history).toHaveLength(17); + expect(legacyCreateCount).toBe(7); + expect(resolved).toHaveLength(history.length); + expect(resolved.every((migration) => migration.new_classes === undefined)).toBe(true); + expect(resolved.flatMap((migration) => migration.new_sqlite_classes ?? [])).toHaveLength(17); + expect(loadCheckedInMigrations()).toEqual(history); + }); + + it('keeps the checked-in migration history append-only with sequential tags', () => { + // The resolver (and Wrangler itself) treat the deployed tag's array position + // as the applied/pending boundary, so the checked-in history must only ever + // be appended to. Sequential v1..vN tags make a mid-array insertion or + // reorder (e.g. a bad merge) fail here instead of silently skipping or + // misclassifying migrations. See .claude/rules/07-env-and-urls.md. + const history = loadCheckedInMigrations(); + + expect(history.length).toBeGreaterThan(0); + history.forEach((migration, index) => { + expect(migration.tag).toBe(`v${index + 1}`); + }); + }); + + it('preserves the complete migration history for an existing legacy deployment at v17', () => { + const history = loadCheckedInMigrations(); + vi.stubEnv('RESOURCE_PREFIX', 's123abc'); + + const resolved = resolveDurableObjectMigrations(history, 'v17'); + const envConfig = generateApiWorkerEnv( + { migrations: history }, + outputs, + 'prod', + false, + false, + 'v17' + ); + + expect(resolved).toEqual(history); + resolved.forEach((migration, index) => expect(migration).toBe(history[index])); + expect(envConfig.migrations).toEqual(history); + }); + + it('preserves an applied prefix and converts only pending legacy creates', () => { + const history: MigrationEntry[] = [ + { tag: 'v1', new_sqlite_classes: ['SqliteFirst'] }, + { tag: 'v2', new_classes: ['AppliedLegacy'] }, + { tag: 'v3', new_classes: ['PendingLegacy'] }, + { + tag: 'v4', + renamed_classes: [{ from: 'PendingLegacy', to: 'RenamedPendingLegacy' }], + }, + { tag: 'v5', deleted_classes: ['DeletedClass'] }, + ]; + + const resolved = resolveDurableObjectMigrations(history, 'v2'); + + expect(resolved).toEqual([ + history[0], + history[1], + { tag: 'v3', new_sqlite_classes: ['PendingLegacy'] }, + history[3], + history[4], + ]); + expect(resolved[0]).toBe(history[0]); + expect(resolved[1]).toBe(history[1]); + expect(resolved[3]).toBe(history[3]); + expect(resolved[4]).toBe(history[4]); + }); + + it('combines pending legacy and SQLite creates without mutating the source entry', () => { + const history: MigrationEntry[] = [ + { + tag: 'v1', + new_classes: ['LegacyClass'], + new_sqlite_classes: ['SqliteClass'], + }, + ]; + + expect(resolveDurableObjectMigrations(history, null)).toEqual([ + { tag: 'v1', new_sqlite_classes: ['SqliteClass', 'LegacyClass'] }, + ]); + expect(history[0]).toEqual({ + tag: 'v1', + new_classes: ['LegacyClass'], + new_sqlite_classes: ['SqliteClass'], + }); + }); + + it('fails closed when the deployed tag is not in the checked-in history', () => { + expect(() => + resolveDurableObjectMigrations([{ tag: 'v1', new_classes: ['LegacyClass'] }], 'v999') + ).toThrow('Deployed Durable Object migration tag "v999" is not present'); + }); + + it('fails closed when the checked-in history contains duplicate tags', () => { + expect(() => + resolveDurableObjectMigrations( + [{ tag: 'v1', new_classes: ['LegacyClass'] }, { tag: 'v1' }], + null + ) + ).toThrow('Durable Object migration tag "v1" is duplicated'); + }); + + it('feeds the resolved clean-install history into the generated environment', () => { + vi.stubEnv('RESOURCE_PREFIX', 's123abc'); + const topLevel = TOML.parse(readFileSync(WRANGLER_TOML_PATH, 'utf8')) as WranglerToml; + + const envConfig = generateApiWorkerEnv(topLevel, outputs, 'prod', false, false, null); + const generatedConfig = TOML.parse( + TOML.stringify({ + env: { production: envConfig }, + } as TOML.JsonMap) + ) as WranglerToml; + const generatedMigrations = generatedConfig.env?.production?.migrations; + + expect(generatedMigrations).toEqual( + resolveDurableObjectMigrations(topLevel.migrations as MigrationEntry[], null) + ); + expect(generatedMigrations?.every((migration) => !migration.new_classes)).toBe(true); + expect(topLevel.migrations).toEqual(loadCheckedInMigrations()); + }); +}); + +describe('getDeployedWorkerMigrationTag', () => { + it('returns null only when the exact Worker lookup confirms a clean target', async () => { + vi.stubEnv('CF_API_TOKEN', 'token'); + const fetchMock = vi + .fn() + .mockResolvedValueOnce(new Response('{"success":true,"result":[]}', { status: 200 })) + .mockResolvedValueOnce(new Response('{}', { status: 404 })); + vi.stubGlobal('fetch', fetchMock); + + await expect(getDeployedWorkerMigrationTag('account-id', 'api-worker')).resolves.toBeNull(); + expect(fetchMock).toHaveBeenCalledTimes(2); + expect(fetchMock).toHaveBeenNthCalledWith( + 1, + 'https://api.cloudflare.com/client/v4/accounts/account-id/workers/scripts?per_page=1000', + { headers: { Authorization: 'Bearer token' } } + ); + expect(fetchMock).toHaveBeenNthCalledWith( + 2, + 'https://api.cloudflare.com/client/v4/accounts/account-id/workers/scripts/api-worker/settings', + { headers: { Authorization: 'Bearer token' } } + ); + }); + + it('reads the migration tag from the Workers scripts list for an existing Worker', async () => { + vi.stubEnv('CF_API_TOKEN', 'token'); + const fetchMock = vi.fn().mockResolvedValue( + new Response('{"success":true,"result":[{"id":"api-worker","migration_tag":"v17"}]}', { + status: 200, + }) + ); + vi.stubGlobal('fetch', fetchMock); + + await expect(getDeployedWorkerMigrationTag('account-id', 'api-worker')).resolves.toBe('v17'); + expect(fetchMock).toHaveBeenNthCalledWith( + 1, + 'https://api.cloudflare.com/client/v4/accounts/account-id/workers/scripts?per_page=1000', + { headers: { Authorization: 'Bearer token' } } + ); + }); + + it('fails closed before exact lookup when the Workers listing is unreadable', async () => { + vi.stubEnv('CF_API_TOKEN', 'token'); + vi.stubEnv('DO_MIGRATION_STATE_PROBE_ATTEMPTS', '1'); + const fetchMock = vi + .fn() + .mockResolvedValue(new Response('sensitive-control-plane-detail', { status: 403 })); + vi.stubGlobal('fetch', fetchMock); + + await expect(getDeployedWorkerMigrationTag('account-id', 'api-worker')).rejects.toThrow( + /^Failed to list Workers while reading migration state for "api-worker" \(HTTP 403\)$/ + ); + expect(fetchMock).toHaveBeenCalledTimes(1); + }); + + it('fails closed when exact Worker state cannot be read without leaking response content', async () => { + vi.stubEnv('CF_API_TOKEN', 'token'); + vi.stubEnv('DO_MIGRATION_STATE_PROBE_ATTEMPTS', '1'); + vi.stubGlobal( + 'fetch', + vi + .fn() + .mockResolvedValueOnce(new Response('{"success":true,"result":[]}', { status: 200 })) + .mockResolvedValueOnce(new Response('sensitive-control-plane-detail', { status: 403 })) + ); + + await expect(getDeployedWorkerMigrationTag('account-id', 'api-worker')).rejects.toThrow( + /^Failed to read Durable Object migration state for Worker "api-worker" \(HTTP 403\)$/ + ); + }); + + it('fails closed when an existing Worker is omitted from the scripts listing', async () => { + vi.stubEnv('CF_API_TOKEN', 'token'); + vi.stubEnv('DO_MIGRATION_STATE_PROBE_ATTEMPTS', '1'); + vi.stubGlobal( + 'fetch', + vi + .fn() + .mockResolvedValueOnce(new Response('{"success":true,"result":[]}', { status: 200 })) + .mockResolvedValueOnce(new Response('{"success":true,"result":{}}', { status: 200 })) + ); + + await expect(getDeployedWorkerMigrationTag('account-id', 'api-worker')).rejects.toThrow( + 'exists but is absent from the Workers scripts listing' + ); + }); + + it('fails closed when an existing Worker has no migration tag', async () => { + vi.stubEnv('CF_API_TOKEN', 'token'); + vi.stubEnv('DO_MIGRATION_STATE_PROBE_ATTEMPTS', '1'); + vi.stubGlobal( + 'fetch', + vi + .fn() + .mockResolvedValue( + new Response('{"success":true,"result":[{"id":"api-worker"}]}', { status: 200 }) + ) + ); + + await expect(getDeployedWorkerMigrationTag('account-id', 'api-worker')).rejects.toThrow( + 'has no migration_tag' + ); + }); + + it('retries a transient ambiguous probe and succeeds on a later attempt', async () => { + vi.stubEnv('CF_API_TOKEN', 'token'); + vi.stubEnv('DO_MIGRATION_STATE_PROBE_RETRY_DELAY_MS', '0'); + const fetchMock = vi + .fn() + // Attempt 1: listing lags the just-created Worker; settings already sees it. + .mockResolvedValueOnce(new Response('{"success":true,"result":[]}', { status: 200 })) + .mockResolvedValueOnce(new Response('{}', { status: 200 })) + // Attempt 2: listing has caught up. + .mockResolvedValueOnce( + new Response('{"success":true,"result":[{"id":"api-worker","migration_tag":"v17"}]}', { + status: 200, + }) + ); + vi.stubGlobal('fetch', fetchMock); + + await expect(getDeployedWorkerMigrationTag('account-id', 'api-worker')).resolves.toBe('v17'); + expect(fetchMock).toHaveBeenCalledTimes(3); + }); + + it('exhausts the bounded attempts and rethrows the final probe error', async () => { + vi.stubEnv('CF_API_TOKEN', 'token'); + vi.stubEnv('DO_MIGRATION_STATE_PROBE_ATTEMPTS', '2'); + vi.stubEnv('DO_MIGRATION_STATE_PROBE_RETRY_DELAY_MS', '0'); + const fetchMock = vi.fn().mockResolvedValue(new Response('unavailable', { status: 500 })); + vi.stubGlobal('fetch', fetchMock); + + await expect(getDeployedWorkerMigrationTag('account-id', 'api-worker')).rejects.toThrow( + /^Failed to list Workers while reading migration state for "api-worker" \(HTTP 500\)$/ + ); + expect(fetchMock).toHaveBeenCalledTimes(2); + }); + + it('falls back to the default attempt bound when the env override is invalid', async () => { + vi.stubEnv('CF_API_TOKEN', 'token'); + vi.stubEnv('DO_MIGRATION_STATE_PROBE_ATTEMPTS', 'not-a-number'); + vi.stubEnv('DO_MIGRATION_STATE_PROBE_RETRY_DELAY_MS', '0'); + const fetchMock = vi.fn().mockResolvedValue(new Response('forbidden', { status: 403 })); + vi.stubGlobal('fetch', fetchMock); + + await expect(getDeployedWorkerMigrationTag('account-id', 'api-worker')).rejects.toThrow( + 'HTTP 403' + ); + expect(fetchMock).toHaveBeenCalledTimes(3); + }); + + it('requires the deployment token before probing migration state', async () => { + vi.stubEnv('CF_API_TOKEN', ''); + vi.stubEnv('CLOUDFLARE_API_TOKEN', ''); + const fetchMock = vi.fn().mockRejectedValue(new Error('network must not be reached')); + vi.stubGlobal('fetch', fetchMock); + + await expect(getDeployedWorkerMigrationTag('account-id', 'api-worker')).rejects.toThrow( + 'CF_API_TOKEN or CLOUDFLARE_API_TOKEN is required' + ); + expect(fetchMock).not.toHaveBeenCalled(); + }); +}); diff --git a/scripts/quality/sync-wrangler-config.test.ts b/scripts/quality/sync-wrangler-config.test.ts index e1e88d7bb..c6fc345ad 100644 --- a/scripts/quality/sync-wrangler-config.test.ts +++ b/scripts/quality/sync-wrangler-config.test.ts @@ -52,7 +52,7 @@ describe('sync wrangler config', () => { analytics_engine_datasets: [{ binding: 'ANALYTICS', dataset: 'legacy_analytics' }], }; - const envConfig = generateApiWorkerEnv(topLevel, outputs, 'prod', false, false); + const envConfig = generateApiWorkerEnv(topLevel, outputs, 'prod', false, false, null); expect(envConfig.vars).toMatchObject({ ANALYTICS_DATASET: 's123abc_analytics', @@ -67,7 +67,7 @@ describe('sync wrangler config', () => { }); it('fails instead of falling back to sam when deployment identity is missing', () => { - expect(() => generateApiWorkerEnv({}, outputs, 'prod', false, false)).toThrow( + expect(() => generateApiWorkerEnv({}, outputs, 'prod', false, false, null)).toThrow( 'RESOURCE_PREFIX or BASE_DOMAIN is required' ); }); @@ -75,7 +75,7 @@ describe('sync wrangler config', () => { it('derives deployment identity from BASE_DOMAIN when RESOURCE_PREFIX is not explicit', () => { vi.stubEnv('BASE_DOMAIN', 'example.com'); - const envConfig = generateApiWorkerEnv({}, outputs, 'prod', false, false); + const envConfig = generateApiWorkerEnv({}, outputs, 'prod', false, false, null); expect(envConfig.name).toBe('sa379a6-api-prod'); expect(envConfig.vars).toMatchObject({ @@ -88,7 +88,7 @@ describe('sync wrangler config', () => { it('enables Cloudflare Container runtime by default and allows explicit opt-out', () => { vi.stubEnv('RESOURCE_PREFIX', 's123abc'); - expect(generateApiWorkerEnv({}, outputs, 'prod', false, false).vars).toMatchObject({ + expect(generateApiWorkerEnv({}, outputs, 'prod', false, false, null).vars).toMatchObject({ CF_CONTAINER_ENABLED: 'true', }); @@ -98,14 +98,15 @@ describe('sync wrangler config', () => { outputs, 'prod', false, - false + false, + null ).vars ).toMatchObject({ CF_CONTAINER_ENABLED: 'false', }); vi.stubEnv('CF_CONTAINER_ENABLED', 'false'); - expect(generateApiWorkerEnv({}, outputs, 'prod', false, false).vars).toMatchObject({ + expect(generateApiWorkerEnv({}, outputs, 'prod', false, false, null).vars).toMatchObject({ CF_CONTAINER_ENABLED: 'false', }); }); @@ -113,13 +114,13 @@ describe('sync wrangler config', () => { it('passes cf-container clone/create tunables through from process env when set', () => { vi.stubEnv('RESOURCE_PREFIX', 's123abc'); - const unset = generateApiWorkerEnv({}, outputs, 'prod', false, false).vars; + const unset = generateApiWorkerEnv({}, outputs, 'prod', false, false, null).vars; expect(unset).not.toHaveProperty('CF_CONTAINER_CREATE_WORKSPACE_TIMEOUT_MS'); expect(unset).not.toHaveProperty('CF_CONTAINER_CLONE_FILTER'); vi.stubEnv('CF_CONTAINER_CREATE_WORKSPACE_TIMEOUT_MS', '180000'); vi.stubEnv('CF_CONTAINER_CLONE_FILTER', 'off'); - expect(generateApiWorkerEnv({}, outputs, 'prod', false, false).vars).toMatchObject({ + expect(generateApiWorkerEnv({}, outputs, 'prod', false, false, null).vars).toMatchObject({ CF_CONTAINER_CREATE_WORKSPACE_TIMEOUT_MS: '180000', CF_CONTAINER_CLONE_FILTER: 'off', }); @@ -133,7 +134,7 @@ describe('sync wrangler config', () => { artifacts: [{ binding: 'ARTIFACTS', namespace: 'default' }], }; - const envConfig = generateApiWorkerEnv(topLevel, outputs, 'prod', false, false); + const envConfig = generateApiWorkerEnv(topLevel, outputs, 'prod', false, false, null); expect(envConfig.artifacts).toBeUndefined(); expect(envConfig.vars).toMatchObject({ ARTIFACTS_ENABLED: 'false' }); @@ -143,7 +144,7 @@ describe('sync wrangler config', () => { vi.stubEnv('RESOURCE_PREFIX', 's123abc'); const artifacts = [{ binding: 'ARTIFACTS', namespace: 'default' }]; - const envConfig = generateApiWorkerEnv({ artifacts }, outputs, 'prod', false, true); + const envConfig = generateApiWorkerEnv({ artifacts }, outputs, 'prod', false, true, null); expect(envConfig.artifacts).toEqual(artifacts); expect(envConfig.vars).toMatchObject({ ARTIFACTS_ENABLED: 'true' }); @@ -152,7 +153,7 @@ describe('sync wrangler config', () => { it('generates a plaintext setup token var without requiring an input secret', () => { vi.stubEnv('RESOURCE_PREFIX', 's123abc'); - const envConfig = generateApiWorkerEnv({}, outputs, 'prod', false, false); + const envConfig = generateApiWorkerEnv({}, outputs, 'prod', false, false, null); expect(envConfig.vars?.SETUP_TOKEN).toEqual(expect.any(String)); expect(String(envConfig.vars?.SETUP_TOKEN).length).toBeGreaterThan(20); @@ -161,12 +162,12 @@ describe('sync wrangler config', () => { it('includes SETUP_FORCE only when explicitly requested', () => { vi.stubEnv('RESOURCE_PREFIX', 's123abc'); - expect(generateApiWorkerEnv({}, outputs, 'prod', false, false).vars).not.toHaveProperty( + expect(generateApiWorkerEnv({}, outputs, 'prod', false, false, null).vars).not.toHaveProperty( 'SETUP_FORCE' ); vi.stubEnv('SETUP_FORCE', 'true'); - expect(generateApiWorkerEnv({}, outputs, 'prod', false, false).vars).toMatchObject({ + expect(generateApiWorkerEnv({}, outputs, 'prod', false, false, null).vars).toMatchObject({ SETUP_FORCE: 'true', }); }); @@ -174,7 +175,7 @@ describe('sync wrangler config', () => { it('fails when Artifacts is enabled without a top-level binding declaration', () => { vi.stubEnv('RESOURCE_PREFIX', 's123abc'); - expect(() => generateApiWorkerEnv({}, outputs, 'prod', false, true)).toThrow( + expect(() => generateApiWorkerEnv({}, outputs, 'prod', false, true, null)).toThrow( 'Artifacts is enabled but no top-level [[artifacts]] binding exists in wrangler.toml' ); }); @@ -195,9 +196,15 @@ describe('sync wrangler config', () => { }); it('requires a Cloudflare API token before checking tail worker status', async () => { + vi.stubEnv('CF_API_TOKEN', ''); + vi.stubEnv('CLOUDFLARE_API_TOKEN', ''); + const fetchMock = vi.fn().mockRejectedValue(new Error('network must not be reached')); + vi.stubGlobal('fetch', fetchMock); + await expect(checkTailWorkerExists('account-id', 'tail-worker')).rejects.toThrow( 'CF_API_TOKEN or CLOUDFLARE_API_TOKEN is required' ); + expect(fetchMock).not.toHaveBeenCalled(); }); }); diff --git a/tasks/backlog/2026-07-21-cloudflare-do-clean-install-migrations.md b/tasks/archive/2026-07-21-cloudflare-do-clean-install-migrations.md similarity index 61% rename from tasks/backlog/2026-07-21-cloudflare-do-clean-install-migrations.md rename to tasks/archive/2026-07-21-cloudflare-do-clean-install-migrations.md index 758019b53..5a5cab03b 100644 --- a/tasks/backlog/2026-07-21-cloudflare-do-clean-install-migrations.md +++ b/tasks/archive/2026-07-21-cloudflare-do-clean-install-migrations.md @@ -39,25 +39,25 @@ SQLite. Clean installations must create every namespace as SQLite-backed. `apps/api/wrangler.toml` contains 17 ordered create migrations and no rename/delete/transfer migrations on `main`: -| Tag | Class | Historical backend | -| --- | --- | --- | -| v1 | `ProjectData` | SQLite | -| v2 | `NodeLifecycle` | legacy KV | -| v3 | `AdminLogs` | legacy KV | -| v4 | `TaskRunner` | legacy KV | -| v5 | `NotificationService` | SQLite | -| v6 | `CodexRefreshLock` | legacy KV | -| v7 | `TrialCounter` | SQLite | -| v8 | `TrialEventBus` | legacy KV | -| v9 | `TrialOrchestrator` | SQLite | -| v10 | `ProjectOrchestrator` | SQLite | -| v11 | `SamSession` | SQLite | -| v12 | `ProjectAgent` | SQLite | -| v13 | `SandboxDO` | SQLite | -| v14 | `AiTokenBudgetCounter` | SQLite | -| v15 | `GitHubUserAccessTokenLock` | legacy KV | -| v16 | `VmAgentContainer` | SQLite | -| v17 | `GitLabUserAccessTokenLock` | legacy KV | +| Tag | Class | Historical backend | +| --- | --------------------------- | ------------------ | +| v1 | `ProjectData` | SQLite | +| v2 | `NodeLifecycle` | legacy KV | +| v3 | `AdminLogs` | legacy KV | +| v4 | `TaskRunner` | legacy KV | +| v5 | `NotificationService` | SQLite | +| v6 | `CodexRefreshLock` | legacy KV | +| v7 | `TrialCounter` | SQLite | +| v8 | `TrialEventBus` | legacy KV | +| v9 | `TrialOrchestrator` | SQLite | +| v10 | `ProjectOrchestrator` | SQLite | +| v11 | `SamSession` | SQLite | +| v12 | `ProjectAgent` | SQLite | +| v13 | `SandboxDO` | SQLite | +| v14 | `AiTokenBudgetCounter` | SQLite | +| v15 | `GitHubUserAccessTokenLock` | legacy KV | +| v16 | `VmAgentContainer` | SQLite | +| v17 | `GitLabUserAccessTokenLock` | legacy KV | The temporary feature-branch version of `v9` used `new_classes`, but neither that commit nor its follow-up rewrite is an ancestor of `main`; the merged @@ -118,27 +118,27 @@ operator hand-edit instructions. ## Implementation checklist -- [ ] Add typed Cloudflare Worker migration-state detection to the Wrangler +- [x] Add typed Cloudflare Worker migration-state detection to the Wrangler sync generator with bounded, redacted diagnostics. -- [ ] Add a pure migration resolver that preserves the applied prefix and +- [x] Add a pure migration resolver that preserves the applied prefix and converts only pending legacy namespace creates to SQLite. -- [ ] Feed the resolved migration list into every generated API environment. -- [ ] Keep local Miniflare bindings and the checked-in historical migration +- [x] Feed the resolved migration list into every generated API environment. +- [x] Keep local Miniflare bindings and the checked-in historical migration chain unchanged. -- [ ] Add clean-install, fully-upgraded legacy, partial-upgrade, unknown-tag, +- [x] Add clean-install, fully-upgraded legacy, partial-upgrade, unknown-tag, missing-Worker, and Cloudflare API failure tests. -- [ ] Add a parsed generated-config assertion proving the deployment env uses +- [x] Add a parsed generated-config assertion proving the deployment env uses the selected list rather than the raw top-level list. -- [ ] Update the Wrangler deployment rule/process guard so future DO migrations +- [x] Update the Wrangler deployment rule/process guard so future DO migrations preserve applied history and use SQLite for new namespaces. -- [ ] Update public deployment documentation only where runtime behavior needs +- [x] Update public deployment documentation only where runtime behavior needs explanation; avoid making operators choose migration history manually. -- [ ] Run focused generator/workflow tests, Wrangler binding checks, lint, +- [x] Run focused generator/workflow tests, Wrangler binding checks, lint, typecheck, tests, build, and migration safety checks proportionate to the change. -- [ ] Run Cloudflare specialist, task completion, test, constitution, and +- [x] Run Cloudflare specialist, task completion, test, constitution, and documentation reviews; address all correctness findings. -- [ ] Deploy through the staging workflow, prove v17 legacy namespaces remain +- [x] Deploy through the staging workflow, prove v17 legacy namespaces remain unchanged, and record the clean-install proof available without mutating production. - [ ] Open a PR that closes #1614, documents evidence/risks/blockers, waits for @@ -146,23 +146,76 @@ operator hand-edit instructions. ## Acceptance criteria -- [ ] A generated config for a confirmed clean target contains no +- [x] A generated config for a confirmed clean target contains no `new_classes` directives and creates all 17 classes with SQLite. -- [ ] A target already at v17 receives the unchanged historical migration list; +- [x] A target already at v17 receives the unchanged historical migration list; no namespace is recreated, deleted, renamed, converted, or replayed. -- [ ] A partially upgraded target preserves applied entries and converts only +- [x] A partially upgraded target preserves applied entries and converts only future legacy creates to SQLite. -- [ ] Unknown or unreadable deployed migration state blocks deployment with an +- [x] Unknown or unreadable deployed migration state blocks deployment with an actionable error instead of assuming a clean account. -- [ ] Canonical staging, canonical production, and self-host production all use +- [x] Canonical staging, canonical production, and self-host production all use the same deterministic generator behavior. -- [ ] Local Miniflare/Worker tests remain compatible. -- [ ] Automated tests discriminate clean-install and existing-upgrade behavior +- [x] Local Miniflare/Worker tests remain compatible. +- [x] Automated tests discriminate clean-install and existing-upgrade behavior and would fail against the current verbatim-copy generator. -- [ ] Official documentation and read-only deployed state support the migration +- [x] Official documentation and read-only deployed state support the migration reasoning recorded in the PR. - [ ] PR CI is green and the PR is left open/unmerged. +## Validation evidence + +- TDD discrimination: the new compatibility suite failed all 12 initial cases + against the verbatim-copy generator, then passed after implementation. +- Focused compatibility/generator tests: 37 tests passed. +- Deployment/config gates passed: deployment-script TypeScript compilation, + Wrangler binding parity, D1 migration safety, Durable Object migration + safety, migration ordering, AST checks, file-size checks, source-contract + checks, agent-install manifest checks, and specialist-review harness tests + (127 tests). +- Full repository gates passed: `pnpm lint`, `pnpm typecheck`, `pnpm test`, and + `pnpm build`. The Worker suite exercised Miniflare and passed 6,208 API tests; + the web suite passed 2,740 tests. +- Targeted formatting and `git diff --check` passed. The repository-wide + formatting check still reports unrelated pre-existing formatting debt, so + changed files were checked directly. +- The final read-only state probe returned `v17` for both `sam-api-staging` and + `sam-api-prod`; namespace backend flags were inspected separately and match + the immutable checked-in history. + +## Staging and clean-install proof + +- Staging workflow run `29830545589` deployed commit `44548e8cb` successfully. + Both API Worker deploys and the workflow health checks passed. +- The generator logged `sam-api-staging` at migration state `v17`; the live + Playwright smoke suite passed all 12 tests. +- Read-only pre/post namespace snapshots were identical: 17 classes remained, + with the same seven classes KV-backed and the other ten SQLite-backed. The + post-deploy Worker migration tag remained `v17`. +- Direct generated-config proof produced 17 tags, zero `new_classes`, and 17 + SQLite class creates for clean state; v17 output was structurally identical. +- A live clean-account deployment was unavailable: the only accessible + Cloudflare accounts are canonical staging and production, and both already + contain legacy namespaces, so neither can exercise the clean-account condition. +- Production was inspected read-only only; no production resource or namespace + was created, changed, or deleted. + +## Specialist review evidence + +- Task completion: PASS; Checks A-F found no planned-vs-actual, acceptance, + UI-path, multi-resource, or vertical-slice gap. +- Cloudflare/infra: PASS; applied tags remain immutable, clean creates are all + SQLite, existing v17 output is unchanged, and both deploy passes use the probe. +- Tests: ADDRESSED; added explicit duplicate-tag and unreadable-list fail-closed + cases identified during review, bringing the focused total to 37. +- Constitution: PASS; the new URL is an external Cloudflare API contract and + the page size is a bounded protocol request, not business configuration. +- Documentation: PASS; the deployment rule and public self-hosting behavior + match the generator and introduce no new operator-managed configuration. +- All attempted specialist subagents were interrupted by the execution + environment, so the repository skill checklists were executed locally and + this limitation will be disclosed in the PR. + ## Post-mortem ### What broke