Skip to content

refactor: file-based migrations — eliminate silent version-collision bugs#18

Open
mschreib28 wants to merge 1 commit into
mainfrom
upstream/refactor/migration-files
Open

refactor: file-based migrations — eliminate silent version-collision bugs#18
mschreib28 wants to merge 1 commit into
mainfrom
upstream/refactor/migration-files

Conversation

@mschreib28
Copy link
Copy Markdown
Owner

Reviewers: for a 5-minute review path that surfaces only what to verify, see the review checklist comment below. The original description is preserved here for full technical detail.\n\n---\n\nrefactor: file-based migrations — eliminate version-collision bug class\n\nToday every PR adding a schema migration claims\nCURRENT_SCHEMA_VERSION = next AND adds an array entry to\nmigrations: Migration[] in src/db/migrations.ts. Two PRs both\nclaiming the same version resolve as: "second PR's v4 silently\nno-ops on existing DBs" — a real silent-data-loss bug class\n(PR colbymchenry#113's reviewer caught one).\n\nAfter this refactor:\n\n Adding a new schema migration:\n 1. Pick the next free 3-digit prefix (git ls-files\n 'src/db/migrations/[0-9]*.ts' shows what's taken).\n 2. Create src/db/migrations/<NNN>-<short-name>.ts exporting a\n MIGRATION: MigrationModule (description + up).\n 3. Add one import line and one entry to\n src/db/migrations/index.ts's REGISTERED_MODULES array.\n\nTwo PRs both creating 004-foo.ts collide on the FILESYSTEM —\nthe maintainer sees it instantly. No more silent skipped\nmigrations.\n\n## What's new\n\n- src/db/migrations/types.tsMigrationModule { description,\n up } and Migration extends MigrationModule { version }.\n- src/db/migrations/002-project-metadata.ts — extracted v2\n body verbatim.\n- src/db/migrations/003-lower-name-index.ts — extracted v3\n body verbatim.\n- src/db/migrations/index.ts — central registry. Static-imports\n each migration, parses the version FROM THE FILENAME (no\n hand-typed version field that can drift), enforces strict\n NNN-kebab-name.ts shape, validates uniqueness/sort at module\n load (throws loudly on collision), exposes ALL_MIGRATIONS and\n CURRENT_SCHEMA_VERSION.\n- src/db/migrations.ts — refactored to a thin runner. Same\n exported surface (CURRENT_SCHEMA_VERSION, getCurrentVersion,\n runMigrations, needsMigration, getPendingMigrations,\n getMigrationHistory, Migration type) — every existing import\n keeps working unchanged.\n- __tests__/migrations-registry.test.ts — 8 invariant tests:\n registry non-empty, versions unique + strictly ascending,\n CURRENT_SCHEMA_VERSION matches max, every file matches the\n strict NNN-kebab-name pattern, no orphan files, no phantom\n registrations.\n\n## Reviewer pass\n\nIndependent reviewer ran once. 3 REQUEST_CHANGES + 1 INFO addressed:\n\n1. Hand-typed version field in REGISTERED_MODULES could drift\n from filename. Fixed: removed the version field; registry\n now parses version from filename via FILENAME_PATTERN regex\n inside validateRegistered.\n2. Filename-pattern test was lenient (allowed 4-digit or 1-digit\n prefixes). Fixed: new "every migration file matches the\n strict NNN-kebab-name.ts pattern" test catches malformed\n filenames as orphan-detection-bypassing offenders.\n3. getPendingMigrations returned readonly Migration[],\n breaking callers that typed the result as Migration[].\n Fixed: returns a fresh mutable array via .slice().\n4. No throw-on-duplicate test for validateRegistered (module\n evaluation timing). Acknowledged; not added.\n\n## Backward compat\n\nEvery existing import works unchanged:\n- import { CURRENT_SCHEMA_VERSION } from './migrations' ✓\n- import { runMigrations } from './migrations' ✓\n- import { needsMigration } from './migrations' ✓\n- import { getMigrationHistory } from './migrations' ✓\n- import { getPendingMigrations } from './migrations' — returns\n mutable Migration[] (preserved)\n- Migration type — re-exported\n\n## Affected open PRs\n\nEvery migration-touching PR (colbymchenry#102 UNIQUE edges, colbymchenry#105 cochange,\n#108 perf db, colbymchenry#111 LLM features, my colbymchenry#112 centrality+churn, colbymchenry#113\nissue-history, colbymchenry#114 config-refs, colbymchenry#115 sql-refs) currently\nclaims migration v4 and conflicts with each other on\nmigrations.ts. After this lands they each become:\n- 1 new file: src/db/migrations/<NNN>-<name>.ts\n- 2 lines in registry.ts (import + array entry)\n\nConflict shape changes from "next free version + array entry +\nCURRENT_SCHEMA_VERSION bump in one file" (4-way conflict) to "1\nnew file" + 2-line registry edit. If two PRs target the same\nNNN, the filesystem collision surfaces immediately — no silent\nskipped migrations.\n\nCo-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com\n


Copied from colbymchenry/codegraph#118

Today every PR adding a schema migration claims
`CURRENT_SCHEMA_VERSION = next` AND adds an array entry to
`migrations: Migration[]` in src/db/migrations.ts. Two PRs both
claiming the same version resolve as: "second PR's v4 silently
no-ops on existing DBs" — a real silent-data-loss bug class
(PR colbymchenry#113's reviewer caught one).

After this refactor:

  Adding a new schema migration:
  1. Pick the next free 3-digit prefix (`git ls-files
     'src/db/migrations/[0-9]*.ts'` shows what's taken).
  2. Create `src/db/migrations/<NNN>-<short-name>.ts` exporting a
     `MIGRATION: MigrationModule` (description + up).
  3. Add one import line and one entry to
     `src/db/migrations/index.ts`'s REGISTERED_MODULES array.

Two PRs both creating `004-foo.ts` collide on the FILESYSTEM —
the maintainer sees it instantly. No more silent skipped
migrations.

## What's new

- `src/db/migrations/types.ts` — `MigrationModule { description,
  up }` and `Migration extends MigrationModule { version }`.
- `src/db/migrations/002-project-metadata.ts` — extracted v2
  body verbatim.
- `src/db/migrations/003-lower-name-index.ts` — extracted v3
  body verbatim.
- `src/db/migrations/index.ts` — central registry. Static-imports
  each migration, parses the version FROM THE FILENAME (no
  hand-typed version field that can drift), enforces strict
  `NNN-kebab-name.ts` shape, validates uniqueness/sort at module
  load (throws loudly on collision), exposes ALL_MIGRATIONS and
  CURRENT_SCHEMA_VERSION.
- `src/db/migrations.ts` — refactored to a thin runner. Same
  exported surface (CURRENT_SCHEMA_VERSION, getCurrentVersion,
  runMigrations, needsMigration, getPendingMigrations,
  getMigrationHistory, Migration type) — every existing import
  keeps working unchanged.
- `__tests__/migrations-registry.test.ts` — 8 invariant tests:
  registry non-empty, versions unique + strictly ascending,
  CURRENT_SCHEMA_VERSION matches max, every file matches the
  strict NNN-kebab-name pattern, no orphan files, no phantom
  registrations.

## Reviewer pass

Independent reviewer ran once. 3 REQUEST_CHANGES + 1 INFO addressed:

1. Hand-typed `version` field in REGISTERED_MODULES could drift
   from filename. **Fixed**: removed the version field; registry
   now parses version from filename via FILENAME_PATTERN regex
   inside validateRegistered.
2. Filename-pattern test was lenient (allowed 4-digit or 1-digit
   prefixes). **Fixed**: new "every migration file matches the
   strict NNN-kebab-name.ts pattern" test catches malformed
   filenames as orphan-detection-bypassing offenders.
3. `getPendingMigrations` returned `readonly Migration[]`,
   breaking callers that typed the result as `Migration[]`.
   **Fixed**: returns a fresh mutable array via `.slice()`.
4. No throw-on-duplicate test for validateRegistered (module
   evaluation timing). Acknowledged; not added.

## Backward compat

Every existing import works unchanged:
- `import { CURRENT_SCHEMA_VERSION } from './migrations'` ✓
- `import { runMigrations } from './migrations'` ✓
- `import { needsMigration } from './migrations'` ✓
- `import { getMigrationHistory } from './migrations'` ✓
- `import { getPendingMigrations } from './migrations'` — returns
   mutable Migration[] (preserved)
- `Migration` type — re-exported

## Affected open PRs

Every migration-touching PR (colbymchenry#102 UNIQUE edges, colbymchenry#105 cochange,
colbymchenry#108 perf db, colbymchenry#111 LLM features, my colbymchenry#112 centrality+churn, colbymchenry#113
issue-history, colbymchenry#114 config-refs, colbymchenry#115 sql-refs) currently
claims migration v4 and conflicts with each other on
`migrations.ts`. After this lands they each become:
- 1 new file: `src/db/migrations/<NNN>-<name>.ts`
- 2 lines in registry.ts (import + array entry)

Conflict shape changes from "next free version + array entry +
CURRENT_SCHEMA_VERSION bump in one file" (4-way conflict) to "1
new file" + 2-line registry edit. If two PRs target the same
NNN, the filesystem collision surfaces immediately — no silent
skipped migrations.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants