perf(db): split embeddings + in-memory similarity cache + drop redundant co_changes index

[andreinknv]

Summary

Three measurable wins on the LLM-tier data layer, validated by spike before implementing each:

	Change	Win	Spike file
F2	Drop idx_co_changes_a	Covered by (file_a, file_b) PK	(analogous to PR #1)
G	Split embeddings into symbol_embeddings table	3.22× faster summary-only scans	scripts/spikes/spike-embedding-split.mjs
H	In-memory EmbeddingCache for similarity search	4.4× faster top-K cosine	scripts/spikes/spike-embedding-split.mjs

Note: This PR sits on top of colbymchenry#111 (LLM symbol summaries). The spike script reproducer covers G and H; F2 follows the same left-prefix-scan-covers-narrow-index pattern as PR #1 (perf/drop-redundant-edge-indexes).

Empirical validation

Run yourself: node scripts/spikes/spike-embedding-split.mjs. Output on a 50K-summary / 768d-embedding synthetic DB:

--- Spike G: storage layout (inline vs split) ---

  inline DB: 196.5 MB
  split  DB: 204.0 MB

  Test: scan summaries by role (common path)
  inline: 46ms avg over 50 queries
  split : 14ms avg over 50 queries
  Δ summary-only: split is 3.22× faster

  Test: scan summaries WITH embedding (rare path)
  inline (single table)   : 71ms avg over 50 queries
  split  (join required)  : 80ms avg over 50 queries
  Δ summary+embedding: 1.12× cost penalty for split

--- Spike H: in-memory embedding cache ---

  cold (per-query SQLite fetch + decode): 104ms avg over 20 queries
  warm (in-memory Float32Array matrix)  : 24ms avg over 20 queries
  Δ similarity search: 4.4× speedup with in-memory cache

The 1.12× cost on summary+embedding scans is dwarfed by the 3.22× win on summary-only scans, which dominate by ~50× in real usage (FTS-anchor lookups, role filters, freshness checks all read summaries-without-embeddings).

What changes

F2 — Migration 015: drop idx_co_changes_a

• co_changes has PRIMARY KEY (file_a, file_b), which automatically creates a B-tree leading on file_a. SQLite covers WHERE file_a = ? lookups via that PK index — the standalone idx_co_changes_a was redundant.
• idx_co_changes_b (on file_b alone) is kept because the PK leads with file_a, so it cannot serve WHERE file_b = ? lookups.
• Fresh-DB schema (src/db/schema.sql) updated to skip idx_co_changes_a and dedupe a pre-existing duplicate of idx_co_changes_b.

G — Migration 016: split embeddings into symbol_embeddings

CREATE TABLE symbol_embeddings (
    node_id TEXT PRIMARY KEY,
    embedding BLOB NOT NULL,
    embedding_model TEXT NOT NULL,
    FOREIGN KEY (node_id) REFERENCES symbol_summaries(node_id) ON DELETE CASCADE
);
INSERT OR IGNORE INTO symbol_embeddings (node_id, embedding, embedding_model)
  SELECT node_id, embedding, embedding_model
  FROM symbol_summaries
  WHERE embedding IS NOT NULL AND embedding_model IS NOT NULL;
DROP INDEX IF EXISTS idx_summaries_embedding_model;
ALTER TABLE symbol_summaries DROP COLUMN embedding;
ALTER TABLE symbol_summaries DROP COLUMN embedding_model;

• Requires SQLite 3.35+ for ALTER TABLE DROP COLUMN. Both better-sqlite3 and node-sqlite3-wasm ship with newer versions, so this is safe.
• queries.ts methods (getEmbeddableSummaries, getAllEmbeddings, upsertSymbolEmbedding) updated to use the new table.
• clear() and clearCoChanges() extended to wipe both tables (the FK cascade would handle it, but explicit is safer if foreign-key enforcement gets disabled).

H — EmbeddingCache in src/llm/embeddings.ts

export class EmbeddingCache {
  get(fetcher: EmbeddingFetcher, model: string): CachedEmbeddings;
  invalidate(): void;
}

• Decodes every embedding into a flat Float32Array matrix once per (model, generation).
• New topKByCosineMatrix(query, matrix, ids, dim, k) operates directly on the flat layout.
• Owned by the CodeGraph instance.
• Invalidated on:
  • indexAll end (when filesIndexed > 0)
  • sync end (when files added/modified/removed)
  • After embedAllSummaries (when generated > 0)
  • clear() (table was emptied)
• Mismatched-dim rows are skipped on rebuild — produces a packed matrix with no sparse holes.

Independent review

Reviewed by an independent reviewer agent (read-only, fresh context). Surfaced 3 issues, all addressed in the same diff:

1. CodeGraph.clear() was missing a cache invalidation — fixed: added this.embeddingCache.invalidate().
2. Duplicate idx_co_changes_b in schema.sql (pre-existing, harmless due to IF NOT EXISTS) — fixed: deduped.
3. EmbeddingCache left sparse undefined holes in ids on dim-mismatch — fixed: rewrote to push aligned entries instead of pre-allocating, with a regression test.

Test plan

• npx tsc --noEmit clean
• npx vitest run — 794/794 tests pass
• New tests: __tests__/migrations-015-016.test.ts (upgrade + fresh-DB paths for both migrations)
• New tests in __tests__/embeddings.test.ts:
  • topKByCosineMatrix matches topKByCosine on the same data
  • EmbeddingCache hit/miss/invalidate
  • Cache returns empty result without re-fetching
  • Cache skips dim-mismatched rows
• Spike script (scripts/spikes/spike-embedding-split.mjs) reproduces the headline numbers

Files changed

File	Change
src/db/migrations/015-prune-co-changes-index.ts	New: drop idx_co_changes_a
src/db/migrations/016-split-symbol-embeddings.ts	New: split embeddings into dedicated table
src/db/migrations/index.ts	Register both new migrations
src/db/schema.sql	Fresh-DB schema reflects new layout; dedupe idx_co_changes_b
src/db/queries.ts	getAllEmbeddings, upsertSymbolEmbedding, getEmbeddableSummaries use symbol_embeddings
src/llm/embeddings.ts	EmbeddingCache + topKByCosineMatrix
src/index.ts	Cache integrated into searchHybrid and findSimilar; invalidations wired
__tests__/migrations-015-016.test.ts	New: focused migration tests
__tests__/embeddings.test.ts	New: cache + matrix tests
__tests__/foundation.test.ts	Updated schema-version expectation
__tests__/pr19-improvements.test.ts	Updated schema-version expectation
scripts/spikes/spike-embedding-split.mjs	New: G + H reproducer

🤖 Generated with Claude Code

[andreinknv]

Superseded by upstream PR colbymchenry#123