Skip to content

feat(plugin:codebase-memory): v0.2.0 — SQLite-direct Cytoscape graph (no daemon, ~30 MB vs 6 GB)#7

Open
SirMinionBot wants to merge 1 commit into
mainfrom
feat/codebase-memory-sqlite-graph
Open

feat(plugin:codebase-memory): v0.2.0 — SQLite-direct Cytoscape graph (no daemon, ~30 MB vs 6 GB)#7
SirMinionBot wants to merge 1 commit into
mainfrom
feat/codebase-memory-sqlite-graph

Conversation

@SirMinionBot

Copy link
Copy Markdown
Owner

Motivation

v0.1.0 (PR #5) embedded the upstream SPA via iframe and required the
codebase-memory-mcp binary running on :9749. The binary allocates
50% of system RAM at startup (mem.init budget_mb = total_ram_mb / 2)
— on this 11 GB / 0-swap host, that OOM-kills within 1 s of --ui=true.

The binary is otherwise excellent: it indexes a repo into a tidy SQLite
DB at ~/.cache/codebase-memory-mcp/<project>.db with FTS5 search, and
its 13-tool MCP API is well-designed. The 6 GB cost is a property of the
in-memory graph engine, not of the data — the DB file itself is 2.8 MB
for our 748-node project.

What this PR does

Replaces the iframe with a native Cytoscape.js renderer that reads
the binary's SQLite index directly
via stdlib sqlite3. The binary
is no longer required at runtime.

Endpoint shape (mounted at /api/plugins/codebase-memory/)

  • GET /health — binary-independent health snapshot
  • GET /projects — list DBs with size-based mega-project filter (50k default)
  • GET /project/{name}/health — per-project stats
  • GET /layout?project=X&max_nodes=N&labels=... — Cytoscape-ready graph
  • GET /search?q=X — FTS5 BM25
  • GET /code?project=X&qualified_name=Y — source snippet read

Performance (codigosinsiesta, 748 nodes, 2.8 MB DB)

Endpoint Latency
/health <10 ms
/projects <50 ms
/layout 19 ms cold, 5 ms warm
/search <20 ms
/code <10 ms

Working-set: ~30 MB (Python + Cytoscape bundle) vs 6 GB for the binary.

Frontend

  • IIFE that mounts a React component under the same codebase-memory slot
  • Cytoscape 3.30 vendored at dist/vendor/cytoscape.min.js (same pattern as knowledge-graph)
  • 2D force-directed cose layout
  • Sidebar with project list + search box (FTS5)
  • Detail panel on node click: file path, complexity metrics, source snippet
  • Status pill shows "sqlite · N db · M project(s)" instead of "UI not running"

Tests

tests/test_plugin_api.py — 9 cases against a synthetic 4-node DB:

  • health/projects shape, layout validation, FTS sanitization (no SQLi),
  • size filter, label filter, 1-hop neighborhood expansion.
  • ~1.2 s on this host.

Compatibility

  • Same plugin slug codebase-memory — existing host wiring unchanged
  • Manifest fields unchanged (name, label, entry, css, api)
  • Bumped version to 0.2.0
  • Backwards-incompatible: iframe UI no longer rendered. To use the
    upstream 3D SPA again, revert to v0.1.0 or open a v0.3.0 PR with
    both modes selectable.

How to verify

# 1. Run unit tests
pytest plugins/codebase-memory/dashboard/tests/test_plugin_api.py -v

# 2. On the iBit host, deploy and reload the dashboard:
ssh ibid 'cd /home/ibid/.hermes/hermes-agent && \
  git fetch origin && git checkout feat/codebase-memory-sqlite-graph && \
  hermes dashboard restart'   # or kill + relaunch via systemd

# 3. Open /codebase-memory in the dashboard. The sidebar should list
#    projects (1 in iBit: codigosinsiesta). The graph renders in
#    ~200 ms, search works, click a node to see the snippet.

Closes #N/A — there's no open issue; this is a follow-up to PR #5
discovered when validating the iframe plugin on the 11 GB host.

…(no daemon)

The v0.1.0 plugin (PR #5) embedded the upstream SPA via iframe and
required the codebase-memory-mcp binary running on :9749. The binary
allocates 50% of system RAM at startup (mem.init budget_mb =
total_ram_mb / 2), which OOM-kills on hosts with <12 GB free — fatal
on this 11 GB / 0-swap host.

This version replaces the iframe with a native Cytoscape.js renderer
that reads the binary's own SQLite index files directly via stdlib
sqlite3. The binary is no longer required at runtime — only the
*.db files it has already written to ~/.cache/codebase-memory-mcp/.

Architecture
  Backend (plugin_api.py)
    GET /health                  binary-independent
    GET /projects                with size-based mega-project filter
    GET /project/{name}/health
    GET /layout?project=X&max_nodes=N&labels=...
                                 Cytoscape-ready graph payload
    GET /search?q=X              FTS5 BM25
    GET /code?project=X&qualified_name=Y
                                 source snippet

  Frontend (dist/index.js, dist/style.css)
    React component, Cytoscape.js 3.30 vendored (dist/vendor/), no SPA
    iframe. Renders graph in 2D with cose layout, sidebar with project
    list + search, detail panel on node click.

  Tests (tests/test_plugin_api.py)
    9 pytest cases against a synthetic 4-node/4-edge DB: health shape,
    project list, layout validation, FTS sanitization, size filter,
    label filter, 1-hop neighborhood expansion. Runs in ~1.2 s.

Performance (codigosinsiesta, 748 nodes / 861 edges / 2.8 MB DB)
  /health         <10 ms
  /projects       <50 ms
  /layout         19 ms cold, 5 ms warm
  /search         <20 ms
  /code           <10 ms
  Cytoscape layout on 518 nodes: ~150-300 ms in browser

Working-set: ~30 MB (Python + Cytoscape bundle) vs 6 GB for the binary.

Size filter: projects with >50k nodes are hidden by default (lobehub
at 401k). Override with CBM_MAX_PROJECT_NODES=0.

Cross-user: the plugin runs as hermesbot but reads DBs at
/home/ibid/.cache/codebase-memory-mcp/ via the path resolved at
startup (CBM_DB_DIR or config.yaml plugin block).

Backwards-compat: this version keeps the same plugin slug
'codebase-memory' so the existing PR #5 host wiring (manifest,
discovery, /api/plugins/codebase-memory/ mount) continues to work.
A future v0.3.0 may re-introduce the iframe as a fallback for users
who want the upstream 3D renderer; v0.2.0 is the recommended path
on memory-constrained hosts.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant