Skip to content

the-claw: Add performance benchmarks for ingestion, SQLite, and embed paths #64

@clean6378-max-it

Description

@clean6378-max-it

Planned Effort

5 story points — sprint item #7 (Medium): Performance benchmarks for ingestion, SQLite, and LLM-related paths

One issue → one PR. claude-code-chat-browser export_count work is a separate issue/PR (Thursday sprint #4).

Depends on

Prerequisite Why
the-claw repo cloned Work lives in the-claw/claw
pnpm install Vitest + native deps (better-sqlite3)
Unit tests green on base branch pnpm --filter @the-claw/claw test before/after

No dependency on the browser export_count PR (different repo). Can run in parallel after morning browser work or on another machine.

Problem

No performance baselines exist for ingestion throughput, SQLite read paths, or LLM/embed latency. Regressions in the webhook → queue → store pipeline are invisible until production feels slow.

Goal

Add a local, repeatable benchmark harness with committed baseline numbers and a short README. No blocking CI gate this week (optional informational workflow step is fine).

Scope

1. Directory and docs

  • claw/benchmarks/README.md — prerequisites, run command, scenario descriptions, how to refresh baselines, CI note (optional/non-blocking).

2. Vitest bench (or tsx timing scripts)

Scenario Suggested target
Ingestion throughput src/pipeline/queue.tsIngestionQueue push/ack (in-memory or temp SQLite)
SQLite read latency src/warehouse/messages.ts (or query helpers) — temp DB + seed rows
LLM / embed path Mocked embed/LLM client — measure orchestration only (no live API keys in CI)

Suggested files:

  • claw/benchmarks/ingestion.bench.ts
  • claw/benchmarks/sqlite-read.bench.ts
  • claw/benchmarks/embed-path.bench.ts

3. Committed baselines

  • claw/benchmarks/baselines.json (or baselines.md) with date, environment note, and key metrics.

4. Package script

Add to claw/package.json:

"bench": "vitest bench"

Document run: pnpm --filter @the-claw/claw bench

Acceptance Criteria

  • Benchmark suite runnable locally (vitest bench documented in README)
  • Covers ingestion, SQLite read, and LLM/embed-related timing (network mocked)
  • Baseline numbers committed under claw/benchmarks/
  • README explains interpretation and refresh process
  • pnpm --filter @the-claw/claw test still green (benchmarks separate from unit tests)

Metadata

Metadata

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions