Add SQLite-backed registry with JSON migration support

[kdush]

Summary

Add SQLite-backed registry as the default storage backend, with automatic JSON migration support.

Changes

• add DbRegistry class with SQLite backend, WAL mode, and JSON migration
• add storage_backend config option (sqlite | json)
• update CLI commands to use get_registry() factory
• document storage backend options and migration behavior
• add tests for SQLite registry, migration, and backend selection

Testing

• 25 new tests passed for SQLite backend and migration
• all existing tests pass
• end-to-end verified: init, add, query, status, list
• confirmed SQLite creates hashes.db, hashes.db-wal, hashes.db-shm

Backward Compatibility

• storage_backend: json still works for existing setups
• automatic migration from hashes.json to hashes.db when switching to SQLite
• hashes.json preserved after migration for safety

[KylinMountain]

Code review

Found 2 issues:

1. run_lint was not migrated to get_registry() and still reads hashes.json directly. With SQLite as the new default backend, hashes.json is never created, so hashes is always {} and openkb lint exits with "Nothing to lint — no documents indexed yet" regardless of how many documents are actually indexed. print_list (line 602) and print_status (line 689) were migrated correctly — only this call site was missed.

OpenKB/openkb/cli.py

Lines 541 to 551 in 9124f51

	# Skip lint entirely when the KB has no indexed documents
	hashes_file = openkb_dir / "hashes.json"
	if hashes_file.exists():
	hashes = json.loads(hashes_file.read_text(encoding="utf-8"))
	else:
	hashes = {}
	if not hashes:
	click.echo("Nothing to lint — no documents indexed yet. Run `openkb add` first.")
	return

2. JSON-to-SQLite migration is not atomic. _init_db() creates and commits the empty schema file before _migrate_from_json() runs. If the process is killed (SIGKILL, OOM, power loss, disk full) between those two steps, the DB file exists on disk but is empty. On the next startup, should_migrate = migrate_from is not None and not path.exists() evaluates to False, so migration is never retried — every entry in hashes.json is silently abandoned and openkb thinks the KB has never been ingested. The JSON file is preserved on disk but never read again. Fix: do _init_db + insert in a single transaction, or use a sentinel (schema_version row, completion marker) instead of file existence to gate migration.

OpenKB/openkb/state.py

Lines 82 to 94 in 9124f51

	def __init__(self, path: Path, migrate_from: Path | None = None) -> None:
	"""Initialize DbRegistry.
	Args:
	path: Path to SQLite database file.
	migrate_from: Optional path to JSON file to migrate from.
	Migration only happens if DB doesn't exist yet.
	"""
	self._path = path
	should_migrate = migrate_from is not None and not path.exists()
	self._init_db()
	if should_migrate:
	self._migrate_from_json(migrate_from)

Coordination note (not an issue, but worth flagging): PR #30 (still open) adds get_by_path and remove_by_doc_name to HashRegistry and calls them from cli.py / converter.py. DbRegistry here does not implement either method, so whichever PR lands second will need to add them to the other backend or users on storage_backend: sqlite will hit AttributeError on re-add.

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}