Skip to content

feat(cli): path p cache sync — incremental session ingestion#128

Closed
benbaarber wants to merge 2 commits into
ben/queryfrom
ben/cache-sync
Closed

feat(cli): path p cache sync — incremental session ingestion#128
benbaarber wants to merge 2 commits into
ben/queryfrom
ben/cache-sync

Conversation

@benbaarber

@benbaarber benbaarber commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

What

Adds path p cache sync [types…] — the first step toward a cache that fills itself. It enumerates sessions across the installed agent harnesses and derives into the cache only what is new or changed since the last sync, so users no longer have to p import each session by hand. Groundwork for path query running sync implicitly.

How

  • Enumeration reuses cmd_share::gather_sessions (the path share aggregation), filtered per requested harness. No args = all six; path p cache sync claude codex narrows.
  • Fingerprint: a session re-derives only when its last_activity or message_count differs from the sync manifest at ~/.toolpath/sync.json ({artifact type → session id → {project?, cache_id, last_activity?, message_count, synced_at}}). Written atomically (temp + rename, 0600), checkpointed after each type so an interrupted first run keeps finished types.
  • Derivation goes through new derive_*_session_with variants of the cmd_import helpers that take the provider manager, so listing and derivation agree on provider roots and tests inject fixture resolvers.
  • Semantics: sync owns refresh — it overwrites cache entries it re-derives (no --force dance; manual p import keeps error-on-hit). Never deletes: sessions removed upstream keep their cache docs and manifest records (archive, not mirror). Derivation failures warn + tally as failed without aborting.
  • Per-type stderr summary (claude 3 new, 1 updated, 240 unchanged); harnesses with no sessions stay silent on default runs.

Numbers (real data, this machine)

34 Claude sessions / 129 MB of JSONL: 5.7 s cold, 0.9 s incremental no-op — and the incremental run correctly caught the one live session that had changed. The 0.9 s floor is the providers' metadata pass (full parse per JSONL file); a stat-level fast path is the planned next optimization before path query runs sync implicitly.

Versioning

path-cli 0.15.0 → 0.16.0 (new subcommand) + toolpath-cli shim bump; workspace deps, site/_data/crates.json, CHANGELOG, CLAUDE.md, README updated per the release checklist. Site builds (11 pages); shim cargo checks.

Tests

  • 12 new unit tests in sync.rs: manifest roundtrip + 0600 + corrupt-file hint, first-sync/second-sync/changed-session lifecycle over resolver-injected Claude fixtures, type filtering, refresh-overwrite of unremembered cache entries, failure tallying, provider-missing errors, summary rendering.
  • 3 new integration tests: end-to-end ingest → skip → update through the real binary with sandboxed $HOME/$TOOLPATH_CONFIG_DIR; default run with no sessions; unknown-type rejection.
  • cargo test --workspace, cargo clippy --workspace -- -D warnings, cargo fmt all clean (includes a drive-by fix for a pre-existing redundant_closure in cmd_export.rs).

Next

Wire path query to run sync before querying (with --no-sync opt-out), then the stat-level enumeration fast path.


View with Codesmith Autofix with Codesmith
Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.

`path p cache sync [types…]` enumerates sessions across the installed
agent harnesses (claude/gemini/codex/opencode/cursor/pi, or just the
named ones) and derives into the cache only what is new or changed.
The manifest at ~/.toolpath/sync.json maps artifact type → session id →
{project?, cache_id, last_activity?, message_count, synced_at}; a
session re-derives only when its fingerprint (last_activity +
message_count) differs. Manifest writes are atomic (temp + rename,
0600) and checkpointed per type; sync overwrites what it re-derives
(refresh semantics) and never deletes — the cache is an archive, not a
mirror.

Enumeration reuses cmd_share::gather_sessions; per-session derivation
goes through new *_with variants of the cmd_import session helpers so
listing and derivation share provider managers (and tests can inject
fixture resolvers). Real-data numbers: 34 sessions / 129 MB of Claude
JSONL → 5.7 s cold, 0.9 s incremental no-op.

path-cli 0.15.0 → 0.16.0 (+ toolpath-cli shim bump).
@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown

🔍 Preview deployed: https://cff3a4c4.toolpath.pages.dev

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant