feat(cli): path p cache sync — incremental session ingestion

CHANGELOG.md

@@ -2,6 +2,37 @@
 
 All notable changes to the Toolpath workspace are documented here.
 
+## `path p cache sync` — incremental session ingestion — 2026-07-09
+
+Adds `path p cache sync [types…]`, the first step toward a cache that fills
+itself: it enumerates sessions across the installed agent harnesses and
+derives into the cache only what is new or changed since the last sync, so
+users no longer have to `p import` each session by hand.
+
+- **`path-cli`** (0.16.0):
+  - `path p cache sync [claude|gemini|codex|opencode|cursor|pi]…` syncs the
+    named artifact types; with no arguments it syncs every harness.
+    Enumeration reuses the `path share` aggregation, and per-session
+    derivation goes through the same provider managers, so listing and
+    derivation always agree on provider roots.
+  - The sync manifest at `~/.toolpath/sync.json` maps artifact type →
+    session id → `{project?, cache_id, last_activity?, message_count,
+    synced_at}`. A session is re-derived when its `last_activity` or
+    `message_count` differs from the record; otherwise it is skipped.
+    The manifest is written atomically (temp file + rename, `0600`) and
+    checkpointed after each type, so an interrupted first run keeps the
+    types it finished.
+  - Sync owns refresh semantics: it overwrites cache entries it
+    re-derives (no `--force` needed), while manual `p import` keeps its
+    error-on-hit default. Sessions deleted upstream keep both their cache
+    document and their manifest record — the cache is an archive, not a
+    mirror.
+  - Per-type summary on stderr (`claude   3 new, 1 updated, 240 unchanged`).
+    On a default run, harnesses with no sessions stay silent; explicit
+    types always report. Derivation failures warn and tally as `failed`
+    without aborting the run.
+- **`toolpath-cli`** (0.16.0): version bump only (tracks `path-cli`).
+
 ## Derive: resolve duplicate step ids — 2026-07-01
 
 - **`toolpath-convo`** (0.11.1): `derive_path` now guarantees the derived

CLAUDE.md

@@ -114,6 +114,8 @@ cargo run -p path-cli -- p export pathbase --input <ref>
 # Plumbing: manage the cache
 cargo run -p path-cli -- p cache ls
 cargo run -p path-cli -- p cache rm <cache-id>
+cargo run -p path-cli -- p cache sync                # ingest new/changed sessions from every harness
+cargo run -p path-cli -- p cache sync claude codex   # only these artifact types
 
 # Inspect / analyze
 cargo run -p path-cli -- p render dot --input doc.json
@@ -150,7 +152,7 @@ cargo run -p path-cli -- auth logout
 and no deprecation shim. They all now live exclusively under
 `path p …`.
 
-The **cache** at `~/.toolpath/documents/<cache-id>.json` is the single landing zone for every `import` (and for `import pathbase` downloads). Cache id is `<source>-<inner-id>` — e.g. `claude-abc123`, `git-main`, `pathbase-alex-pathstash-path-pr-42` (Pathbase paths key on `<owner>-<repo>-<slug>`, anon paths on `anon-pathstash-<uuid>`). Files are `0600`, parent directory `0700`. `$TOOLPATH_CONFIG_DIR` overrides the root. Default behavior: error on cache hit; pass `--force` to overwrite. `--no-cache` sends the JSON to stdout for shell composition.
+The **cache** at `~/.toolpath/documents/<cache-id>.json` is the single landing zone for every `import` (and for `import pathbase` downloads). Cache id is `<source>-<inner-id>` — e.g. `claude-abc123`, `git-main`, `pathbase-alex-pathstash-path-pr-42` (Pathbase paths key on `<owner>-<repo>-<slug>`, anon paths on `anon-pathstash-<uuid>`). Files are `0600`, parent directory `0700`. `$TOOLPATH_CONFIG_DIR` overrides the root. Default behavior: error on cache hit; pass `--force` to overwrite. `--no-cache` sends the JSON to stdout for shell composition. `p cache sync` fills the cache incrementally from the installed agent harnesses (see "Things to know") and always overwrites what it re-derives.
 
 `path auth login` prints `<base>/auth/cli`; the user opens it, logs in, and
 pastes the 8-character code back into the CLI. The CLI calls
@@ -190,7 +192,7 @@ Tests live alongside the code (`#[cfg(test)] mod tests`), plus `path-cli` has in
 - `toolpath-cursor`: 78 unit + 8 integration round-trip + 1 real-DB sanity + 1 doc test (state.vscdb SQLite reader, bubble store + composer header parsing, content-addressed blob lookup, projector with full TOOL_TABLE coverage, JSONL transcript ingest in `examples/dump_fixture.rs`)
 - `toolpath-pi`: 133 unit + 26 integration + 5 doc tests (types, paths, error, reader, io, provider)
 - `toolpath-dot`: 30 unit + 2 doc tests (render, visual conventions, escaping)
-- `path-cli`: 324 unit + 98 integration tests (import/export/cache, track sessions, merge, validate, roundtrip, render-md snapshots, deprecation aliases, pathbase HTTP mock-server tests, fzf-friendly TSV output, `path resume` orchestration with injectable `ExecStrategy`, `path query`/`path kind` jaq filters + kind-selector matching + step wrapping over a `$TOOLPATH_CONFIG_DIR` cache sandbox, streaming-planner recognition + streamed-output-equals-slurp equality checks). For an end-to-end check against a real Pathbase deployment, run `scripts/test-pathbase-live.sh <url>` — it does an anon round-trip in a sandboxed config dir and, if you're logged into that URL, an authed pathstash round-trip too.
+- `path-cli`: 336 unit + 101 integration tests (import/export/cache, track sessions, merge, validate, roundtrip, render-md snapshots, deprecation aliases, pathbase HTTP mock-server tests, fzf-friendly TSV output, `path resume` orchestration with injectable `ExecStrategy`, `path query`/`path kind` jaq filters + kind-selector matching + step wrapping over a `$TOOLPATH_CONFIG_DIR` cache sandbox, streaming-planner recognition + streamed-output-equals-slurp equality checks, `p cache sync` incremental ingestion — manifest fingerprints, refresh-overwrite, per-type filtering, failure tallying — over resolver-injected provider fixtures). For an end-to-end check against a real Pathbase deployment, run `scripts/test-pathbase-live.sh <url>` — it does an anon round-trip in a sandboxed config dir and, if you're logged into that URL, an authed pathstash round-trip too.
 - `toolpath-cli`: 0 tests (it's a one-line `path_cli::run()` shim crate that exists only so `cargo install toolpath-cli` keeps installing the `path` binary)
 
 Validate example documents: `for f in examples/*.json; do cargo run -p path-cli -- p validate --input "$f"; done`
@@ -259,3 +261,4 @@ Build the site after changes: `cd site && pnpm run build` (should produce 11 pag
 - `path share` is the one-shot equivalent of `path p import <harness> | path p export pathbase`. It probes installed agent harnesses (claude/gemini/codex/opencode/pi), aggregates their sessions into a single fzf picker, and ranks rows whose project (claude/gemini/pi) or recorded cwd (codex/opencode) canonicalizes to the current directory at the top. `--harness` narrows the picker to one provider; `--harness X --session Y` (and `--project P` for keyed providers) skips the picker entirely. Pathbase flags (`--url`, `--anon`, `--repo`, `--slug`, `--public`) match `path export pathbase`. By default the derived doc is written to the cache like `import` does; pass `--no-cache` to skip.
 - `path resume <input>` is the inverse of `path share`. It accepts a Pathbase URL, an `owner/repo/slug` shorthand, a local toolpath JSON file, or a cache id; resolves it (caching URL fetches under `~/.toolpath/documents/` unless `--no-cache`); validates that the document is a single agent-bearing `Path`; then opens an `fzf` harness picker (skipped with `--harness X`). The picker pre-selects the source harness inferred from `path.meta.source` (`claude-code`/`gemini-cli`/`codex`/`opencode`/`pi`) when it's installed. After picking, `path resume` projects the session into the harness's on-disk layout under the chosen working directory (default: shell cwd; override with `-C, --cwd P`) and `execvp`'s the harness's resume command (`claude -r <id>` / `gemini --resume <id>` / `codex resume <id>` / `opencode --session <id>` / `pi --session <id>`). On Windows it spawns and waits, propagating the exit code. The exec is mockable via `cmd_resume::ExecStrategy` — production uses `RealExec`; integration tests use `RecordingExec` to capture the recipe without launching a real harness.
 - `path query` does not load the whole cache into memory when it can avoid it. `crates/path-cli/src/query/plan.rs` parses the jaq filter into jaq's own AST (`jaq_core::load::parse::Term`) and classifies it into a `Plan`: `PerFileStream` (`.[] | g` element-wise work — run per document, print as you go), `Decompose { reduce }` (algebraic aggregations — run the whole filter per file, concatenate the per-file outputs, then run a derived combine: `map`→`add` (array concat), top-N `sort_by(k)|.[:N]`→`add | sort_by(k)|.[:N]`, `length`→`add` over exact integer counts), or `Slurp` (the always-correct whole-array fallback). Recognition is conservative — a non-distributive prefix like `unique`/`group_by` slurps, and so do scalar `add` (float sums re-associate across per-file partials), `min`/`max` (`[] | min == null` poisons the merge), and any unrecognized tail — so **the planner never changes an answer** — `crates/path-cli/src/query/filter.rs` tests assert streamed output equals slurp byte-for-byte. `filter::execute` compiles the filter once (jaq's compiled `Filter` is fully owned, so it's reused across files) and drives the plan; `mod.rs::stream_files` yields one document's wrapped steps at a time. `TOOLPATH_QUERY_EXPLAIN=1` prints the chosen plan to stderr. No user-facing flag — it's automatic. Tie-break caveat: a streamed top-N matches slurp's *ranking*, but boundary ties may resolve to different specific rows.
+- Cache sync: `path p cache sync [types…]` (`crates/path-cli/src/sync.rs`) incrementally ingests agent sessions into the cache — no args syncs all six harnesses. It enumerates via the `path share` aggregation (`cmd_share::gather_sessions`) and derives each session through the *same* provider managers (the `derive_*_session_with` helpers in `cmd_import.rs`), so listing and derivation agree on provider roots and tests can inject fixture resolvers. A session is re-derived only when its fingerprint (`last_activity` + `message_count`) differs from the manifest at `~/.toolpath/sync.json` (artifact type → session id → `{project?, cache_id, last_activity?, message_count, synced_at}`; atomic temp+rename writes, `0600`, checkpointed after each type). Sync always writes the cache with force — refresh semantics — and never deletes: sessions removed upstream keep their cache docs and manifest records (archive, not mirror). Derivation failures warn and tally, they don't abort. Caveats: enumeration costs the providers' metadata pass (full parse per JSONL session file — a stat-level fast path is the planned optimization before `path query` runs sync implicitly), and cursor composers without a workspace are skipped, same as `share`.

Cargo.toml

@@ -35,7 +35,7 @@ toolpath-github = { version = "0.6.0", path = "crates/toolpath-github" }
 toolpath-dot = { version = "0.5.0", path = "crates/toolpath-dot" }
 toolpath-md = { version = "0.7.0", path = "crates/toolpath-md" }
 toolpath-pi = { version = "0.6.0", path = "crates/toolpath-pi" }
-path-cli = { version = "0.15.0", path = "crates/path-cli" }
+path-cli = { version = "0.16.0", path = "crates/path-cli" }
 pathbase-client = { version = "0.2.0", path = "crates/pathbase-client" }
 
 reqwest = { version = "0.13", default-features = false, features = ["blocking", "json", "rustls"] }

README.md

@@ -98,6 +98,10 @@ path p import opencode
 # List what's in the cache
 path p cache ls
 
+# Ingest new/changed agent sessions into the cache (all harnesses, or named ones)
+path p cache sync
+path p cache sync claude codex
+
 # Export a cached document back into a Claude Code session
 path p export claude --input claude-<session-id> --project /path/to/resume
 

crates/path-cli/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "path-cli"
-version = "0.15.0"
+version = "0.16.0"
 edition.workspace = true
 license.workspace = true
 repository = "https://github.com/empathic/toolpath"

crates/path-cli/src/cmd_cache.rs

@@ -23,12 +23,22 @@ pub enum CacheOp {
         /// Cache id (filename without `.json`)
         id: String,
     },
+    /// Ingest agent sessions into the cache, deriving only what is new
+    /// or changed since the last sync (tracked in `$CONFIG_DIR/sync.json`)
+    #[cfg(not(target_os = "emscripten"))]
+    Sync {
+        /// Artifact types to sync (default: every agent harness)
+        #[arg(value_enum)]
+        types: Vec<crate::cmd_share::HarnessArg>,
+    },
 }
 
 pub fn run(op: CacheOp) -> Result<()> {
     match op {
         CacheOp::Ls => run_ls(),
         CacheOp::Rm { id } => run_rm(&id),
+        #[cfg(not(target_os = "emscripten"))]
+        CacheOp::Sync { types } => crate::sync::run(types),
     }
 }
 

crates/path-cli/src/cmd_export.rs

@@ -1405,9 +1405,9 @@ fn build_cursor_session(
         // Reuse the existing id when present, otherwise pre-create a
         // workspaceStorage entry so Cursor adopts ours on next open.
         let resolver = PathResolver::new();
-        if let Ok(ensured) = resolver.ensure_workspace_storage_entry(&canonical, |path| {
-            stable_workspace_id_for(path)
-        }) {
+        if let Ok(ensured) =
+            resolver.ensure_workspace_storage_entry(&canonical, stable_workspace_id_for)
+        {
             projector = projector.with_workspace_id(ensured.id);
             if ensured.created {
                 eprintln!(
@@ -1535,13 +1535,8 @@ fn cursor_open_hints(workspace: &std::path::Path) -> Vec<String> {
     }
 }
 
-
 #[cfg(not(target_os = "emscripten"))]
-fn upsert_cursor_kv(
-    tx: &rusqlite::Transaction<'_>,
-    key: &str,
-    value: &str,
-) -> Result<()> {
+fn upsert_cursor_kv(tx: &rusqlite::Transaction<'_>, key: &str, value: &str) -> Result<()> {
     tx.execute(
         "INSERT OR REPLACE INTO cursorDiskKV (key, value) VALUES (?1, ?2)",
         rusqlite::params![key, value],

crates/path-cli/src/cmd_import.rs

@@ -473,7 +473,17 @@ fn derive_claude_with_manager(
 /// Used by `cmd_share` after its picker has resolved the pair; mirrors the
 /// `(Some(p), Some(s), _)` arm in [`derive_claude_with_manager`].
 pub(crate) fn derive_claude_session(project: &str, session: &str) -> Result<DerivedDoc> {
-    let manager = toolpath_claude::ClaudeConvo::new();
+    derive_claude_session_with(&toolpath_claude::ClaudeConvo::new(), project, session)
+}
+
+/// [`derive_claude_session`] against a caller-supplied manager, so sync
+/// derives from the same roots it enumerated (and tests can inject a
+/// fixture resolver).
+pub(crate) fn derive_claude_session_with(
+    manager: &toolpath_claude::ClaudeConvo,
+    project: &str,
+    session: &str,
+) -> Result<DerivedDoc> {
     let cfg = toolpath_claude::derive::DeriveConfig {
         project_path: Some(project.to_string()),
         include_thinking: false,
@@ -698,7 +708,21 @@ pub(crate) fn derive_gemini_session(
     session: &str,
     include_thinking: bool,
 ) -> Result<DerivedDoc> {
-    let manager = toolpath_gemini::GeminiConvo::new();
+    derive_gemini_session_with(
+        &toolpath_gemini::GeminiConvo::new(),
+        project,
+        session,
+        include_thinking,
+    )
+}
+
+/// [`derive_gemini_session`] against a caller-supplied manager.
+pub(crate) fn derive_gemini_session_with(
+    manager: &toolpath_gemini::GeminiConvo,
+    project: &str,
+    session: &str,
+    include_thinking: bool,
+) -> Result<DerivedDoc> {
     let cfg = toolpath_gemini::derive::DeriveConfig {
         project_path: Some(project.to_string()),
         include_thinking,
@@ -885,7 +909,14 @@ fn derive_codex(session: Option<String>, all: bool) -> Result<Vec<DerivedDoc>> {
 
 /// Derive a single Codex session given an explicit session id.
 pub(crate) fn derive_codex_session(session: &str) -> Result<DerivedDoc> {
-    let manager = toolpath_codex::CodexConvo::new();
+    derive_codex_session_with(&toolpath_codex::CodexConvo::new(), session)
+}
+
+/// [`derive_codex_session`] against a caller-supplied manager.
+pub(crate) fn derive_codex_session_with(
+    manager: &toolpath_codex::CodexConvo,
+    session: &str,
+) -> Result<DerivedDoc> {
     let config = toolpath_codex::derive::DeriveConfig { project_path: None };
     let s = manager
         .read_session(session)
@@ -1035,7 +1066,20 @@ pub(crate) fn derive_opencode_session(
     session: &str,
     no_snapshot_diffs: bool,
 ) -> Result<DerivedDoc> {
-    let manager = toolpath_opencode::OpencodeConvo::new();
+    derive_opencode_session_with(
+        &toolpath_opencode::OpencodeConvo::new(),
+        session,
+        no_snapshot_diffs,
+    )
+}
+
+/// [`derive_opencode_session`] against a caller-supplied manager.
+#[cfg(not(target_os = "emscripten"))]
+pub(crate) fn derive_opencode_session_with(
+    manager: &toolpath_opencode::OpencodeConvo,
+    session: &str,
+    no_snapshot_diffs: bool,
+) -> Result<DerivedDoc> {
     let config = toolpath_opencode::derive::DeriveConfig {
         no_snapshot_diffs,
         ..Default::default()
@@ -1144,16 +1188,15 @@ fn derive_cursor(
             Ok(toolpath_cursor::derive_path(&s, &cfg))
         };
 
-        let workspace_filter = project.as_deref().map(|p| {
-            std::fs::canonicalize(p).unwrap_or_else(|_| PathBuf::from(p))
-        });
+        let workspace_filter = project
+            .as_deref()
+            .map(|p| std::fs::canonicalize(p).unwrap_or_else(|_| PathBuf::from(p)));
         let workspace_match = |m: &toolpath_cursor::CursorSessionMetadata| -> bool {
             match (&workspace_filter, &m.workspace_path) {
                 (None, _) => true,
                 (Some(_), None) => false,
                 (Some(want), Some(have)) => {
-                    let canonical =
-                        std::fs::canonicalize(have).unwrap_or_else(|_| have.clone());
+                    let canonical = std::fs::canonicalize(have).unwrap_or_else(|_| have.clone());
                     &canonical == want
                 }
             }
@@ -1209,7 +1252,15 @@ fn derive_cursor(
 /// Derive a single cursor composer given an explicit composer id.
 #[cfg(not(target_os = "emscripten"))]
 pub(crate) fn derive_cursor_session(session: &str) -> Result<DerivedDoc> {
-    let manager = toolpath_cursor::CursorConvo::new();
+    derive_cursor_session_with(&toolpath_cursor::CursorConvo::new(), session)
+}
+
+/// [`derive_cursor_session`] against a caller-supplied manager.
+#[cfg(not(target_os = "emscripten"))]
+pub(crate) fn derive_cursor_session_with(
+    manager: &toolpath_cursor::CursorConvo,
+    session: &str,
+) -> Result<DerivedDoc> {
     let s = manager
         .read_session(session)
         .map_err(|e| anyhow::anyhow!("{}", e))?;
@@ -1410,6 +1461,15 @@ pub(crate) fn derive_pi_session(
     } else {
         toolpath_pi::PiConvo::new()
     };
+    derive_pi_session_with(&manager, project, session)
+}
+
+/// [`derive_pi_session`] against a caller-supplied manager.
+pub(crate) fn derive_pi_session_with(
+    manager: &toolpath_pi::PiConvo,
+    project: &str,
+    session: &str,
+) -> Result<DerivedDoc> {
     let config = toolpath_pi::DeriveConfig::default();
     let session = manager
         .read_session(project, session)

crates/path-cli/src/cmd_share.rs

@@ -77,6 +77,16 @@ pub(crate) enum Harness {
 }
 
 impl Harness {
+    /// Every harness, in presentation order.
+    pub(crate) const ALL: [Harness; 6] = [
+        Harness::Claude,
+        Harness::Gemini,
+        Harness::Codex,
+        Harness::Opencode,
+        Harness::Cursor,
+        Harness::Pi,
+    ];
+
     pub(crate) fn name(&self) -> &'static str {
         match self {
             Harness::Claude => "claude",
@@ -836,10 +846,7 @@ fn harness_status_pi(bundle: &HarnessBundle, home: Option<&std::path::Path>) ->
     }
 }
 
-fn harness_status_cursor(
-    bundle: &HarnessBundle,
-    home: Option<&std::path::Path>,
-) -> HarnessStatus {
+fn harness_status_cursor(bundle: &HarnessBundle, home: Option<&std::path::Path>) -> HarnessStatus {
     let Some(mgr) = &bundle.cursor else {
         return HarnessStatus::unresolved();
     };