feat(pi,opencode): preserve dropped session data via events + Turn.model; remove dead Turn.extra plumbing

CHANGELOG.md

@@ -2,6 +2,31 @@
 
 All notable changes to the Toolpath workspace are documented here.
 
+## Preserve pi metadata entries and opencode user model + snapshot SHAs — 2026-07-08
+
+Plumbs previously-dropped provider data through the existing IR
+affordances (`ConversationView.events`, `Turn.model`) instead of merely
+documenting the loss. No IR changes; `toolpath-convo` is unchanged.
+
+- **`toolpath-pi`** (0.6.2; 0.6.1 is claimed by a sibling PR in flight):
+  `session_to_view` now routes `ModelChange` / `ThinkingLevelChange` /
+  `Label` entries into `ConversationView.events` (`model_change` /
+  `thinking_level_change` / `label`) instead of discarding them, and
+  `PiProjector` re-materializes those events as real Pi entries with
+  their original ids and parentIds — so the entries survive a full
+  pi → view → Path → view → pi round-trip. Each re-emitted entry is
+  inserted directly after its parent for sane file order.
+- **`toolpath-opencode`** (0.5.2; 0.5.1 is claimed by a sibling PR in
+  flight): user turns now carry `Turn.model` from the wire
+  `model.modelID` (same bare-modelID convention as assistant turns) and
+  the projector writes it back into the projected user message.
+  Snapshot SHAs from `step-start` / `step-finish` / `snapshot` parts
+  now also ride `ConversationView.events` as `part.snapshot` events
+  (chained per message so the derived Path stays linear), and the
+  projector restores them onto the emitted step parts — snapshot SHAs
+  survive a full Path round-trip, so a re-imported projection can
+  regenerate diffs.
+
 ## Derive: resolve duplicate step ids — 2026-07-01
 
 - **`toolpath-convo`** (0.11.1): `derive_path` now guarantees the derived

Cargo.toml

@@ -29,12 +29,12 @@ toolpath-git = { version = "0.6.0", path = "crates/toolpath-git" }
 toolpath-claude = { version = "0.12.0", path = "crates/toolpath-claude", default-features = false }
 toolpath-gemini = { version = "0.6.0", path = "crates/toolpath-gemini", default-features = false }
 toolpath-codex = { version = "0.6.0", path = "crates/toolpath-codex" }
-toolpath-opencode = { version = "0.5.0", path = "crates/toolpath-opencode" }
+toolpath-opencode = { version = "0.5.2", path = "crates/toolpath-opencode" }
 toolpath-cursor = { version = "0.2.0", path = "crates/toolpath-cursor" }
 toolpath-github = { version = "0.6.0", path = "crates/toolpath-github" }
 toolpath-dot = { version = "0.5.0", path = "crates/toolpath-dot" }
 toolpath-md = { version = "0.7.0", path = "crates/toolpath-md" }
-toolpath-pi = { version = "0.6.0", path = "crates/toolpath-pi" }
+toolpath-pi = { version = "0.6.2", path = "crates/toolpath-pi" }
 path-cli = { version = "0.14.0", path = "crates/path-cli" }
 pathbase-client = { version = "0.2.0", path = "crates/pathbase-client" }
 

RFC.md

@@ -296,10 +296,11 @@ unrecognized URIs should be treated as a generic path. Kind URIs are
 immutable, semver-versioned, and revisions ship at a new version URI.
 
 Defined kinds are listed at <https://toolpath.net/kinds/>. The only one defined
-so far is `https://toolpath.net/kinds/agent-coding-session/v1.0.0` — a path
-recording an AI coding conversation, where each conversational-turn step
-carries a `"conversation.append"` structural change with the turn's role,
-text, and so on. See the linked spec for the full contract.
+so far is `https://toolpath.net/kinds/agent-coding-session/v1.1.0` (the
+superseded `v1.0.0` URI is still recognized) — a path recording an AI coding
+conversation, where each conversational-turn step carries a
+`"conversation.append"` structural change with the turn's role, text, and so
+on. See the linked spec for the full contract.
 
 #### Actor Definitions
 

crates/path-cli/src/cmd_resume.rs

@@ -17,7 +17,7 @@
 //! Without `--harness`, an `fzf` picker shows installed harnesses
 //! with the source harness pre-selected. Source comes from
 //! `path.meta.source` (`claude-code`, `gemini-cli`, `codex`,
-//! `opencode`, `pi`) with actor-string fallback.
+//! `opencode`, `cursor`, `pi`) with actor-string fallback.
 //!
 //! ## Project directory
 //!

crates/toolpath-claude/README.md

@@ -208,22 +208,25 @@ cache read, cache write) into a single aggregate.
 (deduplicated, first-touch order), derived from `FileWrite`-categorized tool inputs.
 
 **Provider-specific metadata** — Claude log entries often carry extra fields
-(e.g. `subtype`, `data`) that don't map to the common `Turn` schema. These are
-forwarded into `Turn.extra["claude"]` so trait-only consumers can access them
-without importing Claude-specific types:
+(e.g. `subtype`, `data`) that don't map to the common `Turn` schema. `Turn`
+itself has no field to carry them, so a `conversation.append` turn drops
+them; but a non-turn entry (one with no clean `Turn` mapping) surfaces its
+full payload as a `WatcherEvent::Progress` event, with the entry's catch-all
+fields nested under `data["claude"]`:
 
 ```rust,ignore
-// State inference from provider metadata
-if let Some(claude) = turn.extra.get("claude") {
-    if claude.get("subtype").and_then(|v| v.as_str()) == Some("init") {
-        // This is a session initialization entry
-    }
+// State inference from a Progress event's provider metadata
+if let WatcherEvent::Progress { data, .. } = event
+    && let Some(claude) = data.get("claude")
+    && claude.get("subtype").and_then(|v| v.as_str()) == Some("init")
+{
+    // This is a session initialization entry
 }
 ```
 
-For `WatcherEvent::Progress` events, the full entry payload is similarly
-available under `data["claude"]` — carrying fields like `data.type`,
-`data.hookName`, `data.agentId`, and `data.message`.
+The full entry payload is available under `data["claude"]` — carrying
+fields like `data.type`, `data.hookName`, `data.agentId`, and
+`data.message`.
 
 See [`toolpath-convo`](https://crates.io/crates/toolpath-convo) for the full trait and type definitions.
 

crates/toolpath-claude/src/project.rs

@@ -309,12 +309,13 @@ fn tool_result_event_to_entry(
 
 /// Apply Claude-specific metadata from a [`Turn`] onto a [`ConversationEntry`].
 ///
-/// Populates `cwd` and `git_branch` from [`Turn::environment`], and
-/// `version`, `user_type`, `request_id` from `Turn::extra["claude"]`.
-/// Remaining keys from the `"claude"` extras are merged into the entry's
-/// own `extra` map so they serialize as top-level fields (via `#[serde(flatten)]`).
+/// Populates `cwd` and `git_branch` from [`Turn::environment`] when the
+/// entry doesn't already carry them. The IR has no field for `version`,
+/// `user_type`, `request_id`, or a per-entry catch-all, so a
+/// claude → IR → claude round-trip can't recover those — the projected
+/// entry's fields stay `None` and the harness fills in defaults at
+/// write time.
 fn apply_turn_metadata(entry: &mut ConversationEntry, turn: &Turn) {
-    // From Turn.environment
     if let Some(env) = &turn.environment {
         if entry.cwd.is_none() {
             entry.cwd = env.working_dir.clone();
@@ -323,12 +324,6 @@ fn apply_turn_metadata(entry: &mut ConversationEntry, turn: &Turn) {
             entry.git_branch = env.vcs_branch.clone();
         }
     }
-
-    // Source-format details (`version`, `user_type`, `request_id`,
-    // per-entry catch-all) used to ride through `Turn.extra["claude"]` for
-    // claude → IR → claude round-trip. The IR no longer carries
-    // provider-specific extras; the projected entry's fields stay `None`
-    // and the harness fills in defaults at write time.
 }
 
 /// Build a `ConversationEntry` for a user turn.

crates/toolpath-codex/src/derive.rs

@@ -18,8 +18,9 @@ use toolpath::v1::Path;
 /// OpenAI's servers — not useful in a human-readable digest. Plaintext
 /// reasoning summaries (rare) land on `Turn.thinking` automatically
 /// and surface in the derived path without a flag. The raw ciphertext
-/// is preserved under `Turn.extra["codex"]["reasoning_encrypted"]` for
-/// round-trip fidelity but never rendered.
+/// is dropped: the IR carries no provider-namespaced extras field to
+/// preserve it in, so it never reaches `Turn` and cannot be recovered
+/// on a round-trip (see `encrypted_reasoning_does_not_land_on_thinking`).
 #[derive(Debug, Clone, Default)]
 pub struct DeriveConfig {
     /// Override `path.base.uri`. Defaults to the cwd from session_meta.

crates/toolpath-codex/src/project.rs

@@ -8,22 +8,18 @@
 //! line, followed by a `turn_context`, then per-turn `response_item`
 //! and `event_msg` lines).
 //!
-//! The projector consumes provider-specific data the forward path
-//! stashed under `Turn.extra["codex"]`:
-//!
-//! - `reasoning_encrypted`: opaque ciphertext blobs that round-trip
-//!   verbatim as `Reasoning::encrypted_content`
-//! - `tool_extras`: per-`call_id` extras (exec exit_codes, custom-call
-//!   statuses, patch-change manifests, etc.) preserved on the matching
-//!   `function_call_output` / `custom_tool_call_output` line
+//! The provider-agnostic IR (`Turn`) carries no provider-namespaced
+//! extras, so a Codex→View→Codex round-trip cannot recover
+//! Codex-specific data the forward path didn't lift into a typed
+//! `Turn` field: reasoning ciphertext (`Reasoning::encrypted_content`)
+//! and per-`call_id` tool extras (exec exit codes, custom-call
+//! statuses, patch-change manifests) are dropped. `Turn.thinking`
+//! (the public reasoning summary) and `ToolResult::is_error` are
+//! reconstructed from typed fields instead.
 //!
 //! For non-Codex sources, the projector synthesizes sensible defaults
 //! (originator: `"codex-toolpath"`, source: `"cli"`, model_provider:
 //! `"openai"`).
-//!
-//! Foreign-namespace extras (`Turn.extra["claude"]`,
-//! `Turn.extra["gemini"]`, …) are dropped — they have no meaning in
-//! Codex's protocol and would pollute the JSONL.
 
 use std::collections::HashMap;
 use std::path::PathBuf;
@@ -184,9 +180,8 @@ fn project_view(
                 current_group = Some(group);
             }
         }
-        let codex = codex_extras(turn).cloned().unwrap_or_default();
         let is_final_assistant = Some(idx) == last_assistant_idx;
-        emit_turn_lines(turn, &codex, is_final_assistant, &cwd, &mut lines, &mut running);
+        emit_turn_lines(turn, is_final_assistant, &cwd, &mut lines, &mut running);
     }
 
     Ok(crate::types::Session {
@@ -260,27 +255,16 @@ fn make_turn_context_line(
     }
 }
 
-/// Used to return `Turn.extra["codex"]`; the IR no longer carries
-/// provider-namespaced extras. Always `None`. Callers fall back to
-/// reconstructing source-format details from typed IR fields and
-/// reasonable defaults.
-fn codex_extras(_turn: &Turn) -> Option<&'static Map<String, Value>> {
-    None
-}
-
 fn emit_turn_lines(
     turn: &Turn,
-    codex: &Map<String, Value>,
     is_final_assistant: bool,
     session_cwd: &str,
     lines: &mut Vec<RolloutLine>,
     running: &mut toolpath_convo::TokenUsage,
 ) {
     match &turn.role {
         Role::User => emit_user_message(turn, lines),
-        Role::Assistant => {
-            emit_assistant(turn, codex, is_final_assistant, session_cwd, lines, running)
-        }
+        Role::Assistant => emit_assistant(turn, is_final_assistant, session_cwd, lines, running),
         Role::System => emit_developer_message(turn, lines),
         Role::Other(_) => {
             // Unknown roles don't have a clean Codex analog; emit them
@@ -348,7 +332,6 @@ fn emit_developer_message(turn: &Turn, lines: &mut Vec<RolloutLine>) {
 
 fn emit_assistant(
     turn: &Turn,
-    codex: &Map<String, Value>,
     is_final_assistant: bool,
     session_cwd: &str,
     lines: &mut Vec<RolloutLine>,
@@ -357,38 +340,13 @@ fn emit_assistant(
     // Order matches what Codex itself emits per turn:
     //   reasoning? → message → (function_call → function_call_output)*
     //
-    // For Codex→View→Codex round-trips, encrypted ciphertext lives on
-    // `Turn.extra["codex"]["reasoning_encrypted"]` and round-trips
-    // verbatim. For cross-harness sources, `Turn.thinking` is the
-    // public reasoning summary; we put it in `summary` instead of
-    // re-encrypting it.
-    let encrypted_blobs = codex
-        .get("reasoning_encrypted")
-        .and_then(Value::as_array)
-        .cloned()
-        .unwrap_or_default();
-    if !encrypted_blobs.is_empty() {
-        for blob in encrypted_blobs {
-            let enc = blob.as_str().map(str::to_string);
-            let r = Reasoning {
-                id: None,
-                summary: vec![],
-                content: None,
-                encrypted_content: enc,
-                extra: HashMap::new(),
-            };
-            lines.push(response_item_line(
-                &turn.timestamp,
-                "reasoning",
-                serde_json::to_value(&r).unwrap_or(Value::Null),
-            ));
-        }
-    } else if let Some(thinking) = &turn.thinking
+    // The IR carries no reasoning ciphertext, so a Codex→View→Codex
+    // round-trip can't restore `Reasoning::encrypted_content`.
+    // `Turn.thinking` is the public reasoning summary; render it as a
+    // single summary entry instead of re-encrypting it.
+    if let Some(thinking) = &turn.thinking
         && !thinking.is_empty()
     {
-        // Foreign-source: render thinking as a single summary entry.
-        // This is how Codex serializes plain reasoning summaries when
-        // it has them.
         let r = Reasoning {
             id: None,
             summary: vec![json!({"type": "summary_text", "text": thinking})],
@@ -448,14 +406,9 @@ fn emit_assistant(
         }
     }
 
-    let tool_extras = codex
-        .get("tool_extras")
-        .and_then(Value::as_object)
-        .cloned()
-        .unwrap_or_default();
     for tu in &turn.tool_uses {
         let name = tool_native_name(tu);
-        emit_tool_call(turn, tu, &name, &tool_extras, session_cwd, lines);
+        emit_tool_call(turn, tu, &name, session_cwd, lines);
     }
 
     // Advance the session-cumulative counter by this step's contribution
@@ -499,30 +452,22 @@ fn emit_tool_call(
     turn: &Turn,
     tu: &ToolInvocation,
     name: &str,
-    tool_extras: &Map<String, Value>,
     session_cwd: &str,
     lines: &mut Vec<RolloutLine>,
 ) {
-    let extras_for_call = tool_extras
-        .get(&tu.id)
-        .and_then(Value::as_object)
-        .cloned()
-        .unwrap_or_default();
-
     if name == "apply_patch" {
         let input_str = match &tu.input {
             Value::String(s) => s.clone(),
             other => serde_json::to_string(other).unwrap_or_default(),
         };
-        let status = extras_for_call
-            .get("status")
-            .and_then(Value::as_str)
-            .map(str::to_string);
+        // The IR carries no per-call-id status extras, so a
+        // Codex→View→Codex round-trip can't restore the original
+        // `custom_tool_call.status`.
         let call = CustomToolCall {
             name: name.to_string(),
             input: input_str,
             call_id: tu.id.clone(),
-            status,
+            status: None,
             id: None,
             extra: HashMap::new(),
         };

crates/toolpath-codex/src/provider.rs

@@ -14,9 +14,9 @@
 //!    see the matching `*_output` by `call_id`.
 //! 5. `event_msg.exec_command_end` enriches the already-emitted tool
 //!    invocation with the exit code / stdout / stderr.
-//! 6. `event_msg.patch_apply_end` is captured on the current turn's
-//!    `extra["codex"]["patch_changes"]` — the derive layer consumes it
-//!    for file-artifact sibling changes.
+//! 6. `event_msg.patch_apply_end` populates the current turn's
+//!    `Turn.file_mutations` — the derive layer consumes it for
+//!    file-artifact sibling changes.
 //! 7. Token accounting. `turn_context` / `task_started` open an API round
 //!    (`turn_id`); assistant turns in it share that ID as `Turn.group_id`.
 //!    `event_msg.token_count` carries the SESSION-cumulative
@@ -297,11 +297,12 @@ impl<'a> Builder<'a> {
         };
 
         // Producer (originator + cli_version) lifts onto the typed view
-        // field. `model_provider` already lives on each assistant
-        // `ActorDefinition.provider`. Codex's `source` and `forked_from_id`
-        // are wire-level fields with no cross-harness analog — the codex
-        // projector hard-codes defaults on the return path, so we let them
-        // drop on this side.
+        // field. `model_provider`, `source`, and `forked_from_id` are
+        // wire-level fields with no cross-harness analog and no typed
+        // home in the IR — the codex projector hard-codes defaults on
+        // the return path, so we let them drop on this side. (Note:
+        // `ActorDefinition.provider` is the shared derive's static
+        // `view.provider_id` ("codex"), not `session_meta.model_provider`.)
         let producer = meta.as_ref().map(|m| ProducerInfo {
             name: m.originator.clone(),
             version: Some(m.cli_version.clone()),

crates/toolpath-convo/README.md

@@ -133,7 +133,7 @@ if let Some(id) = event.turn_id() {
 }
 ```
 
-Provider-specific metadata lives in `Turn.extra`, namespaced by provider (e.g. `turn.extra["claude"]`). This keeps the common schema clean while giving consumers opt-in access to provider internals.
+`Turn` has no provider-namespaced extras field — it's a fixed, typed schema, so provider-specific data that doesn't map onto one of its fields is dropped on the `ConversationView` projection. Non-turn provider metadata (e.g. session-init markers) is still available at the edges: `WatcherEvent::Progress { kind, data }` carries the full source payload for entries that don't map to a `Turn` at all, with the provider's own fields nested under `data["<provider>"]` by convention (e.g. `data["claude"]`).
 
 ## Provider implementations