Rework: regenerate compaction instructions constantly in the background, surface instructions when prep-compact is called

.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "prep-compact",
-  "description": "When your Claude Code session's real token count (summed input + cache_creation + cache_read from the newest main-chain .message.usage) crosses a configurable threshold (default 450K), a hook tells Claude to invoke the prep-compact skill. The skill surveys current state and emits a tailored /compact <instructions> mini-schema (goal/next/files/decisions/state) so the post-compact session resumes correctly.",
-  "version": "2.1.1",
+  "description": "When your Claude Code session's real token count crosses a configurable threshold (default 450K), an informational reminder names the warm handoff file. A Stop hook continuously maintains an on-disk handoff JSON (cumulative file paths, in-progress todos, active subagent launches, recent user-message quotes). The /prep-compact skill reads this warm handoff and adds an analytical layer (decisions, constraints, blockers, verb-anchored next-step) to emit a tailored /compact <instructions> block.",
+  "version": "3.0.0",
   "author": {
     "name": "Koen van der Heide",
     "url": "https://github.com/koenvdheide"

CHANGELOG.md

@@ -4,6 +4,42 @@ All notable changes to prep-compact will be documented in this file.
 
 The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [3.0.0] - 2026-04-26
+
+Adds a Stop hook maintaining a continuously-fresh on-disk handoff. UserPromptSubmit reminder becomes informational. Skill reads warm handoff plus a targeted analytical pass — no fresh full survey when the handoff is current. Four Codex review rounds shaped the design.
+
+### Why
+
+User feedback: Claude Code's auto-compact feels worse than Codex CLI's despite the larger Opus 1M window. Faster + better continuity + less interruption is the user goal. Research (Codex CLI + pi-mono) clarified that most "smoothness" is runtime-owned, but a sidecar can deliver three concrete wins: (a) less ceremony at the threshold, (b) always-prepared handoff, (c) better-anchored resume material.
+
+### Added
+
+- **Stop hook** (`hooks/update-handoff.sh`) — runs after each assistant message; tail-reads the transcript with bounded buffer + per-line cap; extracts file paths (Tier-A: Read/Edit/Write/NotebookEdit; Tier-B: Glob/Grep paths in tool inputs and result-block content); recent user-message quotes (capped 5 / 20000 chars); in-progress todos; recent Task launches; merges with prior handoff; FIFO-evicts at 200-path cap; writes JSON atomically via `tempfile.mkstemp` + `os.replace` with `PermissionError` retry.
+- **`handoff-<safe_sid>.json`** (new per-session file under `${CLAUDE_PLUGIN_DATA}`) with documented schema.
+- **`PREP_COMPACT_NO_USER_QUOTES=1`** opt-out env var. When set: hook writes empty `recent_user_requests` AND eager-clears any pre-existing quotes from prior handoff during merge.
+- **T-0 fixture gate** (`stop-real.json`): harness skips Stop-hook tests with explicit message if missing (local dev), hard-fails if malformed, hard-fails any skip when `$CI` is set.
+
+### Changed
+
+- **Reminder copy** (UserPromptSubmit): no longer says "Invoke the prep-compact skill." Two informational variants: handoff-aware (names the on-disk path) and no-handoff (legacy short copy).
+- **`prep-compact` skill** repurposed: discovers warm handoff via newest-mtime-matching-cwd; uses extractive fields directly; performs targeted current-conversation pass for analytical fields with concrete per-field source priorities and `unknown` fallbacks.
+- **PRIVACY.md** broadened to describe new persistence scope, opt-out, eager-clear behavior, and Anthropic flow during `/compact`.
+
+### Removed
+
+- Auto-invoke skill steering from threshold reminder.
+- Hook-side `goal` heuristic (violated extractive/analytical split — moved to skill).
+
+### Rationale
+
+Four Codex review rounds (compare-decide vs pi-mono; red-team r2 on draft v1; red-team r3 on draft v2; red-team r4 on draft v3) shaped this design. Codex r3 forced the format switch to pure JSON (eliminated YAML/Markdown escape and fence-breakage classes), FIFO eviction simplification, dropping "warm-but-trailing" as a user-visible signal, eager-clear privacy semantics, and concrete analytical-pass recipe. Codex r4 produced one targeted patch (T-0 gate behavior on malformed fixture and CI strictness) and recommended stopping the review cycle.
+
+### Breaking
+
+- Reminder copy is user-visible; downstream code that screen-scrapes the literal "Invoke the prep-compact skill" string will break. Use the new informational variants.
+- Skill behavior is user-visible; skill output is now sometimes prefixed with a "no warm handoff matched cwd" note when the fallback path runs.
+- New on-disk file format under `${CLAUDE_PLUGIN_DATA}/handoff-<sid>.json`. Plugin uninstall + reinstall is the clean path.
+
 ## [2.1.1] - 2026-04-25
 
 ### Changed

PRIVACY.md

@@ -1,42 +1,52 @@
 # Privacy
 
-prep-compact is a local-only Claude Code plugin. It does not send data over the network, does not use telemetry, and does not record session content.
+prep-compact runs entirely on your machine. The plugin makes no network calls of its own. The Anthropic API is only contacted when you (the user) run `/prep-compact` or `/compact` through Claude Code's normal flow.
 
-## What the plugin accesses
+## Local persistence
 
-The `UserPromptSubmit` hook receives a JSON payload on stdin from Claude Code. The plugin reads two fields from that payload:
+The plugin writes two kinds of files under `${CLAUDE_PLUGIN_DATA}` (default `~/.claude/cache/` if unset):
 
-- `session_id` — used to namespace the per-session state file.
-- `transcript_path` — used to stat the transcript file and to tail-read the last 256 KB. The tail is parsed to extract API-returned `.message.usage` numbers (specifically `input_tokens`, `cache_creation_input_tokens`, and `cache_read_input_tokens` from the newest main-chain assistant turn). No transcript content is persisted, logged, or transmitted.
+- **`compact-warned-<safe_sid>`** — empty presence marker for the threshold reminder (one per session). No content recorded.
+- **`handoff-<safe_sid>.json`** (new in v3.0) — the warm handoff. Contents:
+  - `cumulative_files`, `recent_files` — file paths the session has touched (extracted from `Read`/`Edit`/`Write`/`NotebookEdit`/`Glob`/`Grep` tool calls). No file CONTENT, only paths.
+  - `in_progress`, `recent_task_launches` — your todo state and subagent launches, extracted from `TodoWrite`/`Task` tool calls.
+  - **`recent_user_requests`** — verbatim quotes of your most recent user messages (capped at 5 messages OR 20000 chars). This is the most sensitive field; it is the only place the plugin stores raw text from your prompts.
+  - `version`, `session_id`, `cwd`, `transcript_path`, `transcript_mtime_at_write`, `written_at` — bookkeeping metadata.
 
-`session_id` is validated against `^[A-Za-z0-9_-]{1,64}$` before use as a filename. Values that don't match fall back to a SHA-1 hex hash of the raw value, so unexpected characters cannot escape the cache directory.
+The hook does NOT persist tool result content. It performs a bounded read (≤2000 chars per result) of `Glob` and `Grep` `tool_result` blocks for the sole purpose of extracting path tokens that appear in their output — only the extracted path strings end up in the handoff, never the result text itself. It does NOT read or scan `Read`/`Edit`/`Write` tool results, `Bash` command output, or any other tool result content.
 
-## What the plugin persists
+## Opting out of user-quote persistence
 
-The hook writes one small file under `$CLAUDE_PLUGIN_DATA` (or `~/.claude/cache` as a fallback), per session:
+Set `PREP_COMPACT_NO_USER_QUOTES=1` in your shell profile or `~/.claude/settings.json` under `env`:
 
-- `compact-warned-<session_id>` — an empty flag file used as a presence marker to avoid re-firing the reminder for the same threshold-crossing.
+```json
+{
+  "env": {
+    "PREP_COMPACT_NO_USER_QUOTES": "1"
+  }
+}
+```
 
-The file contains no prompts, responses, tool calls, project file paths, or any session content.
+When set:
 
-(v1.0.x also wrote a `compact-baseline-<session_id>` integer file used by the since-removed byte-path. If any such file is left over on disk, it is unread by v2.0.0 and can be deleted safely.)
+1. The Stop hook writes empty `recent_user_requests` going forward.
+2. **Eager-clear**: the Stop hook also drops any pre-existing `recent_user_requests` from the prior handoff during merge. You do not need to delete the handoff manually — the next assistant turn will overwrite with no quotes.
 
-## What the plugin does not do
+## Network and Anthropic flow
 
-- No network requests, ever.
-- No telemetry, analytics, or usage reporting.
-- No writes outside the cache directory.
-- No full-transcript reads. Only the last 256 KB is read, and only to extract `.message.usage` integer fields — no prompt/response content is parsed or retained.
-- No modifications to your project files, shell environment, or Claude Code settings.
+- The Stop hook and UserPromptSubmit hook make no network calls.
+- The plugin's skill (`/prep-compact`) runs as part of your Claude Code session and uses the same Anthropic API path Claude Code itself uses. Inputs to the skill (including any quoted user messages from the warm handoff) flow through that path.
+- When you run the emitted `/compact <instructions>` block, the contents — including any quoted user-message excerpts embedded in the block — are sent to Claude Code's normal Anthropic compaction pipeline. They are subject to whatever data-handling terms apply to your Anthropic account.
 
-## About the /compact output
+## Session ID safety
 
-The skill drafts a `/compact <instructions>` block. You paste and run it yourself. When you do, the instructions text is sent to Anthropic as part of Claude Code's normal `/compact` flow — the same as any other `/compact` invocation you run by hand. That transmission is governed by Anthropic's privacy policy, not this plugin.
+`session_id` is validated with regex `^[A-Za-z0-9_-]{1,64}$` before use as a filename component. Exotic values are SHA-1-hashed to prevent path-escape via `../` or absolute paths.
 
-## Third parties
+## Uninstall
 
-None. The plugin has no external dependencies at runtime beyond your local shell (bash + coreutils) and Python 3.
+Plugin uninstall via `/plugin uninstall` removes the plugin and (per Claude Code's standard plugin lifecycle) clears `${CLAUDE_PLUGIN_DATA}`. To preserve the data dir, use `/plugin uninstall --keep-data`.
 
-## Contact
+## See also
 
-Issues or questions: <https://github.com/koenvdheide/prep-compact/issues>
+- [README.md](README.md) — feature overview and configuration.
+- [LICENSE](LICENSE) — MIT.

README.md

@@ -10,10 +10,18 @@ A Claude Code plugin that nudges Claude to prepare tailored `/compact` instructi
 
 CC does not programatically expose how many tokens are in use for the current session (even though we can see ourselves with /context), so there is no direct way to fire a reminder based on the current session's token use. However, CC does store how many tokens are in use at the moment of a given user prompt in its transcript file. This plugin works by firing a hook to read the tail end of this transcript file, parse the stated token usage and compare it to a set threshold. Under the hood it works like this:
 
-> A `UserPromptSubmit` hook fires on every prompt submission. It tail-reads the last 256 KB of your session transcript `.jsonl`, parses the newest main-chain (`role=='assistant'`, non-sidechain, non-api-error) `.message.usage`, and sums `input_tokens + cache_creation_input_tokens + cache_read_input_tokens`. When that total crosses `CLAUDE_CONTEXT_WARN_TOKENS` (default `450000`), a one-shot reminder tells Claude to invoke the `prep-compact` skill. The skill surveys the session in four buckets (goal+next, source-of-truth files, decisions+constraints+blockers, execution state) and emits a copy-paste `/compact <mini-schema>` block preserving what the post-compact session needs to resume correctly.
+> A `UserPromptSubmit` hook fires on every prompt submission. It tail-reads the last 256 KB of your session transcript `.jsonl`, parses the newest main-chain (`role=='assistant'`, non-sidechain, non-api-error) `.message.usage`, and sums `input_tokens + cache_creation_input_tokens + cache_read_input_tokens`. When that total crosses `CLAUDE_CONTEXT_WARN_TOKENS` (default `450000`), an informational reminder names the warm handoff path and points the user at `/prep-compact:prep-compact`. The skill reads the warm handoff (extractive fields: cumulative file paths, recent user-message quotes, in-progress todos, recent Task launches) and adds an analytical layer (decisions, constraints, blockers, verb-anchored next-step) to emit a copy-paste `/compact <mini-schema>` block preserving what the post-compact session needs to resume correctly.
 
 The reminder fires once per threshold-crossing. Once the token count drops back below the threshold (after you `/compact`), the flag is auto-cleared on the next turn and future crossings re-arm cleanly. You can also invoke `/prep-compact` manually at any time to refresh the draft right before running `/compact`.
 
+### What's new in v3.0
+
+Between turns, a Stop hook tail-reads the transcript and writes a continuously-updated handoff file at `${CLAUDE_PLUGIN_DATA}/handoff-<sid>.json`. The handoff lists every file the session has touched (cumulative across `/compact` cycles), the most recent user requests quoted verbatim, in-progress todo items, and any active subagent launches. When you run `/prep-compact:prep-compact`, the skill reads this warm file and adds an analytical layer (decisions, constraints, blockers, verb-anchored next-step) — no fresh survey needed. The threshold reminder is now informational; it names the handoff path rather than telling Claude to auto-invoke the skill.
+
+## Honest scope
+
+prep-compact does not replace Claude Code's `/compact` algorithm. Claude Code still owns compaction and may summarize, paraphrase, or omit any instructions passed in. The warm handoff file is durable local source material so `/prep-compact` can generate better `/compact <instructions>`. This is pi-inspired sidecar ergonomics; it does not provide runtime-level verbatim tail retention, non-destructive history navigation, or pi-style `firstKeptEntryId` boundary semantics. If you want those, run [Codex CLI](https://github.com/openai/codex) or [pi-mono](https://github.com/badlogic/pi-mono) — different tools for different runtimes.
+
 ## Install
 
 ```bash
@@ -55,8 +63,8 @@ See [PRIVACY.md](PRIVACY.md) for the full statement.
 ## Known limits
 
 - **Undocumented transcript format.** The hook parses `.message.usage` from the transcript `.jsonl`, which Anthropic doesn't officially document. Silent no-op if the schema changes.
-- **Auto-invoke is prompt-layer.** The reminder tells Claude to invoke the skill; that's best-effort prompt steering. If the skill doesn't auto-run, type `/prep-compact` manually.
-- **Staleness after work.** If you keep working for several turns after the reminder fires, the drafted `/compact` block will be stale by compact-time. Re-invoke `/prep-compact` right before running `/compact` to refresh.
+- **Manual invocation.** The reminder is informational — it names the warm handoff path and points at `/prep-compact:prep-compact`. Claude does not auto-invoke the skill; type `/prep-compact` manually when you're ready to compact.
+- **Staleness across turns.** The Stop hook refreshes the warm handoff after every assistant message, so `/prep-compact` reads current state. If the conversation has been idle and the handoff has been updated since the last user prompt, the draft will reflect that. There's a one-turn window of stale handoff right after `/compact` runs (UserPromptSubmit fires before the next Stop), but the next assistant turn refreshes it.
 
 ## License
 

hooks/check-context-size.sh

@@ -1,10 +1,14 @@
 #!/usr/bin/env bash
-# UserPromptSubmit hook for prep-compact v2.0.0.
+# UserPromptSubmit hook for prep-compact v3.0.
 # Tail-scans the session transcript (last 256 KB) for the newest main-chain
 # assistant .message.usage block. When the sum of input_tokens +
 # cache_creation_input_tokens + cache_read_input_tokens exceeds
-# CLAUDE_CONTEXT_WARN_TOKENS, emits a one-shot reminder telling Claude to
-# invoke the prep-compact skill. Always exits 0 (fail-open).
+# CLAUDE_CONTEXT_WARN_TOKENS, emits an informational reminder. If the warm
+# handoff file exists at $CACHE_DIR/handoff-$SAFE_SID.json (maintained by the
+# Stop hook in update-handoff.sh), the reminder names its path and tells the
+# user to run /prep-compact:prep-compact when ready. Otherwise the reminder
+# falls back to a shorter copy that just names the skill. Always exits 0
+# (fail-open).
 #
 # Main-chain filter: role == 'assistant', isSidechain != true,
 # isApiErrorMessage != true. input_tokens required; cache fields default to 0.
@@ -152,6 +156,11 @@ fi
 
 : >"$FLAG"
 
-printf 'Session context is approximately %s tokens (above configured threshold of %s tokens). Invoke the prep-compact skill to generate a tailored /compact <instructions> command for the user. If you are at the very end of a todo list, you may finish the remaining items first before invoking the skill.\n' "$TOKENS" "$THRESHOLD"
+HANDOFF_PATH="$CACHE_DIR/handoff-$SAFE_SID.json"
+if [[ -e "$HANDOFF_PATH" ]]; then
+  printf 'Session context is approximately %s tokens (above configured threshold of %s tokens). The on-disk handoff at %s is current. When the user is ready to compact, run /prep-compact:prep-compact to add the analytical layer (decisions, constraints, blockers, verb-anchored next-step) and emit a tailored /compact <instructions> block. If you are at the very end of a todo list, you may finish the remaining items first.\n' "$TOKENS" "$THRESHOLD" "$HANDOFF_PATH"
+else
+  printf 'Session context is approximately %s tokens (above configured threshold of %s tokens). Run /prep-compact:prep-compact to survey current state and emit a tailored /compact <instructions> block. If you are at the very end of a todo list, you may finish the remaining items first.\n' "$TOKENS" "$THRESHOLD"
+fi
 
 exit 0

hooks/hooks.json

@@ -3,10 +3,14 @@
     "UserPromptSubmit": [
       {
         "hooks": [
-          {
-            "type": "command",
-            "command": "bash \"${CLAUDE_PLUGIN_ROOT}/hooks/check-context-size.sh\""
-          }
+          { "type": "command", "command": "bash \"${CLAUDE_PLUGIN_ROOT}/hooks/check-context-size.sh\"" }
+        ]
+      }
+    ],
+    "Stop": [
+      {
+        "hooks": [
+          { "type": "command", "command": "bash \"${CLAUDE_PLUGIN_ROOT}/hooks/update-handoff.sh\"" }
         ]
       }
     ]