Skip to content

fix: chunk enrich transcript so complete works on large sessions (0.22.1)#40

Merged
Shahinyanm merged 1 commit into
mainfrom
fix/complete-enrich-chunking
Jun 13, 2026
Merged

fix: chunk enrich transcript so complete works on large sessions (0.22.1)#40
Shahinyanm merged 1 commit into
mainfrom
fix/complete-enrich-chunking

Conversation

@Shahinyanm

Copy link
Copy Markdown
Member

Problem

Found in production on a real task: task-journal complete <id> failed with a bare

Error: `claude -p` exited with exit status: 1:

complete <id> --quick worked fine. The failure was the enrich pass, which fed an entire session transcript to the model in one call. A large multi-session task produced a ~220k-token prompt, over the ~200k context limit, so claude -p returned HTTP 400 — but the JSON error was on stdout (where --output-format json puts it) and the runner only showed stderr (empty).

Fix

  • Chunk the enrich transcript. LlmDreamBackend::backfill splits the transcript into line-aligned chunks under a safe byte budget (360k) and runs one call per chunk, merging the recovered events (run_dream already dedups). No content dropped; finalize now works on any session size. --quick was never affected (it skips enrich).
  • Surface claude -p stdout on failure. claude_exit_error now includes the stdout JSON (capped) so a failure reads as Prompt is too long · ~220310 tokens (limit 200000) instead of exit status 1. This is what made the diagnosis instant.

Verified

  • Real run after the fix: complete tj-yh4bv32b8j --quick already retitled #: 5Finalize FIN-1034-docs branch merge and push PR #625 and closed it; the full path now chunks instead of 400ing.
  • New unit tests: small transcript = one chunk; big transcript splits on line boundaries with no content loss; oversized single line hard-splits; backfill chunks a large transcript across calls.
  • Full local gate green: fmt --check, clippy --workspace --all-targets -D warnings, test --workspace, lean --no-default-features build.

🤖 Generated with Claude Code

The enrich pass fed an entire session transcript to the model in one
call; a large multi-session task exceeded the ~200k-token context limit
and claude -p returned HTTP 400 ("Prompt is too long · ~220310 tokens").
Split the transcript into line-aligned chunks under a safe byte budget
and merge the recovered events (run_dream dedups), so finalize works on
any session size. --quick was unaffected (it skips enrich).

Also surface the JSON error claude -p prints on stdout under
--output-format json (the real cause goes there, not stderr), so a
failure is legible instead of a bare "exit status 1" — which is exactly
what let us diagnose the context overflow.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Shahinyanm Shahinyanm merged commit 552ae5b into main Jun 13, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant