Skip to content

fix(gradient): deterministic timestamps in sanitizeToolParts#124

Merged
BYK merged 1 commit intomainfrom
fix/deterministic-sanitize-timestamps
May 6, 2026
Merged

fix(gradient): deterministic timestamps in sanitizeToolParts#124
BYK merged 1 commit intomainfrom
fix/deterministic-sanitize-timestamps

Conversation

@BYK
Copy link
Copy Markdown
Owner

@BYK BYK commented May 6, 2026

Summary

  • Replace Date.now() with the tool part's existing start time (or 0 for pending parts) in sanitizeToolParts() to ensure repeated transform() calls produce identical bytes
  • Eliminates 196 non-distillation cache busts per long session ($353 savings on analyzed session)

Root Cause

OpenCode's prompt-loop cache fix (e148f00aa) preserves old pending tool parts in the cached message array across prompt loop iterations. Each transform() call, Lore's sanitizeToolParts() re-converts these stale pending parts to error state using Date.now() — producing different serialized bytes on every API call and invalidating Anthropic's prompt cache from that position forward.

All 196 non-distillation busts in session CLI-1AE (1,733 API calls, $1,090 total) followed a tool-calls finish reason and showed cache_read ≈ 41K (only system prompt survived). 154 of 196 were consecutive bust chains.

Fix

Use part.state.time.start for running parts (which already have a timestamp) and 0 for pending parts (which have no time field). Both are deterministic across repeated calls on the same input.

Combined Impact (with PR #123)

On the analyzed CLI-1AE session:

Cache Control Investigation

Also investigated whether explicit Anthropic cache_control breakpoints could help. Findings:

  • OpenCode already uses all 4 available breakpoints (2 system + 2 trailing messages)
  • The experimental.chat.messages.transform hook operates on MessageV2.WithParts[] before cache annotations are added — cannot inject cache_control from plugins
  • The 20-block lookback window works correctly as long as prefix bytes are deterministic, which this fix ensures
  • No action needed on cache_control — the existing setup is optimal

Tests

  • Added sanitizeToolParts determinism test verifying consecutive transform() calls produce byte-identical output for stale pending tool parts
  • Full suite: 686 pass, 0 fail

…revent cache busts

Replace Date.now() with the tool part's existing start time (or 0 for
pending parts without a time field) in sanitizeToolParts(). This ensures
repeated transform() calls on the same stale pending tool parts produce
identical bytes, preserving Anthropic's prompt cache.

Root cause: OpenCode's prompt-loop cache fix (e148f00aa) preserves old
pending tool parts in the cached message array across iterations. Lore's
sanitizeToolParts() re-converts these pending→error with a fresh
Date.now() each call, producing different serialized bytes on every API
call and busting the prompt cache.

Impact: Eliminates ~196 non-distillation cache busts per long session
($353 in cache-write cost on the analyzed CLI-1AE session with 1,733
API calls).
@BYK BYK enabled auto-merge (squash) May 6, 2026 09:01
@BYK BYK merged commit 78b67f5 into main May 6, 2026
1 check passed
@BYK BYK deleted the fix/deterministic-sanitize-timestamps branch May 6, 2026 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant