fix(gradient): deterministic timestamps in sanitizeToolParts#124
Merged
Conversation
…revent cache busts Replace Date.now() with the tool part's existing start time (or 0 for pending parts without a time field) in sanitizeToolParts(). This ensures repeated transform() calls on the same stale pending tool parts produce identical bytes, preserving Anthropic's prompt cache. Root cause: OpenCode's prompt-loop cache fix (e148f00aa) preserves old pending tool parts in the cached message array across iterations. Lore's sanitizeToolParts() re-converts these pending→error with a fresh Date.now() each call, producing different serialized bytes on every API call and busting the prompt cache. Impact: Eliminates ~196 non-distillation cache busts per long session ($353 in cache-write cost on the analyzed CLI-1AE session with 1,733 API calls).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Date.now()with the tool part's existing start time (or0for pending parts) insanitizeToolParts()to ensure repeatedtransform()calls produce identical bytes196 non-distillation cache busts per long session ($353 savings on analyzed session)Root Cause
OpenCode's prompt-loop cache fix (
e148f00aa) preserves old pending tool parts in the cached message array across prompt loop iterations. Eachtransform()call, Lore'ssanitizeToolParts()re-converts these stale pending parts to error state usingDate.now()— producing different serialized bytes on every API call and invalidating Anthropic's prompt cache from that position forward.All 196 non-distillation busts in session CLI-1AE (1,733 API calls, $1,090 total) followed a
tool-callsfinish reason and showedcache_read ≈ 41K(only system prompt survived). 154 of 196 were consecutive bust chains.Fix
Use
part.state.time.startfor running parts (which already have a timestamp) and0for pending parts (which have no time field). Both are deterministic across repeated calls on the same input.Combined Impact (with PR #123)
On the analyzed CLI-1AE session:
Cache Control Investigation
Also investigated whether explicit Anthropic
cache_controlbreakpoints could help. Findings:experimental.chat.messages.transformhook operates onMessageV2.WithParts[]before cache annotations are added — cannot injectcache_controlfrom pluginsTests
sanitizeToolParts determinismtest verifying consecutivetransform()calls produce byte-identical output for stale pending tool parts