fix(coach): strip <think> CoT blocks from LiteRT-LM output before emitting#99
Merged
Conversation
…tting Qwen3 (the default .litertlm model) emits <think>...</think> chain-of-thought before its answer. The Move Coach contract wants only the delivered answer — leaking internal deliberation to the user is a UX bug, and downstream validation (MoveCoachResponseValidator) should judge only the answer, not the reasoning. Adds LitertLmTextGenerator.stripThinkBlocks (companion-level, unit-tested): strips closed <think>...</think> blocks and an unterminated trailing <think>. Applied in generate() before the Token is emitted. Mirrors the Python-side stripping in ferryman's score_litert_outputs.py (defense in depth — either side stripping is sufficient, but both makes a model swap or generation hiccup safe).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
Follow-up to #98, which merged the LiteRT-LM driver + candidates.json fix + the SIGTRAP context-size fix, but not the production-side
<think>chain-of-thought stripping. This PR lands that missing piece.The problem
Qwen3-0.6B-int4 (the default
.litertlmmodel) emits<think>…</think>chain-of-thought blocks before its answer. After #98 merged,LitertLmTextGenerator.generate()emits the full accumulated output unmodified — so the Move Coach panel shows the model's internal deliberation to the user.Two consequences:
MoveCoachResponseValidatorthen judges the CoT, not the delivered answer. A phrase like "does this give check?" inside CoT looks like an invented check claim.ferryman #11's scorer already strips
<think>before scoring, so the published LiteRT scorecard numbers (honesty 100%, faithfulness 50%, piece-type 80%) are honest about the delivered answer. But the production app still leaks CoT. This closes that gap.The fix
LitertLmTextGenerator.stripThinkBlocks(text)(companion-level, unit-tested):<think>…</think>blocks (case-insensitive, DOTALL)<think>…(generation cut off mid-reasoning)generate()before theTokenis emittedDefense in depth with ferryman's Python-side stripping — either alone is sufficient, both makes a model swap or generation hiccup safe.
Verification
./gradlew :onDeviceAi:compileKotlinDesktop./gradlew :onDeviceAi:desktopTest --tests *StripThinkTestScope
One file changed (
LitertLmTextGenerator.kt, +30/-1), one test added (6 cases). Cherry-picked from theeval/litert-faithfulness-bypassbranch where it was originally written alongside #98's foundation work.