feat(classifier): subscription-native agent-sdk backend via Haiku (Track C)#24
Merged
Merged
Conversation
Add a CLI-backed classifier that reaches an LLM through the local, already-authenticated `claude` binary — no ANTHROPIC_API_KEY needed. Resurrects the v0.7.x cli path that was dropped in v0.8.0, this time documented honestly: since 2026-06-15 a headless `claude -p` draws from the separate Agent SDK credit pool, not the interactive Pro/Max pool. - ClaudeCliClassifier (classifier/agent_sdk.rs) runs `claude -p <prompt> --model claude-haiku-4-5 --output-format json --strict-mcp-config`, parses the envelope's `result`, reuses the shared parse_verdict. Command execution injected via a CommandRunner trait so the parse path is unit-testable without shelling out. from_env() returns None unless `claude` is on PATH; model overridable via TJ_AGENT_SDK_MODEL. - Factor http.rs's fence-strip+serde parse into classifier::parse_verdict, reused by both http and agent_sdk so they never diverge. - Hybrid is now an ordered LLM chain: heuristic >= 0.7 -> agent-sdk (if claude on PATH) -> api (if key) -> pending/. Reorder via TJ_HYBRID_LLM_ORDER (default agent-sdk,api). - Wire `agent-sdk` into --backend for ingest-hook and classify-worker, and add --backend to install-hooks (baked into the generated hook command). - README Configuration + doctor note + --backend help document the backend and the Agent SDK credit caveat. Tests: agent_sdk parse/threshold/fence/error (fake runner, no live claude); hybrid proves uncertain -> agent-sdk wins and http is never touched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Gives subscription users without an
ANTHROPIC_API_KEYa real LLM classifier. Previously, uncertain chunks just queued inpending/forever for them. This adds anagent-sdkbackend that classifies through the local, already-logged-inclaudebinary (pinned to Haiku) — riding the Claude subscription instead of an API key.This resurrects the v0.7.x
clipath that was dropped in v0.8.0, documented honestly this time.Changes
ClaudeCliClassifier(crates/tj-core/src/classifier/agent_sdk.rs) runsclaude -p <prompt> --model claude-haiku-4-5 --output-format json --strict-mcp-config, parses the JSON envelope'sresult, and reuses a shared verdict parser. Command execution is injected via aCommandRunnertrait, so the parse path is unit-tested without ever shelling out.from_env()returnsNoneunlessclaudeis on PATH; model overridable viaTJ_AGENT_SDK_MODEL.classifier::parse_verdict— factored http.rs's fence-strip + serde parse into one helper used by bothhttpandagent_sdkso they can't diverge.agent-sdk(ifclaudeon PATH) →api(if key) →pending/. Reorder withTJ_HYBRID_LLM_ORDER(defaultagent-sdk,api).agent-sdkadded to--backendforingest-hookandclassify-worker;install-hooksgains a--backendflag that bakes the choice into the generated hook command.--backendhelp, anddoctorall note that since 2026-06-15 a headlessclaude -pdraws from the separate Agent SDK monthly credit pool (~$20 Pro / $100 Max 5x / $200 Max 20x at API rates), not the interactive pool. Classification is Haiku-class and tiny, so it lasts — but it's not strictly free.Verification
cargo fmt --all --checkclean;cargo clippy --workspace --all-targetscleancargo test --workspacegreen;cargo test classifier::32/32 pass:decide_statusat the 0.85 threshold, code-fence tolerance,is_error→Errparse_verdictrefactorclaude -p ... --output-format jsonreturns valid{is_error:false,result:...}on a subscription with noANTHROPIC_API_KEY(EXIT=0) — confirms the flags before hardcoding.Flags were verified live against the installed
claude(not assumed from memory). No realclaudeinvocation in CI — theCommandRunneris faked.Stacked on Track A (#23) conceptually but branched from
main; merge A first.🤖 Generated with Claude Code