Skip to content

feat(classifier): subscription-native agent-sdk backend via Haiku (Track C)#24

Merged
Shahinyanm merged 1 commit into
mainfrom
feat/track-c-agent-sdk-classifier
Jun 11, 2026
Merged

feat(classifier): subscription-native agent-sdk backend via Haiku (Track C)#24
Shahinyanm merged 1 commit into
mainfrom
feat/track-c-agent-sdk-classifier

Conversation

@Shahinyanm

Copy link
Copy Markdown
Member

What

Gives subscription users without an ANTHROPIC_API_KEY a real LLM classifier. Previously, uncertain chunks just queued in pending/ forever for them. This adds an agent-sdk backend that classifies through the local, already-logged-in claude binary (pinned to Haiku) — riding the Claude subscription instead of an API key.

This resurrects the v0.7.x cli path that was dropped in v0.8.0, documented honestly this time.

Changes

  • ClaudeCliClassifier (crates/tj-core/src/classifier/agent_sdk.rs) runs claude -p <prompt> --model claude-haiku-4-5 --output-format json --strict-mcp-config, parses the JSON envelope's result, and reuses a shared verdict parser. Command execution is injected via a CommandRunner trait, so the parse path is unit-tested without ever shelling out. from_env() returns None unless claude is on PATH; model overridable via TJ_AGENT_SDK_MODEL.
  • Shared classifier::parse_verdict — factored http.rs's fence-strip + serde parse into one helper used by both http and agent_sdk so they can't diverge.
  • Hybrid is now an ordered LLM chain: heuristic (≥ 0.7) → agent-sdk (if claude on PATH) → api (if key) → pending/. Reorder with TJ_HYBRID_LLM_ORDER (default agent-sdk,api).
  • Wiring: agent-sdk added to --backend for ingest-hook and classify-worker; install-hooks gains a --backend flag that bakes the choice into the generated hook command.
  • Honest docs: README "Classifier backends" + env table, --backend help, and doctor all note that since 2026-06-15 a headless claude -p draws from the separate Agent SDK monthly credit pool (~$20 Pro / $100 Max 5x / $200 Max 20x at API rates), not the interactive pool. Classification is Haiku-class and tiny, so it lasts — but it's not strictly free.

Verification

  • cargo fmt --all --check clean; cargo clippy --workspace --all-targets clean
  • cargo test --workspace green; cargo test classifier:: 32/32 pass:
    • agent_sdk: canned-verdict parse, model pin, decide_status at the 0.85 threshold, code-fence tolerance, is_errorErr
    • hybrid: uncertain heuristic + agent-sdk available → uses agent-sdk and never touches http
    • http: both existing tests still pass after the parse_verdict refactor
  • Live: claude -p ... --output-format json returns valid {is_error:false,result:...} on a subscription with no ANTHROPIC_API_KEY (EXIT=0) — confirms the flags before hardcoding.

Flags were verified live against the installed claude (not assumed from memory). No real claude invocation in CI — the CommandRunner is faked.

Stacked on Track A (#23) conceptually but branched from main; merge A first.

🤖 Generated with Claude Code

Add a CLI-backed classifier that reaches an LLM through the local,
already-authenticated `claude` binary — no ANTHROPIC_API_KEY needed.
Resurrects the v0.7.x cli path that was dropped in v0.8.0, this time
documented honestly: since 2026-06-15 a headless `claude -p` draws from
the separate Agent SDK credit pool, not the interactive Pro/Max pool.

- ClaudeCliClassifier (classifier/agent_sdk.rs) runs
  `claude -p <prompt> --model claude-haiku-4-5 --output-format json
  --strict-mcp-config`, parses the envelope's `result`, reuses the shared
  parse_verdict. Command execution injected via a CommandRunner trait so
  the parse path is unit-testable without shelling out. from_env() returns
  None unless `claude` is on PATH; model overridable via TJ_AGENT_SDK_MODEL.
- Factor http.rs's fence-strip+serde parse into classifier::parse_verdict,
  reused by both http and agent_sdk so they never diverge.
- Hybrid is now an ordered LLM chain: heuristic >= 0.7 -> agent-sdk (if
  claude on PATH) -> api (if key) -> pending/. Reorder via
  TJ_HYBRID_LLM_ORDER (default agent-sdk,api).
- Wire `agent-sdk` into --backend for ingest-hook and classify-worker, and
  add --backend to install-hooks (baked into the generated hook command).
- README Configuration + doctor note + --backend help document the backend
  and the Agent SDK credit caveat.

Tests: agent_sdk parse/threshold/fence/error (fake runner, no live claude);
hybrid proves uncertain -> agent-sdk wins and http is never touched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Shahinyanm Shahinyanm merged commit 1da81b7 into main Jun 11, 2026
7 checks passed
@Shahinyanm Shahinyanm deleted the feat/track-c-agent-sdk-classifier branch June 11, 2026 19:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant