Skip to content

feat(classifier): subscription-native agent-sdk backend via Haiku (Track C) #24

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Shahinyanm merged 1 commit into from
Jun 11, 2026
Merged

feat(classifier): subscription-native agent-sdk backend via Haiku (Track C) #24

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added
- **`agent-sdk` classifier backend** — subscription-native LLM classification
via the local, already-authenticated `claude` binary, no `ANTHROPIC_API_KEY`
required. `tj_core::classifier::agent_sdk::ClaudeCliClassifier` invokes
`claude -p <prompt> --model claude-haiku-4-5 --output-format json
--strict-mcp-config`, parses the JSON envelope's `result`, and reuses the
shared verdict parser. Command execution is injected via a `CommandRunner`
trait so the path is unit-testable without shelling out. `from_env()` returns
`None` unless `claude` is on PATH; model overridable with `TJ_AGENT_SDK_MODEL`.
- Wired into `--backend` selection (`ingest-hook`, `classify-worker`) and
added to `install-hooks --backend`, alongside `hybrid` | `api` | `heuristic`.
- **Hybrid fallback is now an ordered chain**: heuristic (≥ 0.7) → `agent-sdk`
(if `claude` on PATH) → `api` (if key) → `pending/`. Reorder via
`TJ_HYBRID_LLM_ORDER` (default `agent-sdk,api`) to prefer the API key.
- **Honest cost note**: since **2026-06-15** a headless `claude -p` run draws
from the separate Agent SDK monthly credit pool (~$20 Pro / $100 Max 5x /
$200 Max 20x at API rates), not the interactive pool. Documented in the
README, `--backend` help, and `doctor`.

## [0.12.0]

### Added
Expand Down
28 changes: 25 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ task-journal pack tj-x9rz1f --mode full
| `doctor` | Self-check the install |
| `rebuild-state` | Rebuild SQLite from JSONL |
| `migrate-project` | Re-key data when a project moves on disk |
| `install-hooks [--scope user\|project]` | Wire Claude Code auto-capture hooks |
| `install-hooks [--scope user\|project] [--backend hybrid\|agent-sdk\|api\|heuristic]` | Wire Claude Code auto-capture hooks |

## MCP tools

Expand All @@ -181,10 +181,32 @@ The MCP server exposes five tools to Claude Code (and any MCP client):

## Configuration

### Classifier backends

The auto-capture classifier (a best-effort backstop — explicit self-tagging via the
MCP tools is the primary path) has a heuristic stage plus an optional LLM stage. The
LLM stage has **two** ways to reach a model, pick via `--backend` on `install-hooks`
or `ingest-hook`:

- **`agent-sdk`** — classify via the local, already-logged-in `claude` binary. **No
`ANTHROPIC_API_KEY` needed** — it rides your Claude subscription. Pinned to Haiku.
⚠️ Since **2026-06-15** a headless `claude -p` run draws from the separate **Agent
SDK** monthly credit pool (~$20 Pro / $100 Max 5x / $200 Max 20x, at API rates),
not the interactive pool. Classification is Haiku-class and tiny (a few hundred
tokens per chunk), so the credit lasts a long time — but it is not strictly free.
- **`api`** — call the Anthropic API directly. Requires `ANTHROPIC_API_KEY`.

`--backend=hybrid` (the default) runs the heuristic first, then falls through the LLM
chain `agent-sdk → api`, using whichever backends are available. Reorder the chain
with `TJ_HYBRID_LLM_ORDER` (e.g. `api,agent-sdk` to prefer the API key). With no LLM
backend available, the heuristic still runs and ambiguous chunks queue in `pending/`.

| Env var | Effect | Default |
|---------|--------|---------|
| `ANTHROPIC_API_KEY` | Powers the API stage of `--backend=hybrid` (default) and is required for `--backend=api`. Without it, only the offline heuristic runs and ambiguous chunks land in the local pending queue. | _unset_ |
| `TJ_CLASSIFIER_MODEL` | Override the Anthropic model used by the API stage. | `claude-haiku-4-5-20251001` |
| `ANTHROPIC_API_KEY` | Enables the `api` LLM backend (and the `api` link of the hybrid chain). Optional — the `agent-sdk` backend needs no key. | _unset_ |
| `TJ_AGENT_SDK_MODEL` | Override the model the `agent-sdk` backend passes to `claude --model`. | `claude-haiku-4-5` |
| `TJ_HYBRID_LLM_ORDER` | Comma-separated fallback order for `--backend=hybrid`. | `agent-sdk,api` |
| `TJ_CLASSIFIER_MODEL` | Override the Anthropic model used by the `api` backend. | `claude-haiku-4-5-20251001` |
| `TJ_AUTO_OPEN_TASKS` | Set to `0` / `false` to disable auto-opening a task from `UserPromptSubmit` when no open task exists. | `1` |

## Event types
Expand Down
83 changes: 64 additions & 19 deletions crates/tj-cli/src/main.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -506,9 +506,10 @@ fn run_doctor() -> Result<DoctorReport> {
}
Ok(_) | Err(_) => {
notes.push(
"claude CLI not on PATH — that's fine if you use the API backend \
(set ANTHROPIC_API_KEY). For the CLI backend (free with Pro/Max), \
install Claude Code from https://claude.com/claude-code"
"claude CLI not on PATH — that's fine if you use the `api` backend \
(set ANTHROPIC_API_KEY). For the `agent-sdk` backend (no API key; \
uses your Claude login, drawing the Agent SDK credit pool since \
2026-06-15), install Claude Code from https://claude.com/claude-code"
.into(),
);
(false, None)
Expand Down Expand Up @@ -736,6 +737,12 @@ enum Commands {
/// `task-journal backfill` afterwards. Onboarding shortcut.
#[arg(long)]
backfill: bool,
/// Classifier backend baked into the installed hook command:
/// "hybrid" (default), "agent-sdk", "api", or "heuristic". Use
/// "agent-sdk" to classify via the local `claude` login without an
/// ANTHROPIC_API_KEY (see `ingest-hook --help` for the credit note).
#[arg(long, default_value = "hybrid")]
backend: String,
},
/// Show local classifier and journal statistics.
Stats,
Expand Down Expand Up @@ -838,13 +845,18 @@ enum Commands {
#[arg(long)]
text: Option<String>,
/// Classifier backend:
/// - "hybrid" (default) — keyword heuristic first (free, offline);
/// Anthropic API fallback when uncertain (needs ANTHROPIC_API_KEY).
/// - "api" — always call the Anthropic API. Best quality, paid.
/// - "hybrid" (default) — keyword heuristic first (free, offline),
/// then the configured LLM fallback chain (agent-sdk, then api;
/// reorder with TJ_HYBRID_LLM_ORDER). Only available backends run.
/// - "agent-sdk" — classify via the local, already-logged-in `claude`
/// binary; no ANTHROPIC_API_KEY needed. Pinned to Haiku (override
/// with TJ_AGENT_SDK_MODEL). NOTE: since 2026-06-15 a headless
/// `claude -p` draws from the separate Agent SDK monthly credit
/// pool (~$20 Pro / $100 Max 5x / $200 Max 20x at API rates), not
/// the interactive pool. Classification is tiny, so it lasts.
/// - "api" — always call the Anthropic API. Needs ANTHROPIC_API_KEY.
/// - "heuristic" — heuristic only, no LLM. Fastest, lowest coverage.
/// - "cli" — deprecated alias for hybrid. The old `claude -p` path
/// was removed in v0.8.0 because Anthropic now bills it
/// separately from Pro/Max.
/// - "cli" — removed in v0.8.0; use "agent-sdk" (its resurrection).
#[arg(long, default_value = "hybrid")]
backend: String,
/// Test/dev override: bypass classifier and force this event type. Hidden from --help.
Expand All @@ -864,7 +876,8 @@ enum Commands {
/// at a time. Hidden from --help; not a public API.
#[command(hide = true)]
ClassifyWorker {
/// Classifier backend: "hybrid", "api", or "heuristic". Defaults to hybrid.
/// Classifier backend: "hybrid", "agent-sdk", "api", or "heuristic".
/// Defaults to hybrid.
#[arg(long, default_value = "hybrid")]
backend: String,
},
Expand Down Expand Up @@ -1378,6 +1391,7 @@ fn main() -> Result<()> {
scope,
uninstall,
backfill,
backend,
} => {
let settings_path = match scope.as_str() {
"user" => {
Expand Down Expand Up @@ -1473,11 +1487,25 @@ fn main() -> Result<()> {
// at env vars Claude Code never sets and therefore always
// fed the classifier empty text. Stdin-only is the correct
// wiring (see claude-memory-rsw).
// No --backend flag: the binary's default (hybrid) wins.
// Hybrid = free heuristic first, Anthropic API fallback when
// uncertain. Users wanting always-api can edit settings.json
// and add `--backend=api`.
let cmd = "task-journal ingest-hook || true";
// Bake the selected backend into the hook command. Default
// "hybrid" stays flag-free (heuristic first, then the agent-sdk
// → api fallback chain). A non-default backend — e.g.
// `--backend=agent-sdk` for subscription users with no API key
// — is passed through so the spawned classify-worker honors it.
if !matches!(
backend.as_str(),
"hybrid" | "agent-sdk" | "api" | "heuristic"
) {
anyhow::bail!(
"unknown --backend: {backend} (expected `hybrid`, `agent-sdk`, `api`, or `heuristic`)"
);
}
let cmd_string = if backend == "hybrid" {
"task-journal ingest-hook || true".to_string()
} else {
format!("task-journal ingest-hook --backend={backend} || true")
};
let cmd = cmd_string.as_str();
let entries = serde_json::json!({
"UserPromptSubmit": [{ "matcher": "", "hooks": [{ "type": "command", "command": cmd }] }],
"PostToolUse": [{ "matcher": "", "hooks": [{ "type": "command", "command": cmd }] }],
Expand Down Expand Up @@ -2241,6 +2269,15 @@ fn main() -> Result<()> {
Box::new(tj_core::classifier::hybrid::HybridClassifier::from_env())
}
"api" => Box::new(tj_core::classifier::http::AnthropicClassifier::from_env()?),
"agent-sdk" => Box::new(
tj_core::classifier::agent_sdk::ClaudeCliClassifier::from_env()
.ok_or_else(|| {
anyhow::anyhow!(
"agent-sdk backend selected but no `claude` binary on PATH — \
install Claude Code (https://claude.com/claude-code) or pick another --backend"
)
})?,
),
"heuristic" => {
// Heuristic-only: no LLM at all. Trades coverage
// for absolute zero-cost / offline operation.
Expand All @@ -2262,7 +2299,7 @@ fn main() -> Result<()> {
Box::new(HeuristicOnly)
}
other => anyhow::bail!(
"unknown backend: {other} (expected `hybrid`, `api`, or `heuristic`)"
"unknown backend: {other} (expected `hybrid`, `agent-sdk`, `api`, or `heuristic`)"
),
};
let input = tj_core::classifier::ClassifyInput {
Expand Down Expand Up @@ -3859,6 +3896,14 @@ fn process_pending_entry(
let classifier: Box<dyn Classifier> = match backend {
"hybrid" | "" => Box::new(tj_core::classifier::hybrid::HybridClassifier::from_env()),
"api" => Box::new(tj_core::classifier::http::AnthropicClassifier::from_env()?),
"agent-sdk" => Box::new(
tj_core::classifier::agent_sdk::ClaudeCliClassifier::from_env().ok_or_else(|| {
anyhow::anyhow!(
"agent-sdk backend selected but no `claude` binary on PATH — \
install Claude Code (https://claude.com/claude-code) or pick another --backend"
)
})?,
),
"heuristic" => {
use tj_core::classifier::heuristic::try_heuristic;
use tj_core::classifier::{ClassifyInput, ClassifyOutput};
Expand All @@ -3874,9 +3919,9 @@ fn process_pending_entry(
}
Box::new(HeuristicOnly)
}
other => {
anyhow::bail!("unknown backend: {other} (expected `hybrid`, `api`, or `heuristic`)")
}
other => anyhow::bail!(
"unknown backend: {other} (expected `hybrid`, `agent-sdk`, `api`, or `heuristic`)"
),
};
let input = tj_core::classifier::ClassifyInput {
text: text.clone(),
Expand Down
Loading
Loading