feat(server): /v1/messages/count_tokens forwarding on the Rust server

[sabhatinas]

What

Adds POST /v1/messages/count_tokens to the Rust switchyard-server. Claude Code calls it to size a request before sending; without it the endpoint 503'd.

How

count_tokens is a direct passthrough, not a routed call. Completions run the router; count_tokens does not:

endpoint	behavior
/v1/messages (completion)	Routes normally — classifier cascade, StageClassifier, escalate/abstain logic
/v1/messages/count_tokens	Direct passthrough — forwards straight to the route's Anthropic target's /v1/messages/count_tokens, no routing

Token counting is a pre-flight estimate with no routing decision, so running the classifier cascade for it is wrong (on a stage router a signal-less request would abstain, or fire a billable judge call). Instead:

• Algorithm::count_tokens(request) forwards via count_tokens_client() — the route's first Anthropic-capable target.
• RoutedLlmClient::count_tokens(request) forwards to that client's Anthropic backend, restamping the model to the upstream id. It shares the encode/POST helper (send_encoded) with the completion path.
• No Anthropic target in the route → 400.

Caveat (deferred): for a route with multiple Anthropic tiers, "first" is arbitrary — choosing which tier count_tokens should reflect is left for later.

Live snapshots

Run against real NVIDIA Anthropic (claude-opus-4-8 @ inference-api) through switchyard-server on a single-target route:

request	response
user: "hi"	{"input_tokens": 8} · 200
user: "count these tokens please" (+ system)	{"input_tokens": 19} · 200
longer user prompt (~1 paragraph)	{"input_tokens": 44} · 200
system + 1 user msg + 1 tool (Claude Code shape)	{"input_tokens": 370} · 200
/v1/messages completion on the same route	200

Counts scale with content and include system + tools, matching Anthropic's own count_tokens.

Stage router

A unit test confirms the fix: on an efficient-first stage router, a completion abstains (no signals to route on) but count_tokens succeeds — it passes through to the strong (Anthropic) tier and returns a count.

Deferred to a follow-up

The Python server keeps its current forwarder for now. Making it symmetric means adding count_tokens to the LlmBackend stack (switchyard-core / switchyard-components) and reaching the backend through the chain. Tracked off SWITCH-1048.

Testing

231 unit/integration tests green (incl. the stage-router passthrough test and the shared send_encoded path), clippy clean, plus the live checks above.

Linear: SWITCH-1048

[coderabbitai]

Walkthrough

Changes

Anthropic count-tokens flow

Layer / File(s)	Summary
Anthropic target capture switchyard/lib/route_table.py, switchyard/cli/route_bundle.py	RouteTable stores the first Anthropic target discovered while expanding supported route types.
Count-tokens upstream forwarding switchyard/lib/endpoints/anthropic_messages_endpoint.py	Adds POST /v1/messages/count_tokens, rewrites the model, forwards the request upstream, and returns JSON or error responses.
Count-tokens endpoint validation tests/test_anthropic_count_tokens_endpoint.py	Tests successful forwarding and the missing-Anthropic-target response.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Poem

I’m a rabbit with routes in my pack,
Sending token-count requests upstream and back.
Anthropic targets now hop into view,
Headers and models follow them too.
Tests twitch their noses: the flow works true!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes the new /v1/messages/count_tokens forwarding feature, which matches the main change.

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/test_anthropic_count_tokens_endpoint.py`:
- Around line 20-38: Replace the _CapturingClient fake and synchronous
TestClient usage with an async ASGI test using httpx.AsyncClient configured with
ASGITransport. Mock the outbound count-tokens request through respx, then assert
the routed request’s URL, JSON payload, and headers to preserve end-to-end
request-shape coverage.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 0f834f53-e4c0-4c21-9e57-1f542dd5a033

📥 Commits

Reviewing files that changed from the base of the PR and between 5b7f4ab and bf4ae62.

📒 Files selected for processing (4)

• switchyard/cli/route_bundle.py
• switchyard/lib/endpoints/anthropic_messages_endpoint.py
• switchyard/lib/route_table.py
• tests/test_anthropic_count_tokens_endpoint.py

[grahamking]

A routing chain can't count tokens, so the handler forwards the request to the route's Anthropic tier and returns its answer. The full target (base_url + key) is gone by request time, so the config capture grabs the first Anthropic-format target at build time and the endpoint forwards to it — swapping the route name for the real model and sending x-api-key + anthropic-version. No hardcoded keys. No Anthropic tier → clean 400.

@sabhatinas Can you re-write that? I think that's AI stringing words together. I've been squinting at it but it doesn't really make any sense.

[sabhatinas]

A routing chain can't count tokens, so the handler forwards the request to the route's Anthropic tier and returns its answer. The full target (base_url + key) is gone by request time, so the config capture grabs the first Anthropic-format target at build time and the endpoint forwards to it — swapping the route name for the real model and sending x-api-key + anthropic-version. No hardcoded keys. No Anthropic tier → clean 400.

@sabhatinas Can you re-write that? I think that's AI stringing words together. I've been squinting at it but it doesn't really make any sense.

I updated the PR body now, it should be easier to understand now.