feat(docs): Serve the Anthropic Messages API (Claude Code) end to end

[haarchri]

Description of your changes

Enables the Anthropic Messages API (/v1/messages) to work through Modelplane's serving path, and adds a validated, copy-paste example showing how to deploy a model on it and drive it from Claude Code via ANTHROPIC_BASE_URL.

Why bump the endpoint-picker image:
Even for Unified single-pod serving, a replica is fronted by a GAIE InferencePool + endpoint picker (EPP), which sits in the request path as an Envoy ext_proc and parses the request body to score endpoints. At the pinned llm-d-inference-scheduler:v0.8.0 (gateway-api-inference-extension v1.5.0) the EPP's body parser is OpenAI-only: an Anthropic Messages request (messages + max_tokens, no prompt) is dispatched to the completions parser and rejected before it reaches vLLM:

inference error: BadRequest - invalid completions request: must have prompt field at gateway-api-inference-extension@v1.5.0/pkg/epp/handlers/server.go:311

vLLM itself serves /v1/messages fine, the picker was the blocker...

llm-d-inference-scheduler v0.9.0 adds a native Anthropic parser (/v1/messages, tools, messages/count_tokens) and enables all HTTP parsers by default, so Anthropic-shaped requests route through the picker. v0.9.0 also renamed the images (the old llm-d-inference-scheduler / llm-d-routing-sidecar packages stop at v0.8.0); the scheduler/sidecar now ship as llm-d-router-endpoint-picker / llm-d-router-disagg-sidecar. ref: llm-d/llm-d-router#1508 llm-d/llm-d-router#1554

updated the APIGroup from inference.networking.x-k8s.io APIGroup to llm-d.ai APIGroup for EndpointPickerConfig llm-d/llm-d-router#972

{"level":"info","ts":1784399821.526125,"caller":"loader/configloader.go:89","msg":"DEPRECATION: apiVersion inference.networking.x-k8s.io/v1alpha1/EndpointPickerConfig is deprecated","replacement":"llm-d.ai/v1alpha1/EndpointPickerConfig"}

Fixes #283

I have:

• Read and followed Modelplane's contribution process.
• Run nix flake check (or ./nix.sh flake check) and made sure it passes.
• Added or updated tests covering any composition function changes.
• Signed off every commit with git commit -s.

Tested

curl -si http://localhost:8080/ml-team/qwen3-8b/v1/messages \
  -H "Content-Type: application/json" -H "anthropic-version: 2023-06-01" \
  -d '{"model":"qwen","max_tokens":1024,"messages":[{"role":"user","content":"Hi"}]}' | head -30

HTTP/1.1 200 OK
Content-Type: application/json
Date: Sat, 18 Jul 2026 18:52:55 GMT
Server: uvicorn
X-Went-Into-Resp-Headers: true
Transfer-Encoding: chunked

{"id":"chatcmpl-f2e85d29-c592-413a-825a-3a18ce3f161e","type":"message","role":"assistant","content":[{"type":"text","text":"Hello! How can I assist you today? 😊"}],"model":"qwen","stop_reason":"end_turn","usage":{"input_tokens":13,"output_tokens":12}}%

curl http://localhost:8080/ml-team/qwen3-8b/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen","messages":[{"role":"user","content":"Hello!"}],"max_tokens":64}'
{"id":"chatcmpl-c4da9e27-eeaf-42bc-8bb1-92436989d262","object":"chat.completion","created":1784316729,"model":"qwen","choices":[{"index":0,"message":{"role":"assistant","content":"Hello! How can I assist you today? 😊","refusal":null,"annotations":null,"audio":null,"function_call":null,"tool_calls":[],"reasoning":null},"logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null,"routed_experts":null}],"service_tier":null,"system_fingerprint":"vllm-0.23.0-134963ba","usage":{"prompt_tokens":14,"total_tokens":26,"completion_tokens":12,"prompt_tokens_details":null},"prompt_logprobs":null,"prompt_token_ids":null,"prompt_t

ANTHROPIC_BASE_URL="http://localhost:8080/ml-team/qwen3-8b" \
ANTHROPIC_AUTH_TOKEN=dummy \
ANTHROPIC_DEFAULT_OPUS_MODEL=qwen \
ANTHROPIC_DEFAULT_SONNET_MODEL=qwen \
ANTHROPIC_DEFAULT_HAIKU_MODEL=qwen \
CLAUDE_CODE_MAX_OUTPUT_TOKENS=8192 \
claude --debug

[vercel]

@haarchri is attempting to deploy a commit to the upbound Team on Vercel.

A member of the Team first needs to authorize it.

[haarchri]

add opt-in NIX_SH_PORTS so services in the in-container kind cluster can be reached from the host for local testing (dev-only, inert unless set)...

helps with: kubectl -n traefik-system port-forward --address 0.0.0.0 svc/traefik 8080:80 from inside the develop nix shell to get access on the host ,,,

[dennis-upbound]

One gap worth closing on the EPP bump: nothing asserts the image or the EndpointPickerConfig apiVersion, so a wrong tag/registry path or config group passes CI and only surfaces as an EPP crashloop at deploy. Two levels:

• Unit guard (cheap): in the tests that already build the EPP objects (test_backends.py → TestUnifiedMultiPod / TestDisaggregated), assert the EPP Deployment image is _EPP_IMAGE, the decode pod's sidecar is _SIDECAR_IMAGE, and the ConfigMap carries apiVersion: llm-d.ai/v1alpha1. That locks the values so any change is deliberate and reviewed — it catches typos, not compatibility.
• Real validation (for a version bump like this): an e2e that composes a multi-pod (or PrefillDecode) replica on the kind cluster and waits for the -epp Deployment to report Available. That's the only thing that actually catches v0.9.0 rejecting the current plugin config or RBAC group — a unit test can't, since it never runs the image.

Given this bump changes the image name (not just the tag) and the config group, can we do the unit guard here, and ideally the e2e before we rely on v0.9.0.

[haarchri]

@dennis-upbound i was running e2e curls visible ... Do we have more e2e ?

[dennis-upbound]

@dennis-upbound i was running e2e curls visible ... Do we have more e2e ?

sorry I missed your curl comment in the PR

[haarchri]

@dennis-upbound added 2 unittests