feat(infra): enable Claude/Anthropic in the gateway with agent-default alias (ADR 09)#131
Conversation
… (ADR 09)
Enable Anthropic/Claude in the LiteLLM gateway so non-interactive CI agents
can route Claude Code through Sluice on API-key billing, per ADR 09
("API-key billing = through Sluice").
- model_list: add anthropic_models (claude-haiku-4-5 cheap tier,
claude-sonnet-4-6 mid tier) plus an `agent-default` alias group mapping to
the cheap tier as primary (order 1), degrading to the mid tier then Azure
`premium` via a new fallback chain. Gated on has_anthropic; concatenated
into litellm_model_list.
- Secret wiring reuses the existing extra_providers path: an `anthropic`
entry creates the provider-anthropic-key KV secret and injects the
LITELLM_ANTHROPIC_API_KEY env var on the container — no bespoke wiring.
- Provider activation: deploy.yaml + deploy-environment.yaml read an
ANTHROPIC_API_KEY GitHub secret and fold it into TF_VAR_extra_providers as
env_name LITELLM_ANTHROPIC_API_KEY; absent key => provider stays inactive.
- keys.yaml: add a dedicated mystira-ci virtual key allow-listing
agent-default + the Claude models (interactive claude-code-cli stays
direct-Anthropic on the Max subscription).
- ADR 09 flipped to Accepted (2026-07-03); operator go-live steps updated to
the real extra_providers flow.
terraform fmt: clean. terraform validate: passes.
Requires `python scripts/manage_keys.py apply` after deploy.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
📝 WalkthroughWalkthroughThis PR adds Anthropic model provider support to Sluice's routing infrastructure. It wires an optional ChangesAnthropic provider integration
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant DeployWorkflow as deploy.yaml
participant DeployEnvWorkflow as deploy-environment.yaml
participant Terraform as Terraform (main.tf)
participant LiteLLM as LiteLLM Config
DeployWorkflow->>DeployWorkflow: Read ANTHROPIC_API_KEY secret
DeployWorkflow->>DeployWorkflow: Set TF_VAR_extra_providers.anthropic
DeployWorkflow->>DeployEnvWorkflow: Forward ANTHROPIC_API_KEY (deploy-prod)
DeployEnvWorkflow->>Terraform: Apply TF_VAR_extra_providers
Terraform->>LiteLLM: Render anthropic_models + agent-default fallback chain
LiteLLM->>LiteLLM: Resolve agent-default via combined_fallbacks
Compact metadata
Poem 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
Enables Anthropic/Claude routing through the Sluice LiteLLM gateway for non-interactive CI agents via the agent-default alias, aligning implementation and operational guidance with ADR 09’s decision (accepted 2026-07-03).
Changes:
- Adds an Anthropic provider gate in Terraform with Claude model entries plus an
agent-defaultmodel group and dedicated fallback chain. - Wires
ANTHROPIC_API_KEYthrough the existingextra_providersmechanism in deploy workflows to activate the provider without bespoke Terraform plumbing. - Introduces a dedicated
mystira-civirtual key allow-list and updates ADR 09 / prod tfvars documentation.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
scripts/keys.yaml |
Adds mystira-ci virtual key configuration for CI agents and Claude model allow-listing. |
infra/modules/sluice_aca/main.tf |
Adds Anthropic model definitions and agent-default fallback chain; composes fallbacks conditionally. |
infra/env/prod/terraform.tfvars |
Documents how to configure the anthropic entry via extra_providers. |
docs/architecture/09-claude-code-via-sluice.md |
Marks ADR 09 accepted and updates the implementation/setup guidance accordingly. |
.github/workflows/deploy.yaml |
Adds Anthropic secret ingestion into TF_VAR_extra_providers and forwards the secret to the environment deploy workflow. |
.github/workflows/deploy-environment.yaml |
Accepts ANTHROPIC_API_KEY as an optional workflow-call secret and folds it into TF_VAR_extra_providers. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| - alias: mystira-ci | ||
| max_budget: 10 | ||
| budget_duration: 7d | ||
| rpm_limit: 120 | ||
| tpm_limit: 200000 | ||
| models: [agent-default, claude-haiku-4-5, claude-sonnet-4-6, gpt-4o, text-embedding-3-large, auto, cheap-fast, cheap-reasoning, premium] |
| CI consumers then set the two env vars from the per-consumer snippet below, | ||
| using `vkey-mystira-ci`, and request `model="agent-default"`. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/architecture/09-claude-code-via-sluice.md`:
- Around line 166-167: The runbook text mixes CI and interactive Claude Code
guidance, causing conflicting auth/routing instructions. Update the surrounding
section so the CI path in the relevant documentation uses the CI-specific key
and agent-default model, and move or rewrite the Claude Code CLI/Sluice example
under a separate interactive section. Use the existing “CI consumers” guidance
and the Claude Code CLI/Sluice example as the symbols to locate the conflicting
text.
- Around line 153-156: The fenced command block in the documentation is missing
a language identifier, which causes markdownlint violations. Update the shell
snippet in the affected markdown section to use a shell language tag such as
bash or sh, and keep the existing command content unchanged so the fenced block
is correctly labeled.
In `@scripts/keys.yaml`:
- Line 80: The mystira-ci allowlist in keys.yaml is missing cheap-long-context,
which breaks the auto fallback chain when LiteLLM resolves auto through
cheap-fast and cheap-reasoning. Update the models list for mystira-ci to include
cheap-long-context alongside the existing auto-related models so manage_keys.py
can pass a complete allowlist to /key/generate and allow the fallback path used
by infra/modules/sluice_aca/main.tf extra_fallbacks.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: a6809fa7-5048-416c-b5f7-4d13edb2809b
📒 Files selected for processing (6)
.github/workflows/deploy-environment.yaml.github/workflows/deploy.yamldocs/architecture/09-claude-code-via-sluice.mdinfra/env/prod/terraform.tfvarsinfra/modules/sluice_aca/main.tfscripts/keys.yaml
| ``` | ||
| az keyvault secret set --vault-name pvc-prod-sluice-kv \ | ||
| --name anthropic-api-key --value <key> | ||
| ``` |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Label the shell snippet.
The fenced command block at Line 153 is missing a language identifier, which violates the Markdown docs guideline and trips markdownlint. Add bash (or sh) to the fence.
Suggested fix
-```
+```bashAs per coding guidelines, docs/**/*.md: Include code blocks with language identifiers in Markdown documentation.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| ``` | |
| az keyvault secret set --vault-name pvc-prod-sluice-kv \ | |
| --name anthropic-api-key --value <key> | |
| ``` |
🧰 Tools
🪛 markdownlint-cli2 (0.22.1)
[warning] 153-153: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/architecture/09-claude-code-via-sluice.md` around lines 153 - 156, The
fenced command block in the documentation is missing a language identifier,
which causes markdownlint violations. Update the shell snippet in the affected
markdown section to use a shell language tag such as bash or sh, and keep the
existing command content unchanged so the fenced block is correctly labeled.
Sources: Coding guidelines, Linters/SAST tools
| CI consumers then set the two env vars from the per-consumer snippet below, | ||
| using `vkey-mystira-ci`, and request `model="agent-default"`. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Split the CI and interactive paths.
Line 166 says CI consumers should use vkey-mystira-ci, but the section below still hard-codes vkey-claude-code-cli and describes Claude Code CLI routing through Sluice. That conflicts with the decision above that interactive Max-backed Claude Code stays direct-Anthropic, and it will send operators to the wrong auth flow. Please separate the CI runbook from the interactive one, or swap the example key if this section is CI-only.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/architecture/09-claude-code-via-sluice.md` around lines 166 - 167, The
runbook text mixes CI and interactive Claude Code guidance, causing conflicting
auth/routing instructions. Update the surrounding section so the CI path in the
relevant documentation uses the CI-specific key and agent-default model, and
move or rewrite the Claude Code CLI/Sluice example under a separate interactive
section. Use the existing “CI consumers” guidance and the Claude Code CLI/Sluice
example as the symbols to locate the conflicting text.
| budget_duration: 7d | ||
| rpm_limit: 120 | ||
| tpm_limit: 200000 | ||
| models: [agent-default, claude-haiku-4-5, claude-sonnet-4-6, gpt-4o, text-embedding-3-large, auto, cheap-fast, cheap-reasoning, premium] |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Missing cheap-long-context in the mystira-ci allowlist breaks the auto fallback chain.
mystira-ci includes auto in its models list, but the auto fallback chain (infra/modules/sluice_aca/main.tf, extra_fallbacks) degrades auto → cheap-fast → cheap-reasoning → cheap-long-context → premium. Every other key with auto in this file also lists cheap-long-context; this one omits it. Since manage_keys.py sends models straight to LiteLLM's /key/generate as a key-level allowlist, a fallback landing on cheap-long-context for this key would be rejected.
🔧 Proposed fix
- models: [agent-default, claude-haiku-4-5, claude-sonnet-4-6, gpt-4o, text-embedding-3-large, auto, cheap-fast, cheap-reasoning, premium]
+ models: [agent-default, claude-haiku-4-5, claude-sonnet-4-6, gpt-4o, text-embedding-3-large, auto, cheap-fast, cheap-reasoning, cheap-long-context, premium]📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| models: [agent-default, claude-haiku-4-5, claude-sonnet-4-6, gpt-4o, text-embedding-3-large, auto, cheap-fast, cheap-reasoning, premium] | |
| models: [agent-default, claude-haiku-4-5, claude-sonnet-4-6, gpt-4o, text-embedding-3-large, auto, cheap-fast, cheap-reasoning, cheap-long-context, premium] |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/keys.yaml` at line 80, The mystira-ci allowlist in keys.yaml is
missing cheap-long-context, which breaks the auto fallback chain when LiteLLM
resolves auto through cheap-fast and cheap-reasoning. Update the models list for
mystira-ci to include cheap-long-context alongside the existing auto-related
models so manage_keys.py can pass a complete allowlist to /key/generate and
allow the fallback path used by infra/modules/sluice_aca/main.tf
extra_fallbacks.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8e98d04bb2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| - **Direct KV** (matches the runbook convention below): | ||
| ``` | ||
| az keyvault secret set --vault-name pvc-prod-sluice-kv \ | ||
| --name anthropic-api-key --value <key> | ||
| ``` |
There was a problem hiding this comment.
If an operator follows this Direct KV option, the deploy will still be Azure-only: the deploy workflows only populate TF_VAR_extra_providers from the GitHub ANTHROPIC_API_KEY secret, and Terraform only creates/mounts the provider-anthropic-key secret from that variable. Nothing in the inspected workflows reads anthropic-api-key, so local.has_anthropic remains false and the Claude models plus agent-default alias are not rendered; the runbook should not present this command as an equivalent go-live path unless the missing wiring is added.
Useful? React with 👍 / 👎.
What & why
Enables Claude/Anthropic in the LiteLLM gateway so non-interactive CI agents can route Claude Code through Sluice with alias-based model selection, implementing ADR
docs/architecture/09-claude-code-via-sluice.md("API-key billing = through Sluice"). Interactive Max-subscription Claude Code use stays direct-Anthropic.Mirrors the existing multi-provider
extra_providerspattern (ADR 11) exactly — no bespoke plumbing.The
agent-defaultaliasCI agents request
model="agent-default". Sluice resolves it to a concrete Claude model so it can be re-routed centrally without touching any workflow:anthropic/claude-haiku-4-5anthropic/claude-sonnet-4-6premium(Azuregpt-4o)Fallback chain:
agent-default → claude-sonnet-4-6 → premium. Emitted whenever the Anthropic provider is active (independent of the Groq/Gemini/etc.has_any_extrachains).Changes
infra/modules/sluice_aca/main.tf—has_anthropicgate +anthropic_models(concreteclaude-haiku-4-5/claude-sonnet-4-6+ theagent-defaultgroup with per-tierorder), concatenated intolitellm_model_list; newanthropic_fallbackschain concatenated intofallbacks.extra_providerspath: ananthropicentry auto-creates theprovider-anthropic-keyKV secret and injects theLITELLM_ANTHROPIC_API_KEYenv var on the container (identical mechanism toGROQ_API_KEY). No new variable or dynamic block needed.deploy.yaml+deploy-environment.yaml: read anANTHROPIC_API_KEYGitHub environment secret and fold it intoTF_VAR_extra_providerswithenv_name = LITELLM_ANTHROPIC_API_KEY. Absent key ⇒ provider stays inactive, plan unaffected.scripts/keys.yaml— new dedicatedmystira-civirtual key allow-listingagent-default+ the Claude models. (Kept separate from the interactiveclaude-code-clikey, which stays on the Max subscription.)infra/env/prod/terraform.tfvars— documented theanthropicextra_providersexample.extra_providersflow.Guardrails run
terraform fmt -recursive: clean (no changes).terraform validateoninfra/modules/sluice_aca: passes.plan/applyrun against any environment (PR only).Operator: the ONE step to go live
Place the Anthropic Console API key so Terraform can pick it up, then deploy + apply keys. Preferred (matches every other provider):
ANTHROPIC_API_KEYas aprodGitHub environment secret — the deploy workflow folds it intoTF_VAR_extra_providers, and Terraform then creates theprovider-anthropic-keyKV secret and mountsLITELLM_ANTHROPIC_API_KEY.Direct-KV alternative (per ADR-09 runbook convention), only if wiring
TF_VAR_extra_providersfrom KV rather than a GitHub secret:Then: run the
deployworkflow, andpython scripts/manage_keys.py apply.🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
agent-defaultmodel route for non-interactive CI use.Bug Fixes
Documentation