Expose reasoning effort for custom Claude models#4194
Conversation
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 89d24938e7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
ApprovabilityVerdict: Needs human review This PR introduces new feature behavior - enabling reasoning effort configuration for custom Claude models by probing each model individually and propagating capabilities through multiple layers. The concurrent probing logic and new capability resolution flow represent significant runtime changes that warrant human review. You can customize Macroscope's approvability policy. Learn more. |
|
Resolved the Codex feedback: removed the default |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit b7cff35. Configure here.

Summary
supportedEffortLevelsannounced for that selected modelhighor any other effort by default, and hide the control when no levels are advertisedxhighend-to-end for Claude-compatible custom modelsWhy
Claude-compatible gateways can expose Codex/OpenAI models through Claude Code using custom model IDs. Claude Code only announces
supportedEffortLevelsfor a custom model when that model is selected during initialization, so static capabilities would either hide valid reasoning levels or advertise options the model does not support. T3 should respect the model's dynamically discovered reasoning capabilities instead of assuming a fixed capability set.The custom model IDs are therefore initialized individually through the Claude Agent SDK. The resulting
initializationResult().modelsmetadata drives the provider capabilities while preserving the existing generic provider-option UI. The adapter and CLI text generation now reuse the provider’s cached capabilities for runtime validation, so they stay aligned with the levels actually announced by each model. Custom-model probes run sequentially to preserve failure isolation without unbounded subprocess fan-out.Closes #4192.
Before / After
Testing
vp test run apps/server/src/provider/Layers/ProviderRegistry.test.ts apps/server/src/provider/Layers/ClaudeAdapter.test.ts apps/server/src/textGeneration/ClaudeTextGeneration.test.ts(111 tests)vp run --filter t3 typecheckvp linton the four changed filesgpt-5.6-solClaude model: the Claude SDK advertisedLow,Medium,High,Extra High, andMax, the selector initially had no selection, and an explicitExtra Highselection worked end-to-endNote
Medium Risk
Increases probe subprocess work and changes effort handling for custom models; behavior is localized to Claude provider paths and covered by new tests, but mis-probes could hide or mislabel reasoning options.
Overview
Custom Claude model IDs now get reasoning effort from Claude Code’s per-model SDK metadata instead of static or built-in assumptions.
The capabilities probe runs a lightweight init for each configured custom model (concurrency 2, 25s timeout per probe), merges
supportedEffortLevelsinto provider model capabilities, and feeds that through a cachedresolveModelCapabilitieshook shared by the adapter, CLI text generation, and provider status. Custom models show a reasoning selector only when levels are advertised; nothing is defaulted when they are not. Explicit selections likexhighpass through when advertised, and built-in-only mappings (e.g. coercingxhightomax) no longer apply to custom slugs.Reviewed by Cursor Bugbot for commit ec95dc6. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Expose reasoning effort options for custom Claude models based on SDK capability probes
probeClaudeCapabilitiesinClaudeProvider.tsnow concurrently initializes the SDK for each custom model, collects advertisedsupportedEffortLevels, and returns them. Individual probes time out without discarding completed results.getClaudeModelCapabilitiesandnormalizeClaudeCliEffortare extended to use SDK-advertised effort levels for custom models, andxhighis no longer coerced tomaxfor custom models.makeClaudeAdapterandmakeClaudeTextGenerationaccept an asyncresolveModelCapabilitiesresolver so capability lookups use the per-instance probe cache at runtime.Macroscope summarized ec95dc6.