Defaulting to using claude-sonnet-4-6 for specification generation and repair#173
Defaulting to using claude-sonnet-4-6 for specification generation and repair#173
claude-sonnet-4-6 for specification generation and repair#173Conversation
|
Warning Rate limit exceeded
Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 20 minutes and 51 seconds. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThe pull request migrates the default LLM model from OpenAI's gpt-4o to Anthropic's claude-sonnet-4-6. It updates documentation to instruct users to provide an ANTHROPIC_API_KEY environment variable for specification generation and repair. The LLM backend implementation is modified to read ANTHROPIC_API_KEY for Claude models and LLM_API_KEY for others, and to use parallel single-request generation for Claude models instead of leveraging the n parameter. 🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
main.py (1)
193-200:⚠️ Potential issue | 🟡 MinorUpdate
--stub-out-llmhelp text — it still references onlyLLM_API_KEY.With the new default model
claude-sonnet-4-6,DefaultLlmBackend.__init__readsANTHROPIC_API_KEY(notLLM_API_KEY) for the default flow. The help text is now stale and will mislead users debugging CI runs.📝 Suggested diff
parser.add_argument( "--stub-out-llm", action="store_true", help=( "Stub out the LLM client (e.g., for integration tests). Otherwise, the constructor " - "looks for an environment variable (LLM_API_KEY) that is not available on a CI runner." + "looks for a model-specific API key (ANTHROPIC_API_KEY for Claude models, " + "LLM_API_KEY otherwise) that is not available on a CI runner." ), )🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@main.py` around lines 193 - 200, The --stub-out-llm argument help text is stale (mentions only LLM_API_KEY) while DefaultLlmBackend.__init__ now reads ANTHROPIC_API_KEY for the default model; update the parser.add_argument help string for "--stub-out-llm" to reference ANTHROPIC_API_KEY (and optionally LLM_API_KEY for backward compatibility) or generalize to "the required LLM API key environment variable (e.g., ANTHROPIC_API_KEY)" so users running CI see the correct env var to set when the LLM client is not stubbed.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@models/default_llm_backend.py`:
- Line 179: The error message raised by ModelError in the Claude/Anthropic retry
branch of _send_one_message is misleadingly hard-coded to "Vertex AI API: Too
many retries"; update the raise call in that branch (and the similar one at the
non‑Claude path) to use a generic message that reflects the actual LLM client,
e.g. "LLM API: Too many retries" or include the model identifier via self.model,
so logs/alerts correctly indicate which backend exhausted retries.
- Around line 112-185: The retry + context-compaction loop in _send_one_message
is duplicated with send_messages; extract that shared logic into a private
helper (suggest name _completion_with_retry) that accepts parameters used in the
loop (messages, temperature, n/top_k as optional, model, api_key,
vertex_credentials, max_tokens) and implements the retry count/backoff, context
compaction, and exception mapping, returning the raw completion response; then
update send_messages to call _completion_with_retry(n=top_k) and unpack the n
choices, and simplify _send_one_message to call _completion_with_retry(n=1) and
return the single choice, leaving _send_parallel unchanged except it will call
the simplified _send_one_message. Ensure the new helper references the existing
completion call and preserves all exception types and retry behavior.
- Around line 37-40: Introduce a single helper method _is_claude(model_name)
that centralizes all Claude-detection logic and replace every predicate
occurrence (model.startswith("claude"), "claude" in model, and "claude" in
self.model) with calls to _is_claude(...) in send_messages, _send_parallel,
_send_one_message and any other places; change the misleading error string in
_send_one_message from "Vertex AI API: Too many retries" to the
provider-agnostic "LLM API: Too many retries"; and factor the duplicated
retry/compaction logic shared by send_messages (non-parallel path) and
_send_one_message (parallel path) into a single helper (e.g.,
_retry_and_compact) that both paths call to avoid duplication.
In `@README.md`:
- Around line 14-22: The README implies both LLM_API_KEY and ANTHROPIC_API_KEY
are mandatory; update it to state that because DEFAULT_MODEL =
"claude-sonnet-4-6" in main.py, the default flow only requires ANTHROPIC_API_KEY
and show a single echo example for ANTHROPIC_API_KEY, then note that LLM_API_KEY
is only needed when using a non-Claude model (e.g., when passing --model gpt-4o)
and show an additional echo command for LLM_API_KEY as a conditional step;
mention DEFAULT_MODEL and the --model flag to make the mapping explicit.
---
Outside diff comments:
In `@main.py`:
- Around line 193-200: The --stub-out-llm argument help text is stale (mentions
only LLM_API_KEY) while DefaultLlmBackend.__init__ now reads ANTHROPIC_API_KEY
for the default model; update the parser.add_argument help string for
"--stub-out-llm" to reference ANTHROPIC_API_KEY (and optionally LLM_API_KEY for
backward compatibility) or generalize to "the required LLM API key environment
variable (e.g., ANTHROPIC_API_KEY)" so users running CI see the correct env var
to set when the LLM client is not stubbed.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 47afb43c-b995-41f8-8490-12b424433d5c
📒 Files selected for processing (3)
README.mdmain.pymodels/default_llm_backend.py
2 participants
This should have ideally been a single-line change (swapping out the model), but the Anthropic API (which is called by LiteLLM) does not support a
top_kparameter.The workaround is to issue
top_kparallel requests, which is implemented by this PR.