feat(R4): make LLM rate limit configurable via LLM_MAX_CALLS_PER_MIN#88
feat(R4): make LLM rate limit configurable via LLM_MAX_CALLS_PER_MIN#88siva23367 wants to merge 1 commit into
Conversation
|
@siva23367 is attempting to deploy a commit to the varun7singh's projects Team on Vercel. A member of the Team first needs to authorize it. |
VarunGitGood
left a comment
There was a problem hiding this comment.
Review
The goal is sound but there is one fatal wiring bug that makes the feature non-functional, plus a few unrelated changes that should not be in this PR.
[BLOCKING] container.py is never updated — the setting is never actually read
container.py:227 is the only place ReactInvestigationLoop is instantiated, and this PR does not touch it:
return ReactInvestigationLoop(
llm=llm,
tools=tools,
...
enable_reflection=settings.ENABLE_REFLECTION,
reflection_interval=settings.REFLECTION_INTERVAL,
# llm_max_calls_per_min missing — always None → hardcoded fallback of 60
)Result: _wait_for_rate_limit always uses 60 regardless of what is in config.json. Fix: add llm_max_calls_per_min=settings.LLM_MAX_CALLS_PER_MIN, to that constructor call.
Once that wire-up exists, the < 1 guard inside __init__ is also dead code — pydantic already enforces ge=1 before the value can reach the constructor via settings.
On the positive side: the 400/500 split in api/config.py is a genuine improvement and the field definition + README docs are well written.
|
|
||
| WATCHER_CONFIG_REFRESH_SECS: int = 30 | ||
|
|
||
| # R4: configurable LLM calls-per-minute cap for the ReAct investigation loop. |
There was a problem hiding this comment.
Four-line comment block reads like a changelog entry, not a code comment. Per project style: only comment when the WHY is non-obvious from the code itself. ge=1 already enforces the constraint; the provider recommendations belong in the README (where they already live). Remove this block — the field definition is self-explanatory.
| self.reflection_interval = reflection_interval | ||
| self.max_reflections = max_reflections | ||
|
|
||
| # Setup dynamic rate limit with a reasonable fallback (e.g., 60 if not specified) |
There was a problem hiding this comment.
Remove this comment — it describes what the code below does, which the code already makes clear.
Also: this block defaults to 60 on None, but that is a hardcoded constant, not settings.LLM_MAX_CALLS_PER_MIN. Until container.py is updated to pass the setting at construction time, this fallback silently overrides the user's config with no warning.
| "click==8.1.7", | ||
| "typer>=0.12.0,<0.13", | ||
| "watchfiles>=0.21.0,<0.22", | ||
| "httpx>=0.28.1", |
There was a problem hiding this comment.
httpx is already a transitive dependency (used directly in repi/llm/adapters.py), so promoting it to an explicit dep is reasonable — but it has nothing to do with this PR. Please move to a separate commit.
| | `OLLAMA_BASE_URL` | `http://localhost:11434` | Ollama endpoint | | ||
| | `LLM_MAX_CALLS_PER_MIN` | `60` | Max LLM calls per rolling 60-second window in the ReAct loop. Lower for free-tier providers; raise for paid/high-tier accounts. | | ||
|
|
||
|
|
There was a problem hiding this comment.
Don't require this patch, the above added variable is good enough
|
@siva23367 Hey thanks for the PR, can you please check and resolve the comments and feel free to ask for clarifications |
- Add LLM_MAX_CALLS_PER_MIN field to Settings (default 60, ge=1) - Replace hardcoded >= 3 check in _wait_for_rate_limit with configurable value - Read setting from settings singleton in ReactInvestigationLoop.__init__ - Document recommended values per provider in README - Improve PUT /config error handling (400 vs 500 separation) Closes #13
55215ee to
23e163b
Compare
|
Hi @VarunGitGood , addressed all review comments:
All 254 tests pass. |
Summary
Replaces the hardcoded
>= 3LLM-calls-per-minute cap in_wait_for_rate_limitwith a configurableLLM_MAX_CALLS_PER_MINsetting.Changes
repi/core/config.py— AddedLLM_MAX_CALLS_PER_MINfield (default 60, ge=1)repi/investigation/react_loop.py—__init__reads from settings,_wait_for_rate_limituses configurable valuerepi/api/config.py— Improved error handling (400 vs 500 separation)README.md— Added rate limiting documentation with provider recommendationsTesting
All 254 tests pass.
Closes #13