feat(R4): make LLM rate limit configurable via LLM_MAX_CALLS_PER_MIN#89
Conversation
|
@siva23367 is attempting to deploy a commit to the varun7singh's projects Team on Vercel. A member of the Team first needs to authorize it. |
480f7c1 to
a7b6d10
Compare
a7b6d10 to
ffdedc6
Compare
|
Hey @siva23367 i see some tests failing also can you take a pull and ensure no conflicts please thanks! |
|
Hey @VarunGitGood — conflicts resolved, all 254 tests passing locally ✅ Ready for review!" |
VarunGitGood
left a comment
There was a problem hiding this comment.
Review
Critical: Dead file added in wrong location
repi/investigation/config.py is an exact duplicate of repi/api/config.py and is placed in the wrong package. repi/api/__init__.py registers the config router via:
from repi.api.config import router as config_routerThe new file at repi/investigation/config.py is never imported, never registered, and will never be served. It's dead code on arrival. This needs to either be deleted (if no change to the config router was intended) or moved/merged into repi/api/config.py.
Breaking behavior change: rate limit default jumps from 3 → 60
The original code:
while len(self._llm_call_timestamps) >= 3:The new default:
LLM_MAX_CALLS_PER_MIN: int = Field(default=60, ge=1)Changing the effective cap from 3 to 60 calls/min silently disables rate limiting for normal usage — the ReAct loop has a max of 10 iterations so it'll never hit 60. Users on free/low-tier API plans who were being protected by the conservative 3-call limit now risk hitting provider-side 429s with no warning.
If the goal is "configurable", the default should stay at or near 3 (the existing conservative behaviour), not 60. A sensible default might be 10 (matches max_iterations), but it should not be 60.
Duplicate entry in web/app/repi/docs/page.tsx
The diff adds this line:
"PostgreSQL full-text search + pgvector HNSW dense retrieval fused with Reciprocal Rank Fusion for best-of-both recall.",
immediately before the existing:
"ParadeDB BM25 full-text search + pgvector HNSW dense retrieval fused with Reciprocal Rank Fusion for best-of-both recall.",
This creates two near-identical bullet points in the FEATURES array. One of them should be removed.
Minor: unused import in repi/investigation/config.py
from pydantic import BaseModelBaseModel is imported but never used anywhere in the file.
Minor: trailing whitespace in container.py
llm_max_calls_per_min=settings.LLM_MAX_CALLS_PER_MIN, Trailing spaces on that line.
Minor: LLM_MAX_CALLS_PER_MIN not documented
The new setting is not documented in CLAUDE.md's Optional configuration section, unlike all other settings. Add an entry there.
What's good
- The core wiring is correct:
Settingsgets the field,container.pypasses it through toReactInvestigationLoop, and the loop usesself.llm_max_calls_per_min. That plumbing is clean. - PR #90 (the importlib fix) is a separate concern and should stay separate — good that it was split out.
|
Hey @siva23367 ill guide you through this one, im leaving some comments on your code please resolve them the AI generated summary is also present. |
|
Hey @siva23367 the rest of the changes look good to me, ive added in the importlib and closed the other PR, please resolve these comments and lets get this merged thanks! |
…e limit to 10, remove duplicate bullet, document LLM_MAX_CALLS_PER_MIN in README
|
Hey @VarunGitGood — all review comments addressed ✅
|
|
Hey @siva23367 thanks LGTM |
feat(R4): make LLM rate limit configurable via LLM_MAX_CALLS_PER_MIN
Summary
Replaces the hardcoded >= 3 LLM-calls-per-minute cap in _wait_for_rate_limit
with a configurable LLM_MAX_CALLS_PER_MIN setting.
Changes
dead code removed, _wait_for_rate_limit uses self.llm_max_calls_per_min
Testing
All 254 tests pass.
Closes #13