Skip to content

feat(R4): make LLM rate limit configurable via LLM_MAX_CALLS_PER_MIN#89

Merged
VarunGitGood merged 7 commits into
VarunGitGood:mainfrom
siva23367:feat/configurable-rate-limit-v2
Jun 23, 2026
Merged

feat(R4): make LLM rate limit configurable via LLM_MAX_CALLS_PER_MIN#89
VarunGitGood merged 7 commits into
VarunGitGood:mainfrom
siva23367:feat/configurable-rate-limit-v2

Conversation

@siva23367

@siva23367 siva23367 commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Summary

Replaces the hardcoded >= 3 LLM-calls-per-minute cap in _wait_for_rate_limit
with a configurable LLM_MAX_CALLS_PER_MIN setting.

Changes

  • repi/investigation/react_loop.py — parameter type changed to int (default 60),
    dead code removed, _wait_for_rate_limit uses self.llm_max_calls_per_min
  • repi/core/container.py — wired llm_max_calls_per_min=settings.LLM_MAX_CALLS_PER_MIN

Testing

All 254 tests pass.

Closes #13

@vercel

vercel Bot commented Jun 19, 2026

Copy link
Copy Markdown

@siva23367 is attempting to deploy a commit to the varun7singh's projects Team on Vercel.

A member of the Team first needs to authorize it.

@siva23367 siva23367 closed this Jun 19, 2026
@siva23367
siva23367 deleted the feat/configurable-rate-limit-v2 branch June 19, 2026 10:17
@siva23367
siva23367 restored the feat/configurable-rate-limit-v2 branch June 19, 2026 10:17
@siva23367 siva23367 reopened this Jun 19, 2026
@siva23367
siva23367 force-pushed the feat/configurable-rate-limit-v2 branch from 480f7c1 to a7b6d10 Compare June 19, 2026 10:30
siva23367 and others added 3 commits June 19, 2026 19:46
feat(R4): make LLM rate limit configurable via LLM_MAX_CALLS_PER_MIN

feat(R4): add LLM_MAX_CALLS_PER_MIN field to Settings
@siva23367
siva23367 force-pushed the feat/configurable-rate-limit-v2 branch from a7b6d10 to ffdedc6 Compare June 19, 2026 14:45
@VarunGitGood

Copy link
Copy Markdown
Owner

Hey @siva23367 i see some tests failing also can you take a pull and ensure no conflicts please thanks!

@siva23367 siva23367 closed this Jun 22, 2026
@siva23367 siva23367 reopened this Jun 22, 2026
@siva23367

Copy link
Copy Markdown
Contributor Author

Hey @VarunGitGood — conflicts resolved, all 254 tests passing locally ✅ Ready for review!"

@VarunGitGood VarunGitGood left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review

Critical: Dead file added in wrong location

repi/investigation/config.py is an exact duplicate of repi/api/config.py and is placed in the wrong package. repi/api/__init__.py registers the config router via:

from repi.api.config import router as config_router

The new file at repi/investigation/config.py is never imported, never registered, and will never be served. It's dead code on arrival. This needs to either be deleted (if no change to the config router was intended) or moved/merged into repi/api/config.py.


Breaking behavior change: rate limit default jumps from 3 → 60

The original code:

while len(self._llm_call_timestamps) >= 3:

The new default:

LLM_MAX_CALLS_PER_MIN: int = Field(default=60, ge=1)

Changing the effective cap from 3 to 60 calls/min silently disables rate limiting for normal usage — the ReAct loop has a max of 10 iterations so it'll never hit 60. Users on free/low-tier API plans who were being protected by the conservative 3-call limit now risk hitting provider-side 429s with no warning.

If the goal is "configurable", the default should stay at or near 3 (the existing conservative behaviour), not 60. A sensible default might be 10 (matches max_iterations), but it should not be 60.


Duplicate entry in web/app/repi/docs/page.tsx

The diff adds this line:

"PostgreSQL full-text search + pgvector HNSW dense retrieval fused with Reciprocal Rank Fusion for best-of-both recall.",

immediately before the existing:

"ParadeDB BM25 full-text search + pgvector HNSW dense retrieval fused with Reciprocal Rank Fusion for best-of-both recall.",

This creates two near-identical bullet points in the FEATURES array. One of them should be removed.


Minor: unused import in repi/investigation/config.py

from pydantic import BaseModel

BaseModel is imported but never used anywhere in the file.


Minor: trailing whitespace in container.py

llm_max_calls_per_min=settings.LLM_MAX_CALLS_PER_MIN,  

Trailing spaces on that line.


Minor: LLM_MAX_CALLS_PER_MIN not documented

The new setting is not documented in CLAUDE.md's Optional configuration section, unlike all other settings. Add an entry there.


What's good

  • The core wiring is correct: Settings gets the field, container.py passes it through to ReactInvestigationLoop, and the loop uses self.llm_max_calls_per_min. That plumbing is clean.
  • PR #90 (the importlib fix) is a separate concern and should stay separate — good that it was split out.

@VarunGitGood

VarunGitGood commented Jun 23, 2026

Copy link
Copy Markdown
Owner

Hey @siva23367 ill guide you through this one, im leaving some comments on your code please resolve them the AI generated summary is also present.

VarunGitGood

This comment was marked as duplicate.

@VarunGitGood

Copy link
Copy Markdown
Owner

Hey @siva23367 the rest of the changes look good to me, ive added in the importlib and closed the other PR, please resolve these comments and lets get this merged thanks!

…e limit to 10, remove duplicate bullet, document LLM_MAX_CALLS_PER_MIN in README
@siva23367

Copy link
Copy Markdown
Contributor Author

Hey @VarunGitGood — all review comments addressed ✅

  • Deleted dead config.py,
    -Changed default from 60→10, removed duplicate bullet, and documented LLM_MAX_CALLS_PER_MIN in README.
    -All 254 tests passing. Ready for merge!"

@VarunGitGood

Copy link
Copy Markdown
Owner

Hey @siva23367 thanks LGTM

@VarunGitGood
VarunGitGood merged commit 62cb8e9 into VarunGitGood:main Jun 23, 2026
2 of 3 checks passed
@siva23367
siva23367 deleted the feat/configurable-rate-limit-v2 branch June 23, 2026 13:53
VarunGitGood added a commit that referenced this pull request Jul 5, 2026
feat(R4): make LLM rate limit configurable via LLM_MAX_CALLS_PER_MIN
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

R4: Make LLM rate limit configurable

2 participants