Skip to content

feat(R4): make LLM rate limit configurable via LLM_MAX_CALLS_PER_MIN#88

Closed
siva23367 wants to merge 1 commit into
VarunGitGood:mainfrom
siva23367:feature/configurable-rate-limit
Closed

feat(R4): make LLM rate limit configurable via LLM_MAX_CALLS_PER_MIN#88
siva23367 wants to merge 1 commit into
VarunGitGood:mainfrom
siva23367:feature/configurable-rate-limit

Conversation

@siva23367

Copy link
Copy Markdown
Contributor

Summary

Replaces the hardcoded >= 3 LLM-calls-per-minute cap in _wait_for_rate_limit with a configurable LLM_MAX_CALLS_PER_MIN setting.

Changes

  • repi/core/config.py — Added LLM_MAX_CALLS_PER_MIN field (default 60, ge=1)
  • repi/investigation/react_loop.py__init__ reads from settings, _wait_for_rate_limit uses configurable value
  • repi/api/config.py — Improved error handling (400 vs 500 separation)
  • README.md — Added rate limiting documentation with provider recommendations

Testing

All 254 tests pass.

Closes #13

@vercel

vercel Bot commented Jun 16, 2026

Copy link
Copy Markdown

@siva23367 is attempting to deploy a commit to the varun7singh's projects Team on Vercel.

A member of the Team first needs to authorize it.

@VarunGitGood VarunGitGood left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review

The goal is sound but there is one fatal wiring bug that makes the feature non-functional, plus a few unrelated changes that should not be in this PR.

[BLOCKING] container.py is never updated — the setting is never actually read

container.py:227 is the only place ReactInvestigationLoop is instantiated, and this PR does not touch it:

return ReactInvestigationLoop(
    llm=llm,
    tools=tools,
    ...
    enable_reflection=settings.ENABLE_REFLECTION,
    reflection_interval=settings.REFLECTION_INTERVAL,
    # llm_max_calls_per_min missing — always None → hardcoded fallback of 60
)

Result: _wait_for_rate_limit always uses 60 regardless of what is in config.json. Fix: add llm_max_calls_per_min=settings.LLM_MAX_CALLS_PER_MIN, to that constructor call.

Once that wire-up exists, the < 1 guard inside __init__ is also dead code — pydantic already enforces ge=1 before the value can reach the constructor via settings.


On the positive side: the 400/500 split in api/config.py is a genuine improvement and the field definition + README docs are well written.

Comment thread docker-compose.yml
Comment thread repi/core/config.py
Comment thread repi/core/config.py Outdated

WATCHER_CONFIG_REFRESH_SECS: int = 30

# R4: configurable LLM calls-per-minute cap for the ReAct investigation loop.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Four-line comment block reads like a changelog entry, not a code comment. Per project style: only comment when the WHY is non-obvious from the code itself. ge=1 already enforces the constraint; the provider recommendations belong in the README (where they already live). Remove this block — the field definition is self-explanatory.

Comment thread repi/investigation/react_loop.py Outdated
Comment thread repi/investigation/react_loop.py Outdated
self.reflection_interval = reflection_interval
self.max_reflections = max_reflections

# Setup dynamic rate limit with a reasonable fallback (e.g., 60 if not specified)

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this comment — it describes what the code below does, which the code already makes clear.

Also: this block defaults to 60 on None, but that is a hardcoded constant, not settings.LLM_MAX_CALLS_PER_MIN. Until container.py is updated to pass the setting at construction time, this fallback silently overrides the user's config with no warning.

Comment thread repi/investigation/react_loop.py Outdated
Comment thread pyproject.toml
"click==8.1.7",
"typer>=0.12.0,<0.13",
"watchfiles>=0.21.0,<0.22",
"httpx>=0.28.1",

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

httpx is already a transitive dependency (used directly in repi/llm/adapters.py), so promoting it to an explicit dep is reasonable — but it has nothing to do with this PR. Please move to a separate commit.

Comment thread web/package-lock.json
Comment thread README.md
| `OLLAMA_BASE_URL` | `http://localhost:11434` | Ollama endpoint |
| `LLM_MAX_CALLS_PER_MIN` | `60` | Max LLM calls per rolling 60-second window in the ReAct loop. Lower for free-tier providers; raise for paid/high-tier accounts. |


Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't require this patch, the above added variable is good enough

@VarunGitGood

Copy link
Copy Markdown
Owner

@siva23367 Hey thanks for the PR, can you please check and resolve the comments and feel free to ask for clarifications

- Add LLM_MAX_CALLS_PER_MIN field to Settings (default 60, ge=1)
- Replace hardcoded >= 3 check in _wait_for_rate_limit with configurable value
- Read setting from settings singleton in ReactInvestigationLoop.__init__
- Document recommended values per provider in README
- Improve PUT /config error handling (400 vs 500 separation)

Closes #13
@siva23367
siva23367 force-pushed the feature/configurable-rate-limit branch from 55215ee to 23e163b Compare June 19, 2026 09:41
@siva23367

Copy link
Copy Markdown
Contributor Author

Hi @VarunGitGood , addressed all review comments:

  • Added llm_max_calls_per_min=settings.LLM_MAX_CALLS_PER_MIN to container.py
  • Removed 4-line comment block from config.py
  • Removed all "what" comments from react_loop.py
  • Reverted docker-compose.yml, pyproject.toml, web/package-lock.json

All 254 tests pass.

@siva23367 siva23367 closed this Jun 19, 2026
@siva23367
siva23367 deleted the feature/configurable-rate-limit branch June 19, 2026 10:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

R4: Make LLM rate limit configurable

2 participants