Skip to content

ReAct: drop unconditional 2s min_iteration_delay (rate limiter already covers it) #94

Description

@VarunGitGood

Why

react_loop.py sleeps min_iteration_delay (default 2.0s) on every gathering iteration after the first:

# repi/investigation/react_loop.py:419
if state.next_step_number > 1 and state.actions_taken > 0:
    await asyncio.sleep(deps.min_iteration_delay)

This fires unconditionally, regardless of how close the loop is to the provider rate limit. On a 10-iteration investigation that is up to ~16-18s of pure sleep added to wall-clock latency for every investigation.

The stated reason for the delay is rate-limit avoidance — but the loop already has a real token-bucket limiter that handles this correctly:

  • _wait_for_rate_limit() (react_loop.py:127) tracks call timestamps in a 60s sliding window against llm_max_calls_per_min (default 60) and only sleeps when the window is actually full.
  • _llm_call_with_retry() honours LLMRateLimitError.retry_after and backs off on actual 429s.

So the blanket 2s sleep is redundant with the limiter and the retry path. It is latency we pay 100% of the time to defend against a condition that almost never occurs (60 calls/min is far above what a 10-step loop hits).

Scope (in)

  • Default min_iteration_delay to 0.0 (rely on _wait_for_rate_limit + retry-after backoff for rate-limit protection).
  • Keep the parameter so it can still be set explicitly if a deployment wants throttling.

Scope (out)

  • Removing the token-bucket limiter or the retry-after handling — those stay; they are the correct mechanism.

Acceptance

  • Investigation wall-clock time drops by ~(n_steps − 1) × old_delay on the eval scenarios, with no change in rate-limit errors.
  • Existing tests in tests/investigation/test_react_loop.py still pass.

Files

  • repi/investigation/react_loop.py — default value at __init__ (line ~745) and the sleep at line ~419.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestgood first issueGood for newcomersreAct improvementImprovements to the ReAct investigation loop / LLM prompt behavior

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions