Why
react_loop.py sleeps min_iteration_delay (default 2.0s) on every gathering iteration after the first:
# repi/investigation/react_loop.py:419
if state.next_step_number > 1 and state.actions_taken > 0:
await asyncio.sleep(deps.min_iteration_delay)
This fires unconditionally, regardless of how close the loop is to the provider rate limit. On a 10-iteration investigation that is up to ~16-18s of pure sleep added to wall-clock latency for every investigation.
The stated reason for the delay is rate-limit avoidance — but the loop already has a real token-bucket limiter that handles this correctly:
_wait_for_rate_limit() (react_loop.py:127) tracks call timestamps in a 60s sliding window against llm_max_calls_per_min (default 60) and only sleeps when the window is actually full.
_llm_call_with_retry() honours LLMRateLimitError.retry_after and backs off on actual 429s.
So the blanket 2s sleep is redundant with the limiter and the retry path. It is latency we pay 100% of the time to defend against a condition that almost never occurs (60 calls/min is far above what a 10-step loop hits).
Scope (in)
- Default
min_iteration_delay to 0.0 (rely on _wait_for_rate_limit + retry-after backoff for rate-limit protection).
- Keep the parameter so it can still be set explicitly if a deployment wants throttling.
Scope (out)
- Removing the token-bucket limiter or the retry-after handling — those stay; they are the correct mechanism.
Acceptance
- Investigation wall-clock time drops by ~(n_steps − 1) × old_delay on the eval scenarios, with no change in rate-limit errors.
- Existing tests in
tests/investigation/test_react_loop.py still pass.
Files
repi/investigation/react_loop.py — default value at __init__ (line ~745) and the sleep at line ~419.
Why
react_loop.pysleepsmin_iteration_delay(default 2.0s) on every gathering iteration after the first:This fires unconditionally, regardless of how close the loop is to the provider rate limit. On a 10-iteration investigation that is up to ~16-18s of pure sleep added to wall-clock latency for every investigation.
The stated reason for the delay is rate-limit avoidance — but the loop already has a real token-bucket limiter that handles this correctly:
_wait_for_rate_limit()(react_loop.py:127) tracks call timestamps in a 60s sliding window againstllm_max_calls_per_min(default 60) and only sleeps when the window is actually full._llm_call_with_retry()honoursLLMRateLimitError.retry_afterand backs off on actual 429s.So the blanket 2s sleep is redundant with the limiter and the retry path. It is latency we pay 100% of the time to defend against a condition that almost never occurs (60 calls/min is far above what a 10-step loop hits).
Scope (in)
min_iteration_delayto0.0(rely on_wait_for_rate_limit+ retry-after backoff for rate-limit protection).Scope (out)
Acceptance
tests/investigation/test_react_loop.pystill pass.Files
repi/investigation/react_loop.py— default value at__init__(line ~745) and the sleep at line ~419.