Tune Claude Code CLI retry/timeout resilience env

[mkonopelski-gd]

Summary

• Forward API_TIMEOUT_MS (120s), CLAUDE_CODE_MAX_RETRIES (3), and CLAUDE_ASYNC_AGENT_STALL_TIMEOUT_MS (120s) to the Claude Code CLI subprocess via a new setup_claude_code_resilience_env(), wired into agent_query's env alongside the existing setup_claude_code_* helpers. Goal: let the CLI itself retry past a transient network blip instead of burning a phase-level resume attempt (or the whole phase) in agent_query_with_resume.
• Fills in the previously-empty __main__ with a self-contained local debug entrypoint (throwaway git workspace with a local origin, redirected AGENT_LOGS_BASE_PATH/WORKSPACE_BASE_PATH/CLAUDE_CODE_TMPDIR_PATH, .env-backfilled provider key) plus a retry-forcing experiment (_debug_retry_experiment, run via DEBUG_RETRY_EXPERIMENT=1) used to verify the retry mechanism end-to-end.

How this was tested

• Real 2-phase debug run (python -m app.services.claude_code): generated a working Express app end-to-end against a throwaway workspace — confirms the harness plumbing (workspace/cache/log paths, provider key resolution) works outside docker-compose.
• Retry verification: _debug_retry_experiment runs a single agent_query with a deliberately low API_TIMEOUT_MS (500ms, far below any real response time) and CLAUDE_CODE_MAX_RETRIES=5, forcing every attempt to time out. At DEBUG log level this produced exactly 5 api_retry SystemMessages (attempt: 1..5, max_retries: 5) over ~21s before the CLI gave up — direct, first-party confirmation that CLAUDE_CODE_MAX_RETRIES is honored by the SDK.
• OpenRouter corroboration: the OpenRouter Activity log showed 6 request rows in the same ~90s window the two experiment runs finished in (timestamps cross-checked against local log file mtimes). This corroborates real requests hit the API in that window; note I could not fully reconcile the exact per-row token counts (4 rows ~304 input tokens, 2 rows ~26,946) against individual retry attempts, so treat this as supporting evidence alongside the agent-log proof above, not an exact accounting.
• make check (ruff clean; the 3 remaining mypy errors are pre-existing, in unrelated files not touched by this change) and make unit-tests (658 passed) both green.

Test plan

• Review CLAUDE_CODE_MAX_RETRIES=3 / API_TIMEOUT_MS=120000 / CLAUDE_ASYNC_AGENT_STALL_TIMEOUT_MS=120000 as the intended production defaults
• Confirm no other call site relies on the CLI's previous default retry/timeout behavior

[mkonopelski-gd]

@awrobel-gd This actually doesn't make sense. Because are way lower than default which I finally found (https://code.claude.com/docs/en/env-vars)

API_TIMEOUT_MS: 600000 (10 minutes).
CLAUDE_CODE_MAX_RETRIES: 10.
CLAUDE_ASYNC_AGENT_STALL_TIMEOUT_MS: 600000 (10 minutes).

[awrobel-gd]

@awrobel-gd This actually doesn't make sense. Because are way lower than default which I finally found (https://code.claude.com/docs/en/env-vars)

API_TIMEOUT_MS: 600000 (10 minutes).
CLAUDE_CODE_MAX_RETRIES: 10.
CLAUDE_ASYNC_AGENT_STALL_TIMEOUT_MS: 600000 (10 minutes).

Ok but maybe our version is too old? We have no retry, or the retries are not logged. Like if we have error about connection lost, the somewhere in this error or in the session JSON metadata printed, should be some time value ~600s, which we dont see.
We dont see either because when session fails, we dont print last agent message, or we dont print the response object metadata... or I dont know.

The default settings you posted seem to have no effect? What if we upgrade version or try to find which Claude SDK added these settins