sandbox-cloudflare: 5-min run watchdog kills healthy long tool calls, and the kill revokes the MCP bridge under a still-running agent

[tombeckenham]

Summary

SandboxCoordinator hardcodes a no-progress watchdog:

const WATCHDOG_MS = 3e4;
const WATCHDOG_STALL_MS = 5 * 6e4; // 5 minutes, module constant, not configurable

"Progress" is measured purely by run-log appends — i.e. stream chunks. A harness like claude-code emits nothing on its stream-json output while a single tool call executes, so any tool call that runs longer than 5 minutes silently (a video render, a big test suite, a long build) trips the watchdog on a perfectly healthy run: the run is marked error ("run watchdog: no progress; orchestrator presumed dead") and the client's stream terminates with that error.

The compounding failure

The kill is doubly destructive in do-drives mode. When the watchdog fails the run, onRunSettled drops the per-run /_bridge state, so from that moment every host-tool callback from the still-running in-container agent gets a 404 — surfacing inside the harness as MCP server \"tanstack\" is not connected.

Observed sequence from a live session:

1. Agent authors a project for ~9 minutes with steady stream progress.
2. It then spends >5 minutes inside silent verification/snapshot Bash calls.
3. Watchdog fires → run marked failed → user sees the run die mid-flight.
4. The agent (still running!) finishes the work, then tries its host tools to expose/publish the result — every call fails MCP server \"tanstack\" is not connected, and the finished artifacts strand on the container's ephemeral disk. Its final "here's your preview" message goes to a stream nobody is reading.

Asks

1. Make the stall window configurable (per-app, ideally per-run). Different workloads have wildly different silent-tool-call ceilings; 5 minutes is tuned for chatty coding agents.
2. Count bridge activity as liveness. A host-tool callback arriving over /_bridge is proof the orchestrator and agent are alive — bumping the run record's updatedAt on bridge traffic would prevent most false positives regardless of the window chosen.
3. Consider making failStalledRun protected rather than private. Working around this today requires subclassing the configured coordinator and re-implementing all of alarm() (list runs, re-derive staleness, replicate the append/finish/settle sequence via the protected log/onRunSettled seams) just to change one constant — and the published Coordinator constructor type returns the abstract base, so the subclass also needs a declare'd buildRunStream to satisfy TS.

Environment

• @tanstack/ai-sandbox-cloudflare do-drives mode with the claude-code adapter (--include-partial-messages stream-json — no output during tool execution)