During the first live self-improvement campaign (2026-07, script: agent-runtime-swe commit 3b93916), Together silently retired its entire serverless 8B-Llama tier. Fail-loud dispatch caught it; with silent fallbacks this would have been an all-zero campaign burning full wall-clock and spend before anyone noticed.
Provider model availability is a live dependency, not a constant. Recommend:
- a preflight assert in campaign scripts and
runBenchmark that sends one cheap real request per configured model and aborts loudly on 404/deprecated-model errors before dispatching the fleet,
- an ops note in repo docs recording the Together 8B-Llama retirement as the precedent.
During the first live self-improvement campaign (2026-07, script: agent-runtime-swe commit 3b93916), Together silently retired its entire serverless 8B-Llama tier. Fail-loud dispatch caught it; with silent fallbacks this would have been an all-zero campaign burning full wall-clock and spend before anyone noticed.
Provider model availability is a live dependency, not a constant. Recommend:
runBenchmarkthat sends one cheap real request per configured model and aborts loudly on 404/deprecated-model errors before dispatching the fleet,