Skip to content

feat: add quota-aware grant failover#87

Merged
steipete merged 1 commit into
mainfrom
feat/quota-aware-grant-failover
Jul 6, 2026
Merged

feat: add quota-aware grant failover#87
steipete merged 1 commit into
mainfrom
feat/quota-aware-grant-failover

Conversation

@steipete

@steipete steipete commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Summary

  • learn sanitized per-grant quota and authentication state from provider response status and standard rate-limit headers
  • skip active cooldowns and prefer the strongest known remaining ratio within one configured priority
  • retry one LLM or GET/HEAD request against a distinct same-provider scoped grant after upstream 401, 403, or 429
  • expose last signal, cooldown, and request/token windows in the admin console
  • fail closed when scoped grants exist but none are available; never fall through to a Worker-global or anonymous provider path

Before / after

Before After
Stable priority/key ordering only Priority, current quota ratio, then stable key
Provider 401/403/429 returned immediately One bounded same-provider alternate attempt on safe routes
No per-grant runtime visibility Sanitized quota windows, signals, and cooldowns in admin
Unavailable scoped grants could be indistinguishable from no scoped grant Configured scoped pools fail closed when exhausted

Safety

  • runtime observations are revision-bound, so rotated credentials never inherit stale or in-flight cooldowns
  • provider-controlled timestamps and numeric values are bounded and normalized
  • router-owned response headers are stripped before being set
  • one budget reservation and one final accounting event cover the final attempt
  • response bodies, credential values, and grant keys are never stored in runtime state or exposed to clients

Proof

  • pnpm check
  • pnpm build
  • pnpm worker:e2e
  • pnpm --dir admin test:browser (desktop/mobile, 6/6)
  • local Worker E2E: 429 primary -> 200 backup; subsequent request skips cooldown; admin state readback; unavailable scoped pool does not reach upstream
  • autoreview: clean, no accepted/actionable findings

Advances #59.

@steipete steipete merged commit 3b1e7d0 into main Jul 6, 2026
6 checks passed
@steipete steipete deleted the feat/quota-aware-grant-failover branch July 6, 2026 19:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant