Skip to content

0.4.0

Choose a tag to compare

View all tags
@lesnik512 lesnik512 released this 05 Jun 12:45
0.4.0
2a9aac1
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

httpware 0.4.0 — Retry, RetryBudget, and Bulkhead

0.4.0 is additive. No breaking changes. Code written against 0.3.0 continues to work unchanged.

This release ships Epic 3 (Resilience) almost entirely: a Retry middleware with sensible defaults, a Finagle-style RetryBudget token bucket that prevents retry storms, a Bulkhead middleware that caps caller-side concurrency, and a refinement to the exception tree (NetworkError) that lets callers tell transient network failures apart from non-retryable transport failures.

New features

  • httpware.Retry — middleware that automatically retries transient failures on idempotent methods. Defaults:
    • max_attempts=3, base_delay=0.1s, max_delay=5.0s, full-jitter exponential backoff (AWS formulation)
    • Retries on 408, 429, 502, 503, 504 for GET / HEAD / OPTIONS / PUT / DELETE (non-idempotent methods like POST and PATCH are not retried by default — pass retry_methods= to opt in per client)
    • Retries on httpware.NetworkError and httpware.TimeoutError for the same method set
    • Honors Retry-After (seconds + HTTP-date forms, capped at max_delay); respect_retry_after=False disables
    • Optional attempt_timeout= wall-clock cap per attempt via asyncio.timeout()
    • On exhaustion, re-raises the original StatusError subclass unwrapped with a PEP 678 __notes__ entry ("httpware: gave up after N attempts")
  • httpware.RetryBudget — Finagle-style token bucket bounding retry rate to prevent retry storms when downstream services degrade. Defaults: ttl=10s, min_retries_per_sec=10, percent_can_retry=0.2 (match Finagle / AWS SDK / Envoy). Per Retry-instance by default; pass an explicit RetryBudget to share across multiple Retry middlewares (e.g., several AsyncClients hitting the same downstream).
  • httpware.RetryBudgetExhaustedError — distinct ClientError raised when the budget refuses a retry. Carries last_response: httpx2.Response | None, last_exception: BaseException | None, and attempts: int. Picklable across process boundaries.
  • httpware.NetworkError(TransportError) — refines the AsyncClient terminal mapping so transient httpx2.NetworkError-family exceptions (ConnectError, ReadError, WriteError, CloseError) raise httpware.NetworkError. InvalidURL and CookieConflict continue to raise bare TransportError. Pool-acquisition timeouts (httpx2.PoolTimeout) continue to raise httpware.TimeoutError.
  • httpware.Bulkhead — middleware that caps in-flight requests at the caller layer via asyncio.Semaphore. Distinct from httpx2.Limits (which caps the connection pool); Bulkhead caps the number of concurrent calls regardless of pool state. Parameters:
    • max_concurrent (required, no default — there's no universally-correct value; depends on downstream capacity)
    • acquire_timeout=1.0 seconds, with None = wait forever and 0 = fail fast on full bulkhead
    • On acquire_timeout elapsed: raises BulkheadFullError(ClientError) carrying max_concurrent and acquire_timeout
    • Slot release is guaranteed by an explicit try/finally around next() — success, exception, and cancellation all release deterministically
    • Bulkhead IS the sharable unit; pass the same instance to multiple AsyncClient(middleware=[shared]) calls to enforce a joint cap across clients
  • httpware.BulkheadFullError — distinct ClientError raised when the Bulkhead refuses to admit a request within acquire_timeout. Carries max_concurrent: int and acquire_timeout: float | None. Picklable across process boundaries.

Backwards compatibility

Subclassing keeps existing catch-blocks working unchanged:

  • except TransportError still catches all transient + permanent transport-layer failures (NetworkError is a subclass).
  • except ClientError still catches everything in the httpware exception tree, including the new RetryBudgetExhaustedError and BulkheadFullError.

The terminal mapping change only narrows what callers see when they check the exact type. Catch-by-isinstance behaves the same.

Usage

from httpware import AsyncClient, Retry, RetryBudget

async with AsyncClient(
    base_url="https://api.example.com",
    middleware=[Retry()],  # default: 3 attempts, full-jitter backoff, fresh RetryBudget
) as client:
    user = await client.get("/users/1", response_model=User)

Share a budget across several clients hitting the same downstream:

from httpware import AsyncClient, Retry, RetryBudget

shared_budget = RetryBudget()  # one bucket, shared

async with AsyncClient(
    base_url="https://upstream-a.example.com",
    middleware=[Retry(budget=shared_budget)],
) as client_a, AsyncClient(
    base_url="https://upstream-b.example.com",
    middleware=[Retry(budget=shared_budget)],
) as client_b:
    ...

Catch budget exhaustion specifically:

from httpware import RetryBudgetExhaustedError

try:
    response = await client.get("/users/1")
except RetryBudgetExhaustedError as exc:
    # Budget refused a retry; the prior failure is preserved.
    logger.warning(
        "retry budget exhausted after %d attempts; last status %s",
        exc.attempts,
        exc.last_response.status_code if exc.last_response else "n/a",
    )

Tune for tighter SLAs:

Retry(
    max_attempts=5,
    base_delay=0.05,
    max_delay=1.0,
    attempt_timeout=0.5,           # cap each attempt at 500ms wall-clock
    retry_methods=frozenset({"GET", "HEAD", "OPTIONS", "PUT", "DELETE", "POST"}),
    budget=RetryBudget(percent_can_retry=0.1),  # tighter cap
)

Cap caller-side concurrency with Bulkhead. Note: Bulkhead goes outside Retry in the middleware stack so a retrying request holds one slot across all attempts (rather than re-acquiring per retry):

from httpware import AsyncClient, Bulkhead, Retry

async with AsyncClient(
    base_url="https://api.example.com",
    middleware=[
        Bulkhead(max_concurrent=10),  # cap total in-flight at 10
        Retry(),                       # retries happen inside the Bulkhead slot
    ],
) as client:
    user = await client.get("/users/1", response_model=User)

Catch a full bulkhead:

from httpware import BulkheadFullError

try:
    response = await client.get("/users/1")
except BulkheadFullError as exc:
    logger.warning(
        "bulkhead full: %d in-flight, waited %s",
        exc.max_concurrent,
        exc.acquire_timeout,
    )

Share a Bulkhead across multiple clients hitting the same downstream:

shared_bulkhead = Bulkhead(max_concurrent=20)

async with AsyncClient(
    base_url="https://upstream.example.com/v1",
    middleware=[shared_bulkhead],
) as client_a, AsyncClient(
    base_url="https://upstream.example.com/v2",
    middleware=[shared_bulkhead],
) as client_b:
    ...  # the 20-slot cap is enforced jointly across A and B

What's still ahead

The only remaining Epic 3 work is 3-6 extension-slot documentation, which ships as a docs-only follow-up. Epic 5 (observability hooks + OTel middleware) is unstarted; logging of retry/bulkhead decisions plumbs through then.

Out of scope for this release (per the specs, may revisit on real-user pain): per-call retry override via extensions, a Backoff protocol abstraction, retry_on_exception= configuration, retrying streamed request bodies (the latter waits for AsyncClient.stream in Epic 4), per-host Bulkhead partitioning, and Bulkhead queue-depth metrics.

References

Assets 2
Loading