docs: align README, CHANGELOG, guides, and API baseline with the shipped surface

Correct the documented default pipeline order and add the subsystems that had
shipped without mention (webhook verification, the pagination package, the
idempotency and client-identity policies, correlation, and the reconnecting SSE
client) across the README, CHANGELOG, project guide, and docs pages. Fix a
broken import in the pipelines guide, a dangling reference and a misplaced
transport in the architecture guide, an inverted claim about retry body
buffering, a reference to a non-existent error type, and several value-object
and auth descriptions that no longer matched the code.

Regenerate the public API surface baseline to reflect the now-narrower surface
(the suffix-range type, the abstract CallContext base, and the removed
RetryConfig).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Author: OmarAlJarrah

CHANGELOG.md

@@ -41,6 +41,12 @@ removed, so existing code continues to work without modification.
 - **Log correlation** (`instrumentation.correlation`). A `contextvar`-backed
   correlation id that flows through the pipeline and is attached to log records,
   so logs from a single logical request can be tied together.
+- **Reconnecting SSE client** (`http.sse.connection`). `SseConnection` and
+  `AsyncSseConnection` resume an interrupted event stream by replaying the
+  `Last-Event-ID` header and reconnecting with jittered backoff that honours the
+  server's `retry:` hint. Built on the shared dispatch seam
+  (`pipeline.dispatch`), which lets both the SSE client and the paginator accept
+  either a pipeline or a bare send-callable.
 
 ### Changed
 

CLAUDE.md

@@ -43,13 +43,18 @@ bodies are modelled as typed Pythonic abstractions instead.
   `from_form` / `from_stream` / `from_iter` / `from_file`. `ResponseBody`
   exposes `iter_bytes` / `bytes` / `string`. Single-use bodies (stream /
   iter) raise `RuntimeError` on second consumption — call `to_replayable()`
-  before the first send if retries are needed.
+  before the first send if retries are needed. `AsyncRequestBody` /
+  `AsyncResponseBody` are the async twins (`aiter_bytes`), and
+  `MultipartField` / `MultipartRequestBody` build `multipart/form-data`
+  payloads.
 - **Body capture for logging uses `BytesIO`.** `LoggableRequestBody` mirrors
   writes into a `BytesIO` tap; `LoggableResponseBody` caches drained bytes
   for repeatable reads. Both honour a configurable byte cap.
 - **Thread-safety where stated.** `ContextStore` is safe under concurrent
-  use; individual bodies and streams are not. Per-context lookups rely on
-  CPython's GIL for atomic dict ops and use a lock only for check-and-set.
+  use; individual bodies and streams are not. Every store operation
+  (`get` / `put` / `set` / `remove`) acquires a `threading.Lock`, so the
+  guarantee survives free-threaded CPython (PEP 703) and runtimes without
+  atomic dict ops rather than relying on the GIL.
 - **Public API is narrow.** Helpers and concrete adapter classes are
   module-private (leading underscore). The public surface for each subpackage
   is what its `__init__.py` re-exports.
@@ -103,14 +108,20 @@ python-sdk/
     │   │   │   ├── common/                 # Headers, HttpHeaderName, MediaType,
     │   │   │   │                           # Protocol, Url, QueryParams, ETag,
     │   │   │   │                           # HttpRange, RequestConditions,
-    │   │   │   │                           # common_media_types
-    │   │   │   ├── request/                # Request, RequestBody, FileRequestBody,
-    │   │   │   │                           # LoggableRequestBody, Method
-    │   │   │   ├── response/               # Response, ResponseBody,
-    │   │   │   │                           # LoggableResponseBody, Status
+    │   │   │   │                           # common_media_types; pagination.py
+    │   │   │   │                           # (ItemPaged/Pager + async twins),
+    │   │   │   │                           # streaming.py (jsonl/chunked-frame iters)
+    │   │   │   ├── request/                # Request, RequestBody, AsyncRequestBody,
+    │   │   │   │                           # FileRequestBody, LoggableRequestBody,
+    │   │   │   │                           # MultipartField/MultipartRequestBody, Method
+    │   │   │   ├── response/               # Response, AsyncResponse, ResponseBody,
+    │   │   │   │                           # AsyncResponseBody, LoggableResponseBody,
+    │   │   │   │                           # Status
     │   │   │   ├── context/                # CallContext, DispatchContext,
     │   │   │   │                           # RequestContext, ExchangeContext,
     │   │   │   │                           # ContextStore
+    │   │   │   ├── sse/                     # Server-Sent Events parser + connection
+    │   │   │   ├── webhooks/                # webhook signature verification
     │   │   │   └── auth/                   # TokenCredential, BearerTokenPolicy,
     │   │   │                               # BasicAuthPolicy, KeyCredentialPolicy,
     │   │   │                               # ChallengeHandler (Basic/Digest/Composite),
@@ -119,19 +130,24 @@ python-sdk/
     │   │   │   │                           # Stage, StagedPipelineBuilder, defaults,
     │   │   │   │                           # sans-io + transport runners under the hood
     │   │   │   │
-    │   │   │   ├── policies/               # retry, redirect, logging, tracing,
-    │   │   │   │                           # set_date (+ async twins)
-    │   │   │   └── step/                   # PipelineStep, StepMetadata, RetryConfig
+    │   │   │   ├── policies/               # redirect, idempotency, retry, set_date,
+    │   │   │   │                           # client_identity, logging, tracing
+    │   │   │   │                           # (async twins only for the first five)
+    │   │   │   └── step/                   # PipelineStep, StepMetadata
     │   │   ├── client/                     # HttpClient + AsyncHttpClient Protocols
     │   │   ├── config/                     # Configuration
     │   │   ├── serde/                      # Serde, Serializer, Deserializer Protocols
     │   │   ├── errors/                     # SDK-level exception hierarchy
     │   │   ├── instrumentation/            # InstrumentationContext, Span, Tracer,
-    │   │   │                               # TracingScope, noops
+    │   │   │                               # TracingScope, noops, metrics,
+    │   │   │                               # correlation, client_logger, http_tracer,
+    │   │   │                               # identifiers, log_level, url_redactor
+    │   │   ├── pagination/                  # Page, Paginator, link-header + strategy
     │   │   └── util/                       # clock, proxy helpers
     │   └── tests/                          # pytest suite — auth/, config/, context/,
     │                                       # errors/, http/, instrumentation/,
-    │                                       # pipeline/, serde/, sse/, util/
+    │                                       # pagination/, pipeline/, serde/, sse/,
+    │                                       # util/, webhooks/
     ├── dexpace-sdk-http-stdlib/            # reference stdlib transports:
     │   │                                   # UrllibHttpClient, AsyncioHttpClient
     │   └── src/dexpace/sdk/http/stdlib/
@@ -143,6 +159,10 @@ python-sdk/
         └── src/dexpace/sdk/http/requests/
 ```
 
+Community-health and tooling files (`CHANGELOG.md`, `CONTRIBUTING.md`,
+`SECURITY.md`, `CODE_OF_CONDUCT.md`, `conftest.py`, `tools/`) are elided from
+the tree above.
+
 Every transport package depends on `dexpace-sdk-core` and adapts its HTTP
 library to the `HttpClient` / `AsyncHttpClient` Protocols. Namespace
 packaging (no `__init__.py` at `src/dexpace/`, `src/dexpace/sdk/`, or
@@ -185,12 +205,16 @@ Layered, bottom-up:
    evict on `CallContext.close()`.
 4. **`pipeline`** — `Policy` (and `AsyncPolicy`) wrap the downstream chain;
    `Pipeline` / `AsyncPipeline` run an ordered set of policies grouped into
-   `Stage`s via `StagedPipelineBuilder`. Shipped policies: retry, redirect,
-   logging, tracing, set-date (each with an async twin under
-   `pipeline/policies/`). `default_pipeline()` / `default_async_pipeline()`
-   assemble the standard stack. The lower-level `pipeline/step/PipelineStep`
-   Protocol (`(input, context) -> output`) plus `StepMetadata` / `RetryConfig`
-   remain for custom composition.
+   `Stage`s via `StagedPipelineBuilder`. Shipped policies: redirect,
+   idempotency, retry, set-date, client-identity, logging, tracing. Async
+   twins under `pipeline/policies/` exist only for redirect, idempotency,
+   retry, set-date, and client-identity; logging and tracing are sync-only.
+   `default_pipeline()` / `default_async_pipeline()` assemble the standard
+   stack in the order redirect → idempotency → retry → set-date →
+   client-identity → [auth] → logging → tracing (the async pipeline omits
+   logging and tracing). The lower-level `pipeline/step/PipelineStep` Protocol
+   (`(input, context) -> output`) plus `StepMetadata` remain for custom
+   composition.
 5. **`client/HttpClient`** — single-method Protocol
    (`execute(request) -> Response`). Transport is **not** provided by `core`;
    the `dexpace-sdk-http-*` packages (stdlib, httpx, aiohttp, requests) each

README.md

@@ -79,8 +79,9 @@ with UrllibHttpClient() as client, client.execute(request) as response:
 ### A configured pipeline
 
 `default_pipeline()` returns a `StagedPipelineBuilder` pre-wired with the
-canonical policy stack (redirect, retry, set-date, logging, tracing). Add
-authentication and adjust whatever the defaults get wrong for you:
+canonical policy stack (redirect, idempotency, retry, set-date,
+client-identity, logging, tracing). Add authentication and adjust whatever
+the defaults get wrong for you:
 
 ```python
 from dexpace.sdk.core.http.auth import BearerTokenPolicy
@@ -112,7 +113,8 @@ from dexpace.sdk.core.http.request import RequestBody
 RequestBody.from_stream(open("payload.bin", "rb"))
 RequestBody.from_iter([b"chunk-1", b"chunk-2"])
 
-# Replayable (transports may use zero-copy sendfile)
+# Replayable; a transport could special-case file bodies (e.g. zero-copy
+# sendfile), though none of the shipped transports do so today
 RequestBody.from_file("upload.bin")
 
 # Convert any single-use body into a replayable one before retrying
@@ -130,8 +132,9 @@ the way back up. The terminal policy hands the request to an `HttpClient`
 transport.
 
 ```
-caller → Pipeline → REDIRECT → RETRY → SET_DATE → AUTH → LOGGING → POST_LOGGING → HttpClient → wire
-                     (pillar)  (pillar)            (pillar) (pillar)
+caller → Pipeline → REDIRECT → POST_REDIRECT → RETRY → POST_RETRY → [AUTH] → LOGGING → POST_LOGGING → HttpClient → wire
+                     (pillar)   idempotency    (pillar)  set-date    (pillar) (pillar)  tracing
+                                                          client-identity
 ```
 
 Ordering is governed by `Stage`, an `IntEnum` whose values sit 100 apart so
@@ -169,12 +172,14 @@ Bottom-up, the layers are:
 | `http.common`       | `Headers`, `HttpHeaderName`, `MediaType`, `Protocol`, `Url`, `QueryParams`, `ETag`, `HttpRange`, `RequestConditions`, paging primitives |
 | `http.context`      | `CallContext` → `DispatchContext` → `RequestContext` → `ExchangeContext` chain, `ContextStore`           |
 | `http.auth`         | `BearerTokenPolicy`, `BasicAuthPolicy`, `KeyCredentialPolicy`, `DigestChallengeHandler`, RFC 7235 challenge parser, `TokenCache` |
-| `http.sse`          | `SseParser` for Server-Sent Events streams                                                               |
+| `http.sse`          | `SseParser`, plus reconnecting `SseConnection` / `AsyncSseConnection` (Last-Event-ID replay + backoff)   |
+| `http.webhooks`     | `WebhookVerifier`, `InvalidWebhookSignatureError` — HMAC signature verification with timestamp tolerance  |
+| `pagination`        | `Page`, `Paginator` / `AsyncPaginator`, `PaginationStrategy` (`CursorStrategy`, `PageNumberStrategy`, `LinkHeaderStrategy`) |
 | `pipeline`          | `Pipeline`, `AsyncPipeline`, `Policy` ABC, `Stage` enum, `StagedPipelineBuilder`, `default_pipeline()`   |
-| `pipeline.policies` | `RetryPolicy`, `RedirectPolicy`, `SetDatePolicy`, `LoggingPolicy`, `TracingPolicy` (+ async twins)       |
+| `pipeline.policies` | `RedirectPolicy`, `IdempotencyPolicy`, `RetryPolicy`, `SetDatePolicy`, `ClientIdentityPolicy`, `LoggingPolicy`, `TracingPolicy` (async twins for all but logging/tracing) |
 | `client`            | `HttpClient` and `AsyncHttpClient` Protocols                                                             |
 | `serde`             | `Serde`, `Serializer`, `Deserializer` Protocols + `JsonSerde` reference impl                             |
-| `instrumentation`   | `ClientLogger`, `UrlRedactor`, `Tracer`, `Span`, `InstrumentationContext`, noop singletons               |
+| `instrumentation`   | `ClientLogger`, `UrlRedactor`, `Tracer`, `Span`, `InstrumentationContext`, `contextvars` correlation helpers, noop singletons |
 | `errors`            | `SdkError` hierarchy: `ServiceRequestError`, `ServiceResponseError`, `HttpResponseError[ModelT]`, …      |
 | `util`              | `Clock`, `AsyncClock`, `ProxyOptions`                                                                    |
 | `config`            | `Configuration` (layered env-var + override lookup) + `ConfigurationBuilder`                             |
@@ -203,7 +208,16 @@ Bottom-up, the layers are:
   OpenTelemetry-compatible spans via `TracingPolicy`, URL redaction with
   allowlisted query parameters, and capped body capture for diagnostics.
 - **Server-Sent Events.** A WHATWG-compliant `SseParser` with a bounded
-  line buffer.
+  line buffer, plus reconnecting `SseConnection` / `AsyncSseConnection`
+  that resume with `Last-Event-ID` and honour server `retry:` backoff.
+- **Pagination.** A top-level `pagination` package: `Page`, sync and async
+  `Paginator`s that iterate item-by-item or page-by-page, and pluggable
+  `PaginationStrategy` (cursor, page-number, and `Link`-header).
+- **Webhooks.** `WebhookVerifier` checks HMAC signatures with a timestamp
+  tolerance and constant-time comparison, raising
+  `InvalidWebhookSignatureError` on mismatch.
+- **Correlation.** `contextvars`-based trace/span propagation so the
+  idempotency and client-identity policies and logging share one id.
 - **A lean core.** `dexpace-sdk-core` carries a single runtime dependency
   (`furl`, which backs `Url` parsing); each transport adapter adds exactly
   one HTTP library.
@@ -220,8 +234,8 @@ uv sync
 ```
 
 ```bash
-uv run pytest -q                 # 646 tests across 5 packages
-uv run mypy --strict             # type-check (171 source files)
+uv run pytest -q                 # run the full test suite across 5 packages
+uv run mypy --strict             # type-check every package under strict mode
 uv run ruff check                # lint
 uv run ruff format --check       # formatting gate
 ```

docs/architecture.md

@@ -17,7 +17,13 @@ plug in a concrete transport via the `HttpClient` Protocol.
          │   - Pipeline      │          │   - BearerToken     │
          │   - Policy        │          │   - KeyCredential   │
          │   - PipelineStep  │          │   - BasicAuth       │
-         │   - retry, log,   │          │   - TokenCache      │
+         │   - redirect,     │          │   - TokenCache      │
+         │     idempotency,  │          │                     │
+         │     retry,        │          │                     │
+         │     set-date,     │          │                     │
+         │     client-       │          │                     │
+         │     identity,     │          │                     │
+         │     logging,      │          │                     │
          │     tracing       │          │                     │
          └──────────┬────────┘          └─────────────────────┘
                     │
@@ -32,9 +38,13 @@ plug in a concrete transport via the `HttpClient` Protocol.
                     │
          ┌──────────▼─────────────────────────────────────────┐
          │   client/  HttpClient + AsyncHttpClient            │
+         │   - Protocols only; transports plug in here        │
+         └──────────┬─────────────────────────────────────────┘
+                    │
+         ┌──────────▼─────────────────────────────────────────┐
+         │   dexpace-sdk-http-stdlib (separate distribution)  │
          │   - UrllibHttpClient (sync reference)              │
          │   - AsyncioHttpClient (async reference)            │
-         │   - real transports plug in here                   │
          └────────────────────────────────────────────────────┘
 ```
 
@@ -67,5 +77,5 @@ The Java port has an `IoProvider` / `Buffer` / `Source` / `Sink` layer
 (a port of Okio). In Python, `bytes` / `bytearray` / `memoryview` /
 `BytesIO` / `BinaryIO` already cover the same surface idiomatically.
 Bodies use `iter_bytes(chunk_size)` for streaming and ordinary stdlib
-primitives for everything else. See `to-implement.md` for the design
-rationale.
+primitives for everything else. See the "Things That Will Bite You"
+section in `CLAUDE.md` for the design rationale.

docs/auth.md

@@ -33,9 +33,10 @@ All concrete credentials redact secrets in their `__repr__`.
 - Calls `AccessTokenInfo.needs_refresh()` before each request; refreshes
   proactively when `refresh_on` has passed or `expires_on - leeway`
   (default 300 s) has been reached.
-- On a 401 response with a `WWW-Authenticate` header, invalidates the
-  cached token and calls `on_challenge(request, response)`. Override
-  `on_challenge` in a subclass to handle CAE / claims-challenge flows.
+- On any 401 response, invalidates the cached token. If the response
+  also carries a `WWW-Authenticate` header, then calls
+  `on_challenge(request, response)`. Override `on_challenge` in a
+  subclass to handle CAE / claims-challenge flows.
 
 ## Token cache
 

docs/bodies.md

@@ -17,10 +17,13 @@ classmethod factories for the common shapes.
 | `RequestBody.from_iter(Iterable[bytes])` | ❌         | Same. The iterable is consumed on first `iter_bytes`. |
 
 Single-use bodies raise `RuntimeError` on the second `iter_bytes` call.
-The retry policy in `pipeline.policies.retry` does **not** automatically
-buffer single-use bodies — that is intentionally explicit: call
-`body.to_replayable()` before the first send if the request is in a
-retryable scope.
+The retry policy in `pipeline.policies.retry` **does** automatically
+buffer single-use bodies when retries are enabled: `RetryPolicy.send`
+calls `body.to_replayable()` before the first attempt whenever
+`total_retries > 0`, so a retry can re-emit the same payload. The
+buffering step is skipped when `total_retries == 0`; if you bypass the
+retry policy you can still call `body.to_replayable()` yourself before
+the first send.
 
 ## Response shape
 
@@ -56,6 +59,8 @@ the response side — first access drains the underlying body into a
 ## Async equivalents
 
 `AsyncRequestBody` / `AsyncResponseBody` mirror the sync APIs with
-`aiter_bytes` and `async def bytes()` / `string()`. Factories: same
-names, plus `from_async_stream` and `from_async_iter` for the
-single-use shapes.
+`aiter_bytes` and `async def bytes()` / `string()`. The factory sets
+differ per side: `AsyncRequestBody` exposes `from_bytes`, `from_string`,
+`from_form`, `from_async_stream`, and `from_async_iter`;
+`AsyncResponseBody` exposes `from_bytes` and `from_async_stream` only
+(there is no `from_async_iter` on the response side).