From 8ef226b13c0965712266a666ae7d12c69af2f1c7 Mon Sep 17 00:00:00 2001 From: MariusVanDerWijden Date: Fri, 8 May 2026 10:50:54 +0200 Subject: [PATCH 1/6] engine: add Rest-SSZ spec --- src/engine/refactor-ssz.md | 484 +++++++++++++++++++ src/engine/refactor.md | 960 +++++++++++++++++++++++++++++++++++++ 2 files changed, 1444 insertions(+) create mode 100644 src/engine/refactor-ssz.md create mode 100644 src/engine/refactor.md diff --git a/src/engine/refactor-ssz.md b/src/engine/refactor-ssz.md new file mode 100644 index 000000000..101ef5042 --- /dev/null +++ b/src/engine/refactor-ssz.md @@ -0,0 +1,484 @@ +# Engine API v2 -- SSZ Container Sketches (Amsterdam) + +> **Status:** Sketch. This is a working draft of the concrete SSZ +> container definitions referenced by the Engine API v2 spec +> ([refactor.md](./refactor.md)). Field types, names, and `MAX_*` +> constants are placeholders and need a final review before +> publication. +> +> All conventions in this document follow +> [refactor.md § SSZ encoding conventions](./refactor.md#ssz-encoding-conventions): +> +> - `Optional[T]` ≡ `List[T, 1]` (length 0 = absent, length 1 = present) +> - `String` ≡ `List[byte, MAX_ERROR_BYTES]`, `MAX_ERROR_BYTES = 1024` +> - `ByteList[N]` ≡ `List[byte, N]` +> - `ByteVector[N]` is fixed-size, `Bytes32` etc. are aliases +> - All uints are little-endian + +--- + +## Table of contents + +- [Primitive aliases](#primitive-aliases) +- [`MAX_*` constants](#max-constants) +- [Shared structures](#shared-structures) + - [`Withdrawal`](#withdrawal) + - [`ExecutionPayload` (Amsterdam)](#executionpayload-amsterdam) + - [`PayloadAttributes` (Amsterdam)](#payloadattributes-amsterdam) + - [`ForkchoiceState`](#forkchoicestate) + - [`PayloadStatus`](#payloadstatus) +- [Endpoint containers](#endpoint-containers) + - [`POST /amsterdam/payloads`](#post-amsterdampayloads) + - [`POST /amsterdam/forkchoice`](#post-amsterdamforkchoice) + - [`GET /amsterdam/payloads/{payloadId}`](#get-amsterdampayloadspayloadid) + - [`POST /amsterdam/bodies/hash` and `GET /amsterdam/bodies?...`](#post-amsterdambodieshash-and-get-amsterdambodies) + - [`POST /blobs/v1`](#post-blobsv1) + - [`POST /blobs/v2`](#post-blobsv2) + - [`POST /blobs/v3`](#post-blobsv3) + - [`POST /blobs/v4`](#post-blobsv4) +- [Open sketch questions](#open-sketch-questions) + +--- + +## Primitive aliases + +| Alias | SSZ type | Notes | +| - | - | - | +| `Hash32` | `ByteVector[32]` | block / payload hashes | +| `Root` | `ByteVector[32]` | beacon-block roots, merkle roots | +| `Address` | `ByteVector[20]` | execution-layer 160-bit address | +| `Bloom` | `ByteVector[256]` | logs bloom filter | +| `VersionedHash` | `ByteVector[32]` | EIP-4844 versioned blob hash | +| `Bytes8` | `ByteVector[8]` | `payload_id` | +| `Bytes32` | `ByteVector[32]` | `prevRandao`, generic 32-byte values | +| `Bytes48` | `ByteVector[48]` | KZG commitments and proofs | +| `Uint64` | `uint64` | LE on the wire | +| `Uint256` | `uint256` | LE on the wire (`block_value`, `base_fee_per_gas`) | +| `Boolean` | `bool` | one byte, `0x00` / `0x01` | + +## `MAX_*` constants + +These are sketch values — final values come from a follow-up that +matches the consensus-specs `Amsterdam` preset. They are listed here +for completeness so readers can size the on-wire bounds. + +| Constant | Sketch value | Where it's used | +| - | - | - | +| `MAX_TXS_PER_PAYLOAD` | `1048576` | `ExecutionPayload.transactions` | +| `MAX_BYTES_PER_TX` | `1073741824` | element bound inside `transactions` | +| `MAX_WITHDRAWALS_PER_PAYLOAD` | `16` | `ExecutionPayload.withdrawals`, `PayloadAttributes.withdrawals` | +| `MAX_EXTRA_DATA_BYTES` | `32` | `ExecutionPayload.extra_data` | +| `MAX_BAL_BYTES` | TBD (EIP-7928) | `ExecutionPayload.block_access_list` | +| `MAX_EXECUTION_REQUESTS_PER_PAYLOAD` | TBD (EIP-7685) | `ExecutionPayloadEnvelope.execution_requests` | +| `MAX_BYTES_PER_EXECUTION_REQUEST` | TBD | element bound inside `execution_requests` | +| `MAX_VERSIONED_HASHES_PER_REQUEST` | `128` | `BlobsRequest.versioned_hashes` | +| `MAX_BODIES_REQUEST` | `128` | bodies request and response lists | +| `MAX_BLOBS_REQUEST` | `128` | blobs request and response lists | +| `MAX_BLOBS_PER_PAYLOAD` | `MAX_VERSIONED_HASHES_PER_REQUEST` | `BlobsBundle.commitments`, `.blobs` | +| `CELLS_PER_EXT_BLOB` | `128` (EIP-7594) | cell-proof and custody bitvectors | +| `BYTES_PER_BLOB` | `131072` | one blob (`4096 * 32`) | +| `MAX_ERROR_BYTES` | `1024` | `validation_error`, JSON error `detail` | + +--- + +## Shared structures + +These containers are used by multiple endpoints. They map directly +onto today's JSON-RPC structures with field renaming +(`camelCase` → `snake_case`) and the type changes that follow from +the SSZ encoding conventions. + +### `Withdrawal` + +Same as the consensus-specs `Withdrawal` container. The `amount` +field is now natively LE in SSZ; the `withdrawals.amount` LE-vs-BE +note in shanghai.md goes away. + +``` +Withdrawal { + index: Uint64 + validator_index: Uint64 + address: Address + amount: Uint64 # gwei +} +``` + +### `ExecutionPayload` (Amsterdam) + +Reflects today's [`ExecutionPayloadV4`](./amsterdam.md#executionpayloadv4). +`block_access_list` is a fixed field for Amsterdam (no `Optional[T]` +here — that's only used for cross-fork `BodyEntry` responses; the +Amsterdam `ExecutionPayload` always carries the BAL). + +``` +ExecutionPayload { + parent_hash: Hash32 + fee_recipient: Address + state_root: Hash32 + receipts_root: Hash32 + logs_bloom: Bloom + prev_randao: Bytes32 + block_number: Uint64 + gas_limit: Uint64 + gas_used: Uint64 + timestamp: Uint64 + extra_data: ByteList[MAX_EXTRA_DATA_BYTES] + base_fee_per_gas: Uint256 + block_hash: Hash32 + transactions: List[ByteList[MAX_BYTES_PER_TX], MAX_TXS_PER_PAYLOAD] + withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] + blob_gas_used: Uint64 + excess_blob_gas: Uint64 + block_access_list: ByteList[MAX_BAL_BYTES] # RLP-encoded EIP-7928 BAL + slot_number: Uint64 +} +``` + +Notes: + +- `block_access_list` is RLP-encoded inside an SSZ `ByteList`. EIP-7928's + encoding is RLP and we don't try to re-encode it as SSZ — the EL + treats it as opaque bytes for transport, decodes it as RLP for + validation. Same pattern as `transactions`. +- `transactions` elements remain RLP-encoded `TransactionType || + TransactionPayload` per EIP-2718. Receiver-side rule: each element + MUST be ≥ 1 byte (see refactor.md § Payload submission). + +### `PayloadAttributes` (Amsterdam) + +Reflects today's [`PayloadAttributesV4`](./amsterdam.md#payloadattributesv4). + +``` +PayloadAttributes { + timestamp: Uint64 + prev_randao: Bytes32 + suggested_fee_recipient: Address + withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] + parent_beacon_block_root: Root + slot_number: Uint64 +} +``` + +### `ForkchoiceState` + +Same fields as today's [`ForkchoiceStateV1`](./paris.md#forkchoicestatev1). + +``` +ForkchoiceState { + head_block_hash: Hash32 + safe_block_hash: Hash32 + finalized_block_hash: Hash32 +} +``` + +### `PayloadStatus` + +Used by `POST /payloads` (full enum) and `POST /forkchoice` +(restricted enum — `ACCEPTED` not allowed). + +``` +PayloadStatus { + status: uint8 # see enum below + latest_valid_hash: Optional[Hash32] + validation_error: Optional[String] +} +``` + +Status enum: + +| Value | Name | Used by | +| - | - | - | +| `1` | `VALID` | both | +| `2` | `INVALID` | both | +| `3` | `SYNCING` | both | +| `4` | `ACCEPTED` | `POST /payloads` only | + +`INVALID_BLOCK_HASH` is removed (already supplanted by `INVALID`). +`POST /forkchoice` MUST return `1`/`2`/`3` only; CLs MUST treat a +`4` from `/forkchoice` as a protocol error. + +`Optional[String]` resolves to `List[List[byte, MAX_ERROR_BYTES], 1]`. + +--- + +## Endpoint containers + +### `POST /amsterdam/payloads` + +Replaces `engine_newPayloadV5`. + +#### Request + +``` +ExecutionPayloadEnvelope { + payload: ExecutionPayload + parent_beacon_block_root: Root + execution_requests: List[ByteList[MAX_BYTES_PER_EXECUTION_REQUEST], MAX_EXECUTION_REQUESTS_PER_PAYLOAD] +} +``` + +`expected_blob_versioned_hashes` is removed (the EL recomputes it +from `payload.transactions`). + +#### Response + +`PayloadStatus` (full enum, `1`/`2`/`3`/`4`). + +### `POST /amsterdam/forkchoice` + +Replaces `engine_forkchoiceUpdatedV4`. + +#### Request + +``` +ForkchoiceUpdate { + forkchoice_state: ForkchoiceState + payload_attributes: Optional[PayloadAttributes] + custody_columns: Optional[Bitvector[CELLS_PER_EXT_BLOB]] +} +``` + +#### Response + +``` +ForkchoiceUpdateResponse { + payload_status: PayloadStatus # restricted: VALID | INVALID | SYNCING + payload_id: Optional[Bytes8] +} +``` + +### `GET /amsterdam/payloads/{payloadId}` + +Replaces `engine_getPayloadV6`. + +#### Response + +``` +BuiltPayload { + payload: ExecutionPayload + block_value: Uint256 + blobs_bundle: BlobsBundleV2 # see consensus-specs Osaka + execution_requests: List[ByteList[MAX_BYTES_PER_EXECUTION_REQUEST], MAX_EXECUTION_REQUESTS_PER_PAYLOAD] + should_override_builder: Boolean +} + +BlobsBundleV2 { + commitments: List[Bytes48, MAX_BLOBS_PER_PAYLOAD] + proofs: List[Bytes48, MAX_BLOBS_PER_PAYLOAD * CELLS_PER_EXT_BLOB] + blobs: List[ByteVector[BYTES_PER_BLOB], MAX_BLOBS_PER_PAYLOAD] +} +``` + +`commitments` and `blobs` MUST have equal length; `proofs` MUST +have length `len(blobs) * CELLS_PER_EXT_BLOB` (mirrors the +`engine_getPayloadV5` rule from osaka.md). + +### `POST /amsterdam/bodies/hash` and `GET /amsterdam/bodies?...` + +Replace `engine_getPayloadBodiesByHashV2` and +`engine_getPayloadBodiesByRangeV2`. Both return the same response +container. + +#### Request — `/bodies/hash` + +``` +BodiesByHashRequest { + block_hashes: List[Hash32, MAX_BODIES_REQUEST] +} +``` + +#### Request — `/bodies?from=N&count=M` + +URL query parameters; no SSZ request body. + +#### Response + +``` +BodiesResponse { + entries: List[BodyEntry, MAX_BODIES_REQUEST] +} + +BodyEntry { + available: Boolean + body: ExecutionPayloadBody +} + +# /amsterdam/bodies/... uses this Amsterdam-fork ExecutionPayloadBody +ExecutionPayloadBody { + transactions: List[ByteList[MAX_BYTES_PER_TX], MAX_TXS_PER_PAYLOAD] + withdrawals: Optional[List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD]] # [] pre-Shanghai + block_access_list: Optional[ByteList[MAX_BAL_BYTES]] # [] pre-Amsterdam or pruned +} +``` + +A CL on the Cancun schema would call `/cancun/bodies/...` and receive +a Cancun-shaped `ExecutionPayloadBody` (no `block_access_list` field +at all). The Cancun-fork variant is sketched here for clarity: + +``` +# /cancun/bodies/... ExecutionPayloadBody (for reference) +ExecutionPayloadBody { + transactions: List[ByteList[MAX_BYTES_PER_TX], MAX_TXS_PER_PAYLOAD] + withdrawals: Optional[List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD]] # [] pre-Shanghai +} +``` + +### `POST /blobs/v1` + +Replaces `engine_getBlobsV1` (Cancun whole-blob). + +#### Request + +``` +BlobsV1Request { + versioned_hashes: List[VersionedHash, MAX_BLOBS_REQUEST] +} +``` + +#### Response + +``` +BlobsV1Response = Optional[List[BlobV1Entry, MAX_BLOBS_REQUEST]] + +BlobV1Entry { + available: Boolean + contents: BlobAndProofV1 +} + +BlobAndProofV1 { + blob: ByteVector[BYTES_PER_BLOB] + proof: Bytes48 +} +``` + +When `available == false`, `contents` carries zero-valued bytes (a +`BYTES_PER_BLOB`-byte zero blob and a 48-byte zero proof). The outer +`Optional` returns `[]` when the EL cannot serve the request at all. + +### `POST /blobs/v2` + +Replaces `engine_getBlobsV2` (Osaka all-or-nothing cell proofs). + +#### Request — same as `/v1` + +``` +BlobsV2Request { + versioned_hashes: List[VersionedHash, MAX_BLOBS_REQUEST] +} +``` + +#### Response + +``` +BlobsV2Response = Optional[List[BlobV2Entry, MAX_BLOBS_REQUEST]] + +BlobV2Entry { + available: Boolean # always true for /v2 (all-or-nothing); included for shape symmetry + contents: BlobAndProofV2 +} + +BlobAndProofV2 { + blob: ByteVector[BYTES_PER_BLOB] + proofs: List[Bytes48, CELLS_PER_EXT_BLOB] +} +``` + +All-or-nothing: if any requested blob is missing, the outer +`Optional` returns `[]` and no per-entry data is sent. CLs that need +partial responses use `/v3`. + +### `POST /blobs/v3` + +Replaces `engine_getBlobsV3` (Osaka partial responses with cell +proofs). + +#### Request — same as `/v2` + +#### Response + +Same shape as `/v2` (`BlobV2Entry` reused), but missing blobs +surface as `available=false` per entry rather than collapsing the +whole response to `[]`. Outer `Optional` returns `[]` only when the +EL cannot serve the request at all (e.g. syncing). + +``` +BlobsV3Response = Optional[List[BlobV2Entry, MAX_BLOBS_REQUEST]] +``` + +### `POST /blobs/v4` + +Replaces `engine_getBlobsV4` (Amsterdam cell-range selection). + +#### Request + +``` +BlobsV4Request { + versioned_hashes: List[VersionedHash, MAX_BLOBS_REQUEST] + indices_bitarray: Bitvector[CELLS_PER_EXT_BLOB] +} +``` + +#### Response + +``` +BlobsV4Response = Optional[List[BlobV4Entry, MAX_BLOBS_REQUEST]] + +BlobV4Entry { + available: Boolean + contents: BlobCellsAndProofs +} + +BlobCellsAndProofs { + blob_cells: List[Optional[ByteVector[BYTES_PER_CELL]], CELLS_PER_EXT_BLOB] + proofs: List[Optional[Bytes48], CELLS_PER_EXT_BLOB] +} +``` + +Per the Amsterdam spec: only the indices set in the request's +`indices_bitarray` carry a value; all other indices are `[]`. Within +the requested indices, individual missing cells are also `[]`, and +the corresponding `proofs` entry MUST also be `[]` (`null` in the +old spec). + +`BYTES_PER_CELL` = `BYTES_PER_BLOB / CELLS_PER_EXT_BLOB` = `1024` +(EIP-7594). + +--- + +## Open sketch questions + +These are the items left to decide before promoting this sketch to +the canonical Amsterdam SSZ schema: + +1. **`MAX_*` placeholder values.** Several constants above are + `TBD` or sketch-only. They need to be pinned to the + consensus-specs `Amsterdam` preset values once those land. +2. **`MAX_BAL_BYTES`.** EIP-7928 defines the BAL but doesn't yet + pin a numeric upper bound that's friendly for SSZ. We need a + concrete number; otherwise the SSZ schema can't round-trip. +3. **`Bitvector` SSZ encoding for `indices_bitarray` and + `custody_columns`.** Both are `Bitvector[CELLS_PER_EXT_BLOB]` + = `Bitvector[128]` = 16 bytes packed. Double-check that's the + reading the Amsterdam spec wants (it currently describes it as + "16 bytes interpreted as a bitarray"). +4. **`should_override_builder` typing.** SSZ has `bool` but it's + a 1-byte field. Keeping it inside `BuiltPayload` (rather than + moving to a header) was the [refactor.md](./refactor.md) + decision; this sketch follows that. +5. **`PayloadStatus` enum encoding.** A `uint8` with sentinel + values matches the JSON-RPC enum; SSZ has no native enum type + so this is the cleanest mapping. Alternative: `Container { ... }` + wrapping a `uint8`. Open for discussion. +6. **`ExecutionPayloadBody` shared definition.** Today every fork + redefines `ExecutionPayloadBody` from scratch. The new spec + would benefit from a small set of fork-named containers + (`ExecutionPayloadBodyParis`, `ExecutionPayloadBodyShanghai`, + `ExecutionPayloadBodyAmsterdam`, …) with the URL `{fork}` + selecting which one. Not worked out here. +7. **Naming convention.** The legacy spec used `camelCase`; this + sketch uses `snake_case` to match consensus-specs. Worth + confirming. +8. **`ByteVector[BYTES_PER_BLOB]` vs `ByteList[BYTES_PER_BLOB]`.** + A blob is fixed-size (131072 bytes), so `ByteVector` is the + correct typing. Verify against consensus-specs to keep + alignment. diff --git a/src/engine/refactor.md b/src/engine/refactor.md new file mode 100644 index 000000000..f0aa6ac14 --- /dev/null +++ b/src/engine/refactor.md @@ -0,0 +1,960 @@ +# Engine API -- Refactor Proposal (REST + SSZ) + +> **Status:** Draft / discussion document. This file proposes a v2 of the +> Engine API that moves from JSON-RPC over a single endpoint to a +> resource-oriented HTTP/REST API where request and response bodies are +> SSZ-encoded. It also takes the opportunity to simplify the surface that +> has accumulated since Paris. +> +> **Target fork:** Amsterdam. The new API ships *as* the Amsterdam Engine +> API; clients implement it instead of `engine_*` JSON-RPC at the +> Amsterdam activation timestamp. + +This document is meant to be read alongside the existing fork-scoped specs +([Paris](./paris.md), [Shanghai](./shanghai.md), [Cancun](./cancun.md), +[Prague](./prague.md), [Osaka](./osaka.md), [Amsterdam](./amsterdam.md)). +Concrete byte-level structures are deferred to a later iteration; the goal +here is to align on the *shape* of the new API. + +--- + +## Table of contents + +- [Mapping from old → new](#mapping-from-old--new) +- [Resource model (overview)](#resource-model-overview) +- [Endpoints](#endpoints) + - [Payload submission](#payload-submission) + - [Forkchoice update](#forkchoice-update) + - [Payload retrieval](#payload-retrieval) + - [Historical bodies](#historical-bodies) + - [Blob pool](#blob-pool) + - [Capabilities & identification](#capabilities--identification) +- [Error model](#error-model) +- [Versioning model](#versioning-model) +- [Authentication](#authentication) +- [Transport & framing](#transport--framing) +- [SSZ encoding conventions](#ssz-encoding-conventions) +- [Message ordering & idempotency](#message-ordering--idempotency) +- [Motivation](#motivation) + - [Goals & non-goals](#goals--non-goals) + - [Why move away from JSON-RPC?](#why-move-away-from-json-rpc) + - [Why SSZ?](#why-ssz) + - [Simplifications & removed concepts](#simplifications--removed-concepts) + - [Summary of design decisions](#summary-of-design-decisions) + +> **Reading order note.** The endpoint sketches reference SSZ types +> like `Optional[T]`, `BodyEntry`, and `BlobEntry`. If a definition +> isn't immediately clear, jump to +> [SSZ encoding conventions](#ssz-encoding-conventions) and +> [Message ordering & idempotency](#message-ordering--idempotency) +> further down — they fully define the wire-level details. + +--- + +## Mapping from old → new + +If you're migrating from the JSON-RPC engine API, this is the lookup +table. Detail on each new endpoint follows in the sections below. + +| Old method | New endpoint | Notes | +| - | - | - | +| `engine_newPayloadV{1..5}` | `POST /{fork}/payloads` | `parentBeaconBlockRoot` and `executionRequests` folded into the SSZ envelope; `expectedBlobVersionedHashes` removed; `INVALID_BLOCK_HASH` removed from the status enum | +| `engine_forkchoiceUpdatedV{1..4}` | `POST /{fork}/forkchoice` | one atomic call; carries forkchoice state, optional `payload_attributes`, and (Amsterdam+) optional `custody_columns` | +| `engine_getPayloadV{1..6}` | `GET /{fork}/payloads/{id}` | poll-style, same semantics as today | +| `engine_getPayloadBodiesByHashV{1,2}` | `POST /{fork}/bodies/hash` | `{fork}` selects the response *schema* (not the era of requested blocks); `POST` because hash lists are too large for URLs | +| `engine_getPayloadBodiesByRangeV{1,2}` | `GET /{fork}/bodies?from=...&count=...` | `{fork}` selects the response schema | +| `engine_getBlobsV1` | `POST /blobs/v1` | independently versioned; legacy version numbers carry forward | +| `engine_getBlobsV2` | `POST /blobs/v2` | all-or-nothing cell proofs | +| `engine_getBlobsV3` | `POST /blobs/v3` | partial-response cell proofs | +| `engine_getBlobsV4` | `POST /blobs/v4` | cell-range selection | +| `engine_getClientVersionV1` | `GET /identity` + `X-Engine-Client-Version` request header | unscoped | +| `engine_exchangeCapabilities` | `GET /capabilities` | unscoped | +| `engine_exchangeTransitionConfigurationV1` | *removed* | already deprecated since Cancun | + +--- + +## Resource model (overview) + +Hot-path endpoints are scoped under `/engine/v2/{fork}/...`. Diagnostic +endpoints are unscoped. + +| Resource | Endpoint | Purpose | +| - | - | - | +| Payload | `POST /engine/v2/{fork}/payloads` | Submit a payload received from the CL gossip network for the EL to validate / import. Replaces `engine_newPayload`. | +| Payload | `GET /engine/v2/{fork}/payloads/{payloadId}` | Retrieve a built payload by id. Replaces `engine_getPayload`. CL polls when it wants a fresher snapshot. | +| Forkchoice | `POST /engine/v2/{fork}/forkchoice` | Atomic forkchoice update: update head/safe/finalized, optionally start a payload build, optionally update custody set. Replaces `engine_forkchoiceUpdated`. | +| Bodies | `POST /engine/v2/{fork}/bodies/hash` | Replaces `engine_getPayloadBodiesByHash`. Fork-scoped: `{fork}` selects the *response schema*, not the fork of the requested blocks. | +| Bodies | `GET /engine/v2/{fork}/bodies?from=N&count=M` | Replaces `engine_getPayloadBodiesByRange`. Fork-scoped on response shape. | +| Blob pool | `POST /engine/v2/blobs/v{1..4}` | Replaces `engine_getBlobsV{1..4}`. The `vN` segment carries forward the legacy version numbers; `/v4` is the Amsterdam cell-range variant, `/v1` is the original Cancun whole-blob variant, and intermediate revisions live alongside. ELs MUST serve at least the current-fork revision (`/v4` for Amsterdam) and MAY serve older revisions alongside. | +| Capabilities | `GET /engine/v2/capabilities` | Replaces `engine_exchangeCapabilities`. Unscoped; advertises supported forks, `/blobs/vN` revisions, and per-endpoint request-size limits. | +| Identity | `GET /engine/v2/identity` | Replaces `engine_getClientVersion`. Unscoped. | + +Every hot-path body uses SSZ; every metadata endpoint uses JSON. + +--- + +## Endpoints + +### Payload submission + +#### `POST /engine/v2/{fork}/payloads` + +Replaces `engine_newPayloadV{1..5}`. + +- **Request body:** SSZ-encoded `ExecutionPayloadEnvelope`, a container + that bundles together everything that today travels alongside the + payload as separate JSON-RPC params: + + ``` + ExecutionPayloadEnvelope { + payload: ExecutionPayload # the fork's payload SSZ container + parent_beacon_block_root: Root # was a separate param since Cancun + execution_requests: List[Bytes, MAX_REQUESTS] # was a separate param since Prague + } + ``` + + `expected_blob_versioned_hashes` is **removed**: it was a + defense-in-depth cross-check, but the block-hash check already covers + the transactions, so the EL recomputes the array from + `payload.transactions` during validation and a mismatch between CL + and EL views surfaces as `INVALID` exactly as before. + +- **Response body:** SSZ-encoded `PayloadStatus`: + + ``` + PayloadStatus { + status: uint8 # VALID=1, INVALID=2, SYNCING=3, ACCEPTED=4 + latest_valid_hash: Optional[Hash32] + validation_error: Optional[String] + } + ``` + + `INVALID_BLOCK_HASH` is dropped (already supplanted by `INVALID`). + `ACCEPTED` is **kept** — CLs rely on it during sync to acknowledge + side-branch payloads that are well-formed but don't extend the + canonical chain. + +- **HTTP status:** `200 OK` for any of the four validation outcomes. + Validation results are not transport errors. + +### Forkchoice update + +#### `POST /engine/v2/{fork}/forkchoice` + +Replaces `engine_forkchoiceUpdatedV{1..4}`. This is the **single +atomic** call that updates the EL's forkchoice state, optionally +triggers a payload build, and (post-Amsterdam) optionally updates the +CL's custody set. Atomicity matters: the CL relies on the EL having +applied the new head before — and only if — the build is started, and +on the build being keyed against the freshly-applied head. + +- **Request body:** SSZ-encoded `ForkchoiceUpdate`: + + ``` + ForkchoiceUpdate { + forkchoice_state: ForkchoiceState # head / safe / finalized + payload_attributes: Optional[PayloadAttributes] # if present, start a build on top of head + custody_columns: Optional[Bitvector[CELLS_PER_EXT_BLOB]] # Amsterdam+, optional + } + ``` + + All three fields are processed in one transaction: the EL MUST apply + the forkchoice state, then (if `payload_attributes` is present and + the new head is `VALID`) start the build, then (if `custody_columns` + is present) update the custody set, all before returning. If the + forkchoice update fails, no build is started and no custody change + is applied. + +- **Response body:** SSZ-encoded `ForkchoiceUpdateResponse`: + + ``` + ForkchoiceUpdateResponse { + payload_status: PayloadStatus # VALID | INVALID | SYNCING + payload_id: Optional[Bytes8] # server-assigned opaque token; set iff a build was started + } + ``` + + The `payload_id` is an **opaque server-assigned token**. The EL + chooses how to mint it (counter, random, hash-tree-root over the + attributes — anything). CLs MUST treat it as opaque bytes and MUST + NOT recompute or validate its contents. This is a change from + today's behavior where both sides derived an 8-byte hash over + `(headBlockHash, payloadAttributes)`. + +- **HTTP status:** `200 OK` for all three payload-status outcomes. + `409 Conflict` is returned for an inconsistent forkchoice state + (today's `-38002`); `422 Unprocessable Entity` for invalid + `payload_attributes` (today's `-38003`); `409 Conflict` for a too-deep + reorg (today's `-38006`). + +- **Skip-allowed semantics:** the EL MAY skip applying the forkchoice + state and instead return `{VALID, latest_valid_hash: head}` if the + new `head` is a `VALID` ancestor of the latest known finalized block. + This preserves the existing Paris-spec rule (point 2 of the + `engine_forkchoiceUpdated` specification) and is deliberate: a CL + that emits a malformed or stale FCU referencing a head behind + finalization should not be able to roll the EL back. We keep the + behaviour that has caught buggy CLs in the past. + +- **Stale-fork URL:** an FCU at `/engine/v2/{fork}/forkchoice` + referencing a `head` from an earlier fork is **allowed**, *as long + as `payload_attributes` is absent*. The CL needs to update head / + safe / finalized across fork boundaries during sync and reorg + recovery, and the URL fork has no bearing on which historical + block can be referenced. + + If `payload_attributes` is present, the URL `{fork}` MUST match + the fork that the new payload would belong to (i.e. the fork + determined by `payload_attributes.timestamp`). Mismatch returns + `400 unsupported-fork`. Building a payload is the only operation + where the URL fork is load-bearing on shape, so it's the only one + we strictly police. + +- **Custody-set semantics** (Amsterdam+): the custody update runs + independently of the forkchoice processing flow, matching the + Amsterdam spec's "MUST run custody set update independently to the + fork choice update". An execution-time custody-set error MUST NOT + affect the `payload_status` returned for the forkchoice update. + A `custody_columns` value, once accepted, remains in effect until + the next `POST /forkchoice` whose body *also* contains a + `custody_columns` field. FCUs that omit the field leave the + custody set unchanged. + +- **No body cap.** `POST /forkchoice` bodies are bounded by the SSZ + schema's `MAX_*` constants (small for `ForkchoiceState` and + `PayloadAttributes`, fixed for `custody_columns`). No additional + HTTP-layer cap is imposed. + +### Payload retrieval + +#### `GET /engine/v2/{fork}/payloads/{payloadId}` + +Replaces `engine_getPayloadV{1..6}`. + +- **Response body:** SSZ-encoded `BuiltPayload`: + + ``` + BuiltPayload { + payload: ExecutionPayload + block_value: Uint256 + blobs_bundle: BlobsBundle + execution_requests: List[Bytes, MAX_REQUESTS] + should_override_builder: bool + } + ``` +- **404** if `payloadId` is unknown or expired. + +Polling semantics are unchanged from `engine_getPayload`: the CL calls +`GET /{fork}/payloads/{payloadId}` whenever it wants the latest +snapshot of the build. Each call returns the most recent version +available at the time of receipt; the EL MAY stop the build process +after serving a call. `payloadId` values are opaque server-assigned +tokens issued by `POST /forkchoice`. + +**Token TTL.** A `payloadId` is valid for **at least 10 minutes** +after its issuing `POST /forkchoice` returns. After 10 minutes the +EL MAY garbage-collect the token and respond `404 unknown-payload` +to subsequent `GET`s. ELs MUST NOT recycle a token within its TTL +(no collisions); after expiry the token namespace is free to reuse. +A CL that needs a fresh `payloadId` after expiry simply issues a new +`POST /forkchoice` with the same attributes. + +### Historical bodies + +These endpoints are **fork-scoped on the response schema, not on the +era of the requested blocks**. The `{fork}` segment tells the EL which +`ExecutionPayloadBody` shape to use when serialising the response. +A CL that has just upgraded to the Amsterdam schema can ask for +`/amsterdam/bodies/hash` and receive `block_access_list` populated +for Amsterdam blocks and `[]` (the SSZ optional sentinel — see +[SSZ encoding conventions](#ssz-encoding-conventions)) for older +blocks; a CL still on Cancun asks `/cancun/bodies/hash` and +gets responses serialised against the Cancun container, never seeing +the trailing `block_access_list` field at all. + +This is different from the `/payloads` and `/forkchoice` `{fork}` +segments, where the URL fork *must* match the timestamp of the +referenced block. For `/bodies` the URL fork is purely a schema +selector and the requester chooses freely. + +The blob endpoint takes yet another approach: it carries a `/vN` +revision instead of a `{fork}` segment, because blob protocol +evolution has historically not aligned with fork activations. See +the [Blob pool](#blob-pool) section. + +#### `POST /engine/v2/{fork}/bodies/hash` + +Replaces `engine_getPayloadBodiesByHashV{1,2}`. Uses `POST` so that +large hash lists travel in the request body rather than the URL. + +- **Request body:** SSZ-encoded `List[Hash32, MAX_BODIES_REQUEST]`. + +#### `GET /engine/v2/{fork}/bodies?from=N&count=M` + +Replaces `engine_getPayloadBodiesByRangeV{1,2}`. Range fits comfortably +in the URL. + +- **Response body** (both endpoints): SSZ-encoded + `List[BodyEntry, MAX_BODIES_REQUEST]`. Each `BodyEntry` carries an + `available: boolean` flag (false for unavailable / pruned blocks, + matching today's `null` semantics) and an `ExecutionPayloadBody` + serialised against the **`{fork}` schema from the URL**. Fields + introduced in `{fork}` or earlier are present (with `Optional[T]` + set to `None` for blocks predating the field's introduction); fields + introduced in forks newer than `{fork}` are absent from the + container entirely. See + [SSZ encoding conventions](#ssz-encoding-conventions). + +### Blob pool + +The blob endpoint is **independently versioned**: blobs are looked up +by versioned hash (not by fork), so the `{fork}` URL segment doesn't +help. But the blob protocol *has* evolved on its own clock — four +distinct semantics across two forks (V1 single proof in Cancun, V2 +cell proofs in Osaka, V3 partial responses, V4 cell-range selection +in Amsterdam). The new spec carries those legacy version numbers +forward onto the URL: `engine_getBlobsVN` becomes `POST /blobs/vN`. +ELs **MUST** serve at least the revision matching their current fork +(`/blobs/v4` for Amsterdam) and **MAY** serve any subset of older +revisions alongside; `GET /capabilities` advertises the actual list. + +This is a different versioning axis from the fork-scoped endpoints +(`/{fork}/payloads`, `/{fork}/forkchoice`, `/{fork}/bodies`). Those +track *consensus protocol* changes coupled to fork activations. +`/blobs/vN` tracks *engine-API blob protocol* changes that have +historically not aligned with fork activations. + +All revisions use `POST` so that 128 versioned hashes (8 KiB hex) +don't have to fit in the URL. All revisions return SSZ +`Optional[List[BlobEntry, MAX_BLOBS_REQUEST]]`, where the outer +`Optional` is the "all-or-nothing"/syncing channel (`None` = +"cannot serve this request, retry later or fall back") and each +`BlobEntry` carries an `available: boolean` per-entry flag for +per-blob misses on revisions that support partial responses. +Revision-specific contents live inside `BlobEntry.contents`. + +#### `POST /engine/v2/blobs/v1` + +Replaces `engine_getBlobsV1` (Cancun, single-proof whole-blob). + +- **Request body:** SSZ `List[VersionedHash, MAX_BLOBS_REQUEST]`. +- **Response `BlobEntry.contents`:** `BlobAndProofV1 { blob, proof }` + (one blob, one 48-byte KZG proof). +- Partial responses supported: missing blobs surface as + `available=false` per entry. The outer `Optional` returns `None` + only if the EL cannot serve the request at all (e.g. syncing). + +#### `POST /engine/v2/blobs/v2` + +Replaces `engine_getBlobsV2` (Osaka, all-or-nothing cell proofs). + +- **Request body:** SSZ `List[VersionedHash, MAX_BLOBS_REQUEST]`. +- **Response `BlobEntry.contents`:** `BlobAndProofV2 { blob, proofs }` + (one blob plus `CELLS_PER_EXT_BLOB` cell proofs). +- **All-or-nothing:** if any requested blob is missing, the outer + `Optional[List[...]]` returns `None`. Otherwise all entries have + `available=true`. This matches today's V2 semantics. + +#### `POST /engine/v2/blobs/v3` + +Replaces `engine_getBlobsV3` (Osaka, partial responses with cell +proofs). + +- **Request body:** same as `/v2`. +- **Response:** same `BlobEntry.contents` shape as `/v2`, but missing + blobs surface as `available=false` per entry rather than collapsing + the whole response to `None`. The outer `Optional` returns `None` + only when the EL cannot serve the request at all. + +#### `POST /engine/v2/blobs/v4` + +Replaces `engine_getBlobsV4` (Amsterdam, cell-range selection). + +- **Request body:** SSZ `BlobsV4Request`: + ``` + BlobsV4Request { + versioned_hashes: List[VersionedHash, MAX_BLOBS_REQUEST] + indices_bitarray: Bitvector[CELLS_PER_EXT_BLOB] # which cells to return + } + ``` +- **Response `BlobEntry.contents`:** `BlobCellsAndProofsV1` + (per-cell `blob_cells` and `proofs` arrays, with `Optional[T]` = + `[]` at indices where individual cells are unavailable). + +### Capabilities & identification + +#### `GET /engine/v2/capabilities` + +Returns JSON. The advertisement includes per-endpoint maximum request +sizes so the CL knows how many block-bodies / blob-cells / payloads +the server is willing to serve in one request: + +```json +{ + "supported_forks": ["paris", "shanghai", "cancun", "prague", "osaka", "amsterdam"], + "fork_scoped_endpoints": ["payloads", "forkchoice", "bodies"], + "independently_versioned": { "blobs": ["v1", "v2", "v3", "v4"] }, + "unscoped_endpoints": ["capabilities", "identity"], + "limits": { + "bodies.max_count": 128, + "blobs.max_versioned_hashes": 128, + "payload.max_bytes": 67108864 + } +} +``` + +The `independently_versioned` map advertises endpoints whose URL +carries an explicit `/vN` revision. ELs MAY support multiple +revisions concurrently (e.g. `["v1", "v2"]`); CLs pick whichever they +implement. + +#### `GET /engine/v2/identity` + +Returns JSON `ClientVersion[]` (same shape as today's +`engine_getClientVersionV1`). The CL identifies itself with a +`X-Engine-Client-Version` header on every request, removing the +mutual-exchange handshake. + +--- + +## Error model + +Errors are signalled by HTTP status code and an +`application/problem+json` body (RFC 7807). To keep responses compact, +we use only **two** of the RFC 7807 fields: + +- **`type`** (required) — relative URI identifying the problem class. + Stable across releases. CLs branch on this string. +- **`detail`** (optional) — human-readable, instance-specific message. + Omitted when the EL has nothing more to say than the `type` already + conveys (e.g. canned SSZ-decode failures). + +We deliberately drop the other RFC 7807 fields: + +- `title` would just duplicate `type` (RFC 7807 says it SHOULD NOT + vary between occurrences of the same `type`); CLs can render their + own from a static `type → title` map. +- `status` duplicates the HTTP status line. +- `instance` adds a per-request URI; operators get correlation from + logs already. + +There is **no** legacy `engine_code` extension. CLs migrating from +the JSON-RPC API map old codes to new `type` strings via the table +below; after migration the codes are gone. + +| HTTP status | `type` | Old JSON-RPC code | When | +| - | - | - | - | +| 400 Bad Request | `/engine-api/errors/parse-error` | -32700 | Body is not valid JSON / SSZ | +| 400 Bad Request | `/engine-api/errors/invalid-request` | -32600 | Request shape is wrong (missing required field, etc.) | +| 400 Bad Request | `/engine-api/errors/ssz-decode-error` | (new) | SSZ decode failed; canned error, no `detail` | +| 400 Bad Request | `/engine-api/errors/unsupported-fork` | -38005 | URL `{fork}` is not supported by this EL | +| 404 Not Found | `/engine-api/errors/method-not-found` | -32601 | URL does not match any endpoint | +| 404 Not Found | `/engine-api/errors/unknown-payload` | -38001 | `payloadId` does not exist | +| 409 Conflict | `/engine-api/errors/invalid-forkchoice` | -38002 | Forkchoice state is inconsistent (e.g. finalized not ancestor of head) | +| 409 Conflict | `/engine-api/errors/reorg-too-deep` | -38006 | Reorg depth exceeds the EL's limit | +| 413 Payload Too Large | `/engine-api/errors/request-too-large` | -38004 | Body exceeds an advertised `limits.*` value | +| 422 Unprocessable Entity | `/engine-api/errors/invalid-body` | -32602 | Body decoded fine but has invalid values | +| 422 Unprocessable Entity | `/engine-api/errors/invalid-attributes` | -38003 | `payload_attributes` validation failed | +| 500 Internal Server Error | `/engine-api/errors/internal` | -32603 / -32000 | Unrecoverable server error; `detail` carries the message | + +`type` URIs are written as **relative references** rooted at +`/engine-api/errors/...`. RFC 7807 allows relative URIs, and the +short form keeps error bodies small without losing identifier +stability. CLs MUST treat them as opaque strings — they MUST NOT +attempt to dereference them. + +Example error body: + +```json +{ + "type": "/engine-api/errors/invalid-forkchoice", + "detail": "finalized 0xab.. is not an ancestor of head 0xcd.." +} +``` + +Canned error (no `detail`): + +```json +{ "type": "/engine-api/errors/ssz-decode-error" } +``` + +Validation outcomes for a payload (`VALID`, `INVALID`, `SYNCING`, +`ACCEPTED`) are **not** errors — they remain part of the response +body with HTTP `200 OK`. HTTP errors are reserved for transport, +format, and authentication problems. + +--- + +## Versioning model + +Three layers: + +1. **Major (`/v2`)** — bumped only for breaking transport changes + (e.g. moving away from REST, swapping SSZ for something else). +2. **Per-fork body schema** — selected via the `{fork}` URL segment + on hot-path endpoints (`/{fork}/payloads`, `/{fork}/forkchoice`, + `/{fork}/bodies`). Tracks consensus-protocol changes that ride + along with fork activations. +3. **Per-endpoint revisions** — selected via a `/vN` URL segment on + endpoints whose protocol evolves independently of the fork + schedule (currently just `/blobs/vN`). Tracks engine-API protocol + changes that don't align with fork activations. + +The server advertises which forks and which `/vN` revisions it +understands via `GET /engine/v2/capabilities`. + +`engine_exchangeCapabilities` is **removed**. Instead the server lists +its supported fork schemas and endpoint set in a single JSON document +at `/engine/v2/capabilities`. + +--- + +## Authentication + +Unchanged in spirit: JWT (HS256, 256-bit shared secret). Differences: + +- The token MUST be presented as `Authorization: Bearer ` on every + request. The HTTP/2 connection itself is not authenticated; each + request stream carries its own bearer token. This means a single + long-lived h2 connection between CL and EL is fine — token rotation + happens per-request, not per-connection. +- IPC (UNIX socket) authentication remains optional, as today. +- JWT claims: + - `iat` (required, unchanged from today: ±60s window) + - `id` (optional, unchanged) + - `clv` is **removed** — the CL version travels in the + `X-Engine-Client-Version` request header instead. Keeping it in + two places caused drift; the header is structured, cheap, and + surfaces in normal HTTP logs. +- **Trace propagation:** CLs MAY include a W3C `traceparent` header + on each request. ELs that record a `traceparent` SHOULD propagate + it into their own logs / spans so a slot-level trace can cross the + CL→EL boundary. Not required, not authenticated, purely diagnostic. + +--- + +## Transport & framing + +- **Protocol:** HTTP/2 is **required**. Both TCP and IPC transports + use **h2c** (HTTP/2 cleartext); JWT-on-every-request provides + authentication, so TLS termination is left to a reverse proxy if + the operator wants it. HTTP/2 multiplexing means a single CL→EL + connection can carry the full request mix (forkchoice, payload + submission, blob fetches, body fetches) without head-of-line + blocking. HTTP/1.1 is not supported. +- **Default port:** `8551`, shared with the legacy JSON-RPC engine API. + The two surfaces are distinguished by path: legacy JSON-RPC remains + at `/` (and accepts JSON-RPC method calls), the new API lives under + `/engine/v2/...`. The same JWT secret authenticates both. +- **Base path:** `/engine/v2/{fork}/...`. The `/v2` segment is the + major-protocol version; the `{fork}` segment selects the fork-scoped + body schema (`paris`, `shanghai`, `cancun`, `prague`, `osaka`, + `amsterdam`, …). Adding a fork = adding one path prefix and one set + of SSZ schemas. See [Versioning](#versioning-model). +- **Trailing slashes are forbidden.** `/engine/v2/payloads` is the + canonical form; `/engine/v2/payloads/` MUST return + `404 method-not-found`. No automatic redirect. +- **Request body encoding:** `application/octet-stream` carrying SSZ + bytes for hot-path endpoints. JSON for diagnostic / metadata + endpoints (capabilities, identity, error bodies). +- **Response body encoding:** SSZ for hot-path data, JSON + (`application/json`) for diagnostics and error bodies. +- **Compression:** Servers MAY support `Accept-Encoding: zstd, gzip`. + Not required to implement; CLs MUST tolerate uncompressed responses. + Blob bundles compress well, so operators are encouraged to enable + `zstd` where available. +- **Flow-control window:** servers and CLs **SHOULD** set HTTP/2 + `INITIAL_WINDOW_SIZE` to at least 1 MiB. Default 64 KiB causes + excessive flow-control round-trips for blob bundles and large + `getPayload` responses. `MAX_FRAME_SIZE` and `MAX_HEADER_LIST_SIZE` + use HTTP/2 defaults — not pinned by this spec. +- **Connection lifecycle:** CLs MAY open fresh h2 connections per + request or reuse a long-lived connection. JWT is per-request so + token rotation works the same way in both patterns. + +### Why fork-in-URL instead of method versioning? + +Today every change of a single field bumps the method version +(`engine_newPayloadV1..V5`). The new API puts the fork in the URL: + +``` +POST /engine/v2/amsterdam/payloads +Content-Type: application/octet-stream +Authorization: Bearer + + +``` + +The EL routes by fork segment, parses the body according to that fork's +SSZ schema, and returns a fork-shaped response. Adding a fork = adding +one path prefix and one set of SSZ schemas. URLs stay greppable and +discoverable in logs. + +--- + +## SSZ encoding conventions + +- **`Optional[T]` ≡ `List[T, 1]`.** SSZ has no native optional type; + we use a length-0-or-1 list as the convention (`[]` = absent, + `[t]` = present). The notation `Optional[T]` in this document is + syntactic sugar for `List[T, 1]`. We picked this over + `Union[None, T]` because `List` is universally supported across + SSZ libraries. +- **`String` ≡ `List[byte, MAX_ERROR_BYTES]`** (UTF-8). Empty list + is the empty string; use `Optional[String]` if absence must be + distinguishable from empty. +- **Endianness:** SSZ uints are **little-endian**. The JSON-RPC API + encoded `QUANTITY` values as big-endian hex, so anything that + carries a uint (`block_value`, `gas_used`, `gas_limit`, `timestamp`, + `base_fee_per_gas`, `excess_blob_gas`, `blob_gas_used`, + `block_number`, the `index`/`validatorIndex`/`amount` triple in + `Withdrawal`) flips byte order on the wire. +- **`MAX_*` constants** live in the fork-scoped SSZ schema files + (e.g. `MAX_TXS_PER_PAYLOAD`, `MAX_WITHDRAWALS_PER_PAYLOAD`, + `MAX_BAL_BYTES`, `MAX_VERSIONED_HASHES_PER_REQUEST`). + `MAX_ERROR_BYTES` is global and pinned at `1024` here. + +### Cross-fork response containers + +Endpoints that return data spanning multiple block-eras come in two +flavours: + +1. **Fork-scoped** (e.g. `/bodies`): the URL `{fork}` selects the + container schema. Within that schema, fields that didn't exist in + earlier block-eras are `Optional[T]` (= `[]` for those blocks). + The outer entry carries an explicit `available` flag so + "pruned / unavailable" stays distinct from "field-not-applicable": + + ``` + # /amsterdam/bodies/hash response + BodyEntry { + available: boolean + body: ExecutionPayloadBody + } + + ExecutionPayloadBody { + transactions: List[Transaction, MAX_TXS] + withdrawals: Optional[List[Withdrawal, MAX_WITHDRAWALS]] # [] pre-Shanghai + block_access_list: Optional[ByteList[MAX_BAL_BYTES]] # [] pre-Amsterdam or if pruned + } + ``` + + A CL on the Cancun schema calls `/cancun/bodies/hash` and + receives the Cancun container (no `block_access_list` field at + all). Old CLs never see schemas they don't know. + +2. **Independently versioned** (e.g. `/blobs/vN`): each revision is + its own container, no nullable optionals across revisions. Old + CLs keep using `/blobs/v1`; new shapes ship as `/blobs/vN+1` + alongside. + +Per-entry fork tags (a `Union` of fork-shaped variants) were +rejected: every fork would bump the union and break old decoders. + +--- + +## Message ordering & idempotency + +HTTP/2 multiplexes streams over a single connection and a server +handler may complete in any order. The Engine API is sensitive to +ordering, so we pin two rules explicitly: + +- **CL-driven ordering.** The CL is responsible for serialising + dependent requests. In particular: + - Only one `POST /forkchoice` may be in flight at a time. + - If a `POST /payloads` is logically before a `POST /forkchoice` + (or vice versa), the CL MUST wait for the first response before + issuing the second. + - The EL processes streams in receive order. h2 multiplexing + across independent CL→EL flows is fine; the CL MUST NOT rely on + the EL to reorder its own dependent requests. + + This matches today's [`common.md`](./common.md) "Message ordering" + guarantee in spirit; it makes explicit that h2 multiplexing does + not relax it. There is **no sequence number on the wire** — the + protocol stays simple and CL bugs that break ordering are CL bugs. + +- **Idempotency, narrowly defined.** Today's + [`paris.md`](./paris.md) #4 specifies idempotency only with respect + to `VALID | INVALID`: once a payload is decided one way, it cannot + flip. But `SYNCING → VALID`, `SYNCING → INVALID`, and + `ACCEPTED → VALID/INVALID` transitions are explicitly allowed — + the same payload submitted twice can return different statuses if + the EL has acquired more state in between. The new spec preserves + this: an EL MUST NOT short-circuit a retry by returning the cached + status, and a CL MUST NOT assume two responses to the same envelope + match. The only invariant is the `VALID ↔ INVALID` boundary. + +--- + +## Motivation + +The remainder of this document is rationale and reference material: +why we made the choices the spec encodes above, plus a consolidated +decision log for quick scanning. + +### Goals & non-goals + +#### Goals + +1. **Reduce wire size and parse cost.** SSZ-encoded bodies are 30–50% + smaller than hex-JSON for payload-shaped data and parse in linear time + without nibble decoding. This matters most for blob bundles (multi-MB + per slot) and the new `blockAccessList`. +2. **Stop the version sprawl.** Today every fork bumps every method that + touches a changed structure (`engine_newPayload` is at V5, + `engine_getPayload` at V6, etc.). The new API puts the fork in the + URL (`/engine/v2/{fork}/...`) so a single endpoint accepts whatever + schema that fork mandates; adding a fork = adding one path prefix + plus one set of SSZ schemas, not bumping every method name. +3. **Self-contained requests.** No more side-channel parameters + (`expectedBlobVersionedHashes`, `parentBeaconBlockRoot`, + `executionRequests`) that travel beside the payload — they live + inside the payload envelope or are unnecessary. +4. **Idiomatic HTTP.** Use HTTP status codes for transport-level outcomes, + `Content-Type` for negotiation, and a small problem-detail JSON body + for errors. + +#### Non-goals + +- Dropping the EL/CL split, changing trust boundaries, or moving CL state + into the EL (or vice versa). +- Removing JWT. Authentication is unchanged; only the *transport* of the + bearer token differs (HTTP `Authorization` header, same as today). +- Replacing `eth_*` JSON-RPC. The `eth` namespace stays JSON-RPC. This + document only refactors the `engine_*` namespace. +- Wire-perfect SSZ container definitions. The encoding *conventions* + are pinned in this document; the concrete field-by-field SSZ + containers per fork (e.g. the Amsterdam `ExecutionPayload` schema) + are deferred to a follow-up. + +### Why move away from JSON-RPC? + +JSON-RPC over HTTP has served the Engine API since Paris. The pain points +that prompt this refactor: + +- **Encoding overhead.** Every `DATA` field is a `0x`-prefixed lowercase + hex string. A 128 KiB blob becomes a 256 KiB+ string. With Osaka / + Fulu blob counts and the Amsterdam `blockAccessList`, payloads are + routinely multi-megabyte. +- **No content negotiation.** A new fork structure forces a new method + name (`engine_newPayloadV5`), even when the only change is one added + field. With a REST endpoint, the fork is part of the URL + (`/engine/v2/amsterdam/payloads`) and the body schema is selected by + routing, not by method-name suffix. +- **Side-channel params.** JSON-RPC's positional params encourage + bolting on extras like `parentBeaconBlockRoot` and + `executionRequests` next to the payload, instead of inside it. +- **Errors are non-standard.** `-38001..-38006` are bespoke and require + client-side mapping. HTTP status codes + a typed problem body are + universally understood. + +JSON-RPC is fine for the casual `eth_*` query API. For the hot path +between CL and EL, we want something denser and more disciplined. + +### Why SSZ? + +- The CL already speaks SSZ natively for its block, attestation, blobs, + KZG, and request structures. The CL today **converts SSZ → JSON → + hex-strings** when it forwards a payload, then the EL parses hex-JSON + back to bytes. This conversion is pure overhead and has been a + recurring source of subtle field-encoding bugs (e.g. the + `withdrawals.amount` LE-vs-BE note in shanghai.md). +- SSZ's fixed/variable-length distinction lets us validate sizes + cheaply at the transport layer. +- It's already what consensus-specs uses to define `ExecutionPayload`, + `Withdrawal`, `BlobsBundle`, etc. We'd be aligning, not inventing. + +We keep JSON available for **error bodies, capability discovery, and +client identification** because those are ergonomic to debug with `curl` +and not on the hot path. + +### Simplifications & removed concepts + +1. **`expectedBlobVersionedHashes`** — **removed**. The block-hash check + already covers the transactions, so the EL recomputes the array + from `payload.transactions` during validation and surfaces a + mismatch as `INVALID`. The CL no longer sends a redundant copy. +2. **`INVALID_BLOCK_HASH`** — **removed** from the enum. Already + supplanted by `INVALID` since Shanghai. +3. **`ACCEPTED`** — **kept**. CLs use this status during sync to + acknowledge well-formed side-branch payloads that don't extend the + canonical chain. +4. **`shouldOverrideBuilder`** — kept, lives inside the SSZ + `BuiltPayload` body. (Considered moving to a response header but it + complicates the SSZ canonicalisation; better inside the body.) +5. **`engine_exchangeCapabilities`** as a polling handshake — replaced + by a single `GET /capabilities`. +6. **`engine_exchangeTransitionConfigurationV1`** — dropped. Already + deprecated since Cancun. +7. **`payloadId` derivation** — today both sides recompute an 8-byte + hash over `(headBlockHash, payloadAttributes)`. The new + `POST /forkchoice` returns `payload_id` directly in the response; + it is an **opaque server-assigned token**. The EL chooses how to + mint it; CLs MUST treat it as opaque bytes. +8. **The split between `engine_*` namespace and the `eth_*` subset + the EL must expose** — out of scope for this refactor; the `eth_*` + namespace stays JSON-RPC. +9. **Per-method `timeout` SHOULDs** — replaced with HTTP-standard + request timeouts and `Retry-After` semantics on 503. + +### Summary of design decisions + +This is the consolidated decision log. Every item below is normative +and is also detailed in the relevant section earlier in the document; +the summary exists for quick scanning. + +#### Scope + +- **Target fork:** Amsterdam. The new API ships *as* the Amsterdam + Engine API. Pre-Amsterdam timestamps continue to be served by the + legacy JSON-RPC API on the same port; clients run both surfaces. +- **Backwards compatibility** is out of scope. The legacy JSON-RPC + engine API is left in place by clients; this spec does not require + or forbid sunset. +- **`eth_*` JSON-RPC subset** (`eth_blockNumber`, `eth_call`, + `eth_chainId`, `eth_getCode`, `eth_getBlockByHash`, + `eth_getBlockByNumber`, `eth_getLogs`, `eth_sendRawTransaction`, + `eth_syncing`) is **not** mirrored under `/engine/v2/...`. CLs that + need state / log access continue to call them via the legacy + JSON-RPC root. + +#### Transport + +- **HTTP/2 required**, h2c (cleartext) for both TCP and IPC. No + HTTP/1.1 fallback. JWT-on-every-request authenticates; TLS + termination is left to a reverse proxy. +- **IPC** is h2c over UNIX socket — same paths and headers as TCP, + single code path. +- **Default port `8551`**, shared with the legacy JSON-RPC API + (distinguished by path). +- **Trailing slashes are forbidden** — return `404 method-not-found`. +- **Flow-control:** SHOULD set `INITIAL_WINDOW_SIZE` ≥ 1 MiB. + `MAX_FRAME_SIZE` and `MAX_HEADER_LIST_SIZE` use HTTP/2 defaults. +- **Connection lifecycle:** CLs MAY open fresh h2 connections per + request or reuse a long-lived connection. +- **Compression:** `zstd` and `gzip` MAY be implemented. CLs MUST + tolerate uncompressed responses. + +#### Versioning + +- **Fork-scoped endpoints:** `/{fork}/payloads`, `/{fork}/forkchoice`, + `/{fork}/bodies`. Fork in the URL, no `Eth-Consensus-Version` + header. +- **Independently versioned endpoints:** `/blobs/vN`. Legacy + `engine_getBlobsVN` numbers carry forward onto the URL. ELs MUST + serve at least the revision matching their current fork + (`/blobs/v4` for Amsterdam) and MAY serve older revisions + alongside. Future blob-shape changes ship as `/blobs/v5`, `/v6`, + etc. +- **Unscoped endpoints:** `/capabilities`, `/identity`. +- **Major version `/v2`** is bumped only for breaking transport + changes (e.g. dropping REST or SSZ). + +#### Encoding + +- **Hot-path bodies use SSZ.** Diagnostic / metadata endpoints + (`/capabilities`, `/identity`, error bodies) use JSON. +- **`Optional[T]` ≡ `List[T, 1]`** (length 0 = absent, length 1 = + present). Universally supported by SSZ libraries. +- **Strings ≡ `List[byte, MAX_ERROR_BYTES]`**, `MAX_ERROR_BYTES = 1024`. +- **Endianness:** SSZ uints are little-endian. This flips byte order + vs the JSON-RPC `QUANTITY` type for `block_value`, `gas_used`, + `timestamp`, `base_fee_per_gas`, `excess_blob_gas`, + `blob_gas_used`, `block_number`, and the + `index`/`validatorIndex`/`amount` triple in `Withdrawal`. +- **`MAX_*` constants** are defined in fork-scoped SSZ schema files; + `MAX_ERROR_BYTES` is global. +- **Cross-fork response containers** come in two flavours: + fork-scoped (`/bodies`) uses the URL `{fork}` to pick a schema, + with `Optional[T]` for fields absent in pre-fork blocks; + independently versioned (`/blobs/vN`) gives each revision its own + dedicated container. Both wrap their entries in + `BodyEntry { available, body }` / `BlobEntry { available, contents }` + with an outer `Optional[List[...]]` for the syncing / + all-or-nothing channel. Per-entry fork tags were rejected. + +#### Error model + +- **RFC 7807 with two fields:** `type` (required, relative URI rooted + at `/engine-api/errors/...`) and `detail` (optional). Drop `title`, + `status`, `instance`, `engine_code`. +- **CLs MUST NOT dereference `type`** — opaque strings. +- **SSZ-decode failures** are a canned `400 Bad Request` with + `type=/engine-api/errors/ssz-decode-error`, no `detail`. + +#### Ordering & idempotency + +- **CL-driven ordering.** Only one `POST /forkchoice` in flight at a + time; `POST /payloads` ordered with respect to surrounding FCUs by + the CL. No sequence number on the wire. +- **Idempotency is narrow.** `VALID ↔ INVALID` cannot flip. All + other transitions (`SYNCING → VALID/INVALID`, + `ACCEPTED → VALID/INVALID`) are allowed; ELs MUST NOT short-circuit + retries. + +#### Forkchoice update (`POST /{fork}/forkchoice`) + +- **Single atomic call** carrying forkchoice state, optional + `payload_attributes`, and optional `custody_columns`. +- **Skip-allowed semantics:** EL MAY skip applying state when the + new `head` is a `VALID` ancestor of the latest finalized block, + guarding against malformed CL FCUs. +- **Stale-fork URL** is allowed when `payload_attributes` is absent; + with `payload_attributes` present, URL `{fork}` MUST match the + timestamp's fork (otherwise `400 unsupported-fork`). +- **No HTTP-layer body cap** beyond SSZ `MAX_*` constants. +- **Custody-set updates** run independently of the forkchoice flow; + custody errors do not affect `payload_status`. +- **Custody-set lifetime:** set until the next FCU that includes a + `custody_columns` field. FCUs that omit it leave the set unchanged. + +#### Payload submission (`POST /{fork}/payloads`) + +- **`expectedBlobVersionedHashes` removed.** EL recomputes from + `payload.transactions`; block-hash check covers transactions. +- **`INVALID_BLOCK_HASH` removed** from the status enum. +- **`ACCEPTED` kept** — CLs use it during sync. +- **Transaction min-length** ("at least 1 byte") remains a + receiver-side validation rule, not an SSZ schema invariant. + +#### Payload retrieval (`GET /{fork}/payloads/{payloadId}`) + +- **Poll-only**, same semantics as today's `engine_getPayload`. No + SSE / long-poll. +- **`payload_id` is an opaque server-assigned token** issued by + `POST /forkchoice`. CLs MUST NOT recompute or validate it. +- **`payload_id` TTL ≥ 10 minutes.** After expiry the EL MAY GC and + reuse the token namespace; within the TTL no collisions. +- **`shouldOverrideBuilder`** lives inside the SSZ `BuiltPayload` + body. + +#### Authentication & telemetry + +- **JWT (HS256, 256-bit secret)** unchanged in spirit, presented as + `Authorization: Bearer ` on every request. +- **JWT claims:** `iat` required (±60s), `id` optional, **`clv` + removed**. +- **`X-Engine-Client-Version`** is the canonical CL version channel. +- **`traceparent`** (W3C trace context) is supported but optional. + +#### Operations + +- **Multi-CL setups** are operator-managed. The spec does not track + CL identity or restrict who calls `POST /forkchoice`. Today's "one + writer, many readers" pattern carries forward unchanged. +- **`GET /capabilities`** advertises supported forks, fork-scoped + endpoints, independently-versioned endpoints (with the available + `/vN` list), unscoped endpoints, and per-endpoint maximum request + sizes. + +#### Removed concepts + +- `engine_exchangeCapabilities` — replaced by `GET /capabilities`. +- `engine_exchangeTransitionConfigurationV1` — already deprecated + since Cancun. +- Per-method `timeout` SHOULDs — replaced by HTTP-standard request + timeouts and `Retry-After` semantics on 503. +- The mutual-exchange handshake of `engine_getClientVersionV1` — + replaced by one-way `GET /identity` plus the + `X-Engine-Client-Version` request header. From e1861e48788a0f249cbcf23e954f4ddc47fa4991 Mon Sep 17 00:00:00 2001 From: MariusVanDerWijden Date: Fri, 8 May 2026 11:31:49 +0200 Subject: [PATCH 2/6] engine: cleanup --- src/engine/refactor-ssz.md | 35 +++++-- src/engine/refactor.md | 183 +++++++++++++------------------------ 2 files changed, 90 insertions(+), 128 deletions(-) diff --git a/src/engine/refactor-ssz.md b/src/engine/refactor-ssz.md index 101ef5042..f9c2d3dc4 100644 --- a/src/engine/refactor-ssz.md +++ b/src/engine/refactor-ssz.md @@ -302,27 +302,46 @@ BodyEntry { available: Boolean body: ExecutionPayloadBody } +``` + +`available` is `false` when the requested block is unavailable / +pruned, **or** when the block's timestamp falls outside the URL +fork's active range, **or** for range queries when the block number +is past the latest known block. When `available=false`, `body` is +zero-valued and CLs MUST ignore its contents. + +Each fork URL pairs with its own `ExecutionPayloadBody` schema. The +Amsterdam variant carries every field unconditionally: -# /amsterdam/bodies/... uses this Amsterdam-fork ExecutionPayloadBody +``` +# Amsterdam ExecutionPayloadBody (used by /amsterdam/bodies/...) ExecutionPayloadBody { transactions: List[ByteList[MAX_BYTES_PER_TX], MAX_TXS_PER_PAYLOAD] - withdrawals: Optional[List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD]] # [] pre-Shanghai - block_access_list: Optional[ByteList[MAX_BAL_BYTES]] # [] pre-Amsterdam or pruned + withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] + block_access_list: ByteList[MAX_BAL_BYTES] } ``` -A CL on the Cancun schema would call `/cancun/bodies/...` and receive -a Cancun-shaped `ExecutionPayloadBody` (no `block_access_list` field -at all). The Cancun-fork variant is sketched here for clarity: +Earlier-fork variants drop the fields their fork didn't have. For +reference: ``` -# /cancun/bodies/... ExecutionPayloadBody (for reference) +# Cancun ExecutionPayloadBody (used by /cancun/bodies/...) +ExecutionPayloadBody { + transactions: List[ByteList[MAX_BYTES_PER_TX], MAX_TXS_PER_PAYLOAD] + withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] +} + +# Paris ExecutionPayloadBody (used by /paris/bodies/...) ExecutionPayloadBody { transactions: List[ByteList[MAX_BYTES_PER_TX], MAX_TXS_PER_PAYLOAD] - withdrawals: Optional[List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD]] # [] pre-Shanghai } ``` +No `Optional[T]` cross-fork nullability anywhere — each fork URL +returns only blocks from its own era, so every field is always +present. + ### `POST /blobs/v1` Replaces `engine_getBlobsV1` (Cancun whole-blob). diff --git a/src/engine/refactor.md b/src/engine/refactor.md index f0aa6ac14..52d9d7d99 100644 --- a/src/engine/refactor.md +++ b/src/engine/refactor.md @@ -9,13 +9,6 @@ > **Target fork:** Amsterdam. The new API ships *as* the Amsterdam Engine > API; clients implement it instead of `engine_*` JSON-RPC at the > Amsterdam activation timestamp. - -This document is meant to be read alongside the existing fork-scoped specs -([Paris](./paris.md), [Shanghai](./shanghai.md), [Cancun](./cancun.md), -[Prague](./prague.md), [Osaka](./osaka.md), [Amsterdam](./amsterdam.md)). -Concrete byte-level structures are deferred to a later iteration; the goal -here is to align on the *shape* of the new API. - --- ## Table of contents @@ -85,12 +78,11 @@ endpoints are unscoped. | Forkchoice | `POST /engine/v2/{fork}/forkchoice` | Atomic forkchoice update: update head/safe/finalized, optionally start a payload build, optionally update custody set. Replaces `engine_forkchoiceUpdated`. | | Bodies | `POST /engine/v2/{fork}/bodies/hash` | Replaces `engine_getPayloadBodiesByHash`. Fork-scoped: `{fork}` selects the *response schema*, not the fork of the requested blocks. | | Bodies | `GET /engine/v2/{fork}/bodies?from=N&count=M` | Replaces `engine_getPayloadBodiesByRange`. Fork-scoped on response shape. | -| Blob pool | `POST /engine/v2/blobs/v{1..4}` | Replaces `engine_getBlobsV{1..4}`. The `vN` segment carries forward the legacy version numbers; `/v4` is the Amsterdam cell-range variant, `/v1` is the original Cancun whole-blob variant, and intermediate revisions live alongside. ELs MUST serve at least the current-fork revision (`/v4` for Amsterdam) and MAY serve older revisions alongside. | +| Blob pool | `POST /engine/v2/blobs/v{1..4}` | Replaces `engine_getBlobsV{1..4}`. The `vN` segment carries forward the legacy version numbers; `/v4` is the Amsterdam cell-range variant. | | Capabilities | `GET /engine/v2/capabilities` | Replaces `engine_exchangeCapabilities`. Unscoped; advertises supported forks, `/blobs/vN` revisions, and per-endpoint request-size limits. | | Identity | `GET /engine/v2/identity` | Replaces `engine_getClientVersion`. Unscoped. | Every hot-path body uses SSZ; every metadata endpoint uses JSON. - --- ## Endpoints @@ -101,9 +93,7 @@ Every hot-path body uses SSZ; every metadata endpoint uses JSON. Replaces `engine_newPayloadV{1..5}`. -- **Request body:** SSZ-encoded `ExecutionPayloadEnvelope`, a container - that bundles together everything that today travels alongside the - payload as separate JSON-RPC params: +- **Request body:** SSZ-encoded `ExecutionPayloadEnvelope` ``` ExecutionPayloadEnvelope { @@ -130,9 +120,6 @@ Replaces `engine_newPayloadV{1..5}`. ``` `INVALID_BLOCK_HASH` is dropped (already supplanted by `INVALID`). - `ACCEPTED` is **kept** — CLs rely on it during sync to acknowledge - side-branch payloads that are well-formed but don't extend the - canonical chain. - **HTTP status:** `200 OK` for any of the four validation outcomes. Validation results are not transport errors. @@ -141,19 +128,14 @@ Replaces `engine_newPayloadV{1..5}`. #### `POST /engine/v2/{fork}/forkchoice` -Replaces `engine_forkchoiceUpdatedV{1..4}`. This is the **single -atomic** call that updates the EL's forkchoice state, optionally -triggers a payload build, and (post-Amsterdam) optionally updates the -CL's custody set. Atomicity matters: the CL relies on the EL having -applied the new head before — and only if — the build is started, and -on the build being keyed against the freshly-applied head. +Replaces `engine_forkchoiceUpdatedV{1..4}`. - **Request body:** SSZ-encoded `ForkchoiceUpdate`: ``` ForkchoiceUpdate { forkchoice_state: ForkchoiceState # head / safe / finalized - payload_attributes: Optional[PayloadAttributes] # if present, start a build on top of head + payload_attributes: Optional[PayloadAttributes] # if present, start a build custody_columns: Optional[Bitvector[CELLS_PER_EXT_BLOB]] # Amsterdam+, optional } ``` @@ -177,9 +159,7 @@ on the build being keyed against the freshly-applied head. The `payload_id` is an **opaque server-assigned token**. The EL chooses how to mint it (counter, random, hash-tree-root over the attributes — anything). CLs MUST treat it as opaque bytes and MUST - NOT recompute or validate its contents. This is a change from - today's behavior where both sides derived an 8-byte hash over - `(headBlockHash, payloadAttributes)`. + NOT recompute or validate its contents. - **HTTP status:** `200 OK` for all three payload-status outcomes. `409 Conflict` is returned for an inconsistent forkchoice state @@ -202,6 +182,7 @@ on the build being keyed against the freshly-applied head. safe / finalized across fork boundaries during sync and reorg recovery, and the URL fork has no bearing on which historical block can be referenced. + TODO(MariusVanDerWijden) Is that really the case? If `payload_attributes` is present, the URL `{fork}` MUST match the fork that the new payload would belong to (i.e. the fork @@ -211,20 +192,14 @@ on the build being keyed against the freshly-applied head. we strictly police. - **Custody-set semantics** (Amsterdam+): the custody update runs - independently of the forkchoice processing flow, matching the - Amsterdam spec's "MUST run custody set update independently to the - fork choice update". An execution-time custody-set error MUST NOT - affect the `payload_status` returned for the forkchoice update. + independently of the forkchoice processing flow. An execution-time + custody-set error MUST NOT affect the `payload_status` returned for + the forkchoice update. A `custody_columns` value, once accepted, remains in effect until the next `POST /forkchoice` whose body *also* contains a `custody_columns` field. FCUs that omit the field leave the custody set unchanged. -- **No body cap.** `POST /forkchoice` bodies are bounded by the SSZ - schema's `MAX_*` constants (small for `ForkchoiceState` and - `PayloadAttributes`, fixed for `custody_columns`). No additional - HTTP-layer cap is imposed. - ### Payload retrieval #### `GET /engine/v2/{fork}/payloads/{payloadId}` @@ -251,36 +226,28 @@ available at the time of receipt; the EL MAY stop the build process after serving a call. `payloadId` values are opaque server-assigned tokens issued by `POST /forkchoice`. -**Token TTL.** A `payloadId` is valid for **at least 10 minutes** -after its issuing `POST /forkchoice` returns. After 10 minutes the -EL MAY garbage-collect the token and respond `404 unknown-payload` -to subsequent `GET`s. ELs MUST NOT recycle a token within its TTL -(no collisions); after expiry the token namespace is free to reuse. -A CL that needs a fresh `payloadId` after expiry simply issues a new -`POST /forkchoice` with the same attributes. +**Token TTL.** A `payloadId` is valid until either the payload was +retrieved by `GET /{fork}/payloads/{payloadId}` or another payload +was built via a forkchoice with payload attributes. ### Historical bodies -These endpoints are **fork-scoped on the response schema, not on the -era of the requested blocks**. The `{fork}` segment tells the EL which -`ExecutionPayloadBody` shape to use when serialising the response. -A CL that has just upgraded to the Amsterdam schema can ask for -`/amsterdam/bodies/hash` and receive `block_access_list` populated -for Amsterdam blocks and `[]` (the SSZ optional sentinel — see -[SSZ encoding conventions](#ssz-encoding-conventions)) for older -blocks; a CL still on Cancun asks `/cancun/bodies/hash` and -gets responses serialised against the Cancun container, never seeing -the trailing `block_access_list` field at all. - -This is different from the `/payloads` and `/forkchoice` `{fork}` -segments, where the URL fork *must* match the timestamp of the -referenced block. For `/bodies` the URL fork is purely a schema -selector and the requester chooses freely. - -The blob endpoint takes yet another approach: it carries a `/vN` -revision instead of a `{fork}` segment, because blob protocol -evolution has historically not aligned with fork activations. See -the [Blob pool](#blob-pool) section. +These endpoints are **fork-scoped on both the response schema and the +era of the returned blocks.** The `{fork}` segment tells the EL which +`ExecutionPayloadBody` schema to use, *and* limits the response to +blocks whose timestamp falls in `{fork}`'s active time range. A CL +fetching bodies that span a fork boundary issues separate requests +against each fork URL. + +Concretely: + +- `/cancun/bodies/hash` returns bodies *only* for blocks in the + Cancun time range. Requesting a Shanghai or Amsterdam hash yields + `available=false` for that entry. +- `/amsterdam/bodies/hash` returns bodies *only* for Amsterdam blocks. + All fields (including `block_access_list`) are unconditionally + present; older blocks the CL accidentally requested come back as + `available=false`. #### `POST /engine/v2/{fork}/bodies/hash` @@ -292,43 +259,38 @@ large hash lists travel in the request body rather than the URL. #### `GET /engine/v2/{fork}/bodies?from=N&count=M` Replaces `engine_getPayloadBodiesByRangeV{1,2}`. Range fits comfortably -in the URL. +in the URL. Block numbers outside the URL fork's active range come +back as `available=false`; if the requested range straddles a fork +boundary the CL re-issues against the next fork URL for the unfilled +suffix. - **Response body** (both endpoints): SSZ-encoded `List[BodyEntry, MAX_BODIES_REQUEST]`. Each `BodyEntry` carries an - `available: boolean` flag (false for unavailable / pruned blocks, - matching today's `null` semantics) and an `ExecutionPayloadBody` - serialised against the **`{fork}` schema from the URL**. Fields - introduced in `{fork}` or earlier are present (with `Optional[T]` - set to `None` for blocks predating the field's introduction); fields - introduced in forks newer than `{fork}` are absent from the - container entirely. See - [SSZ encoding conventions](#ssz-encoding-conventions). + `available: boolean` flag and an `ExecutionPayloadBody` serialised + against the **`{fork}` schema from the URL**. `available` is false + in any of the following cases: + - the block is unavailable / pruned, + - the block's timestamp falls outside the URL fork's active range, + - or for range queries, the block number is past the latest known + block. + + When `available=false`, the `body` field is zero-valued and CLs + MUST ignore its contents. See + [SSZ encoding conventions](#ssz-encoding-conventions) for the + `BodyEntry` wrapper definition. ### Blob pool -The blob endpoint is **independently versioned**: blobs are looked up -by versioned hash (not by fork), so the `{fork}` URL segment doesn't -help. But the blob protocol *has* evolved on its own clock — four -distinct semantics across two forks (V1 single proof in Cancun, V2 -cell proofs in Osaka, V3 partial responses, V4 cell-range selection -in Amsterdam). The new spec carries those legacy version numbers +The blob endpoint is **independently versioned**. +The new spec carries those legacy version numbers forward onto the URL: `engine_getBlobsVN` becomes `POST /blobs/vN`. ELs **MUST** serve at least the revision matching their current fork (`/blobs/v4` for Amsterdam) and **MAY** serve any subset of older revisions alongside; `GET /capabilities` advertises the actual list. -This is a different versioning axis from the fork-scoped endpoints -(`/{fork}/payloads`, `/{fork}/forkchoice`, `/{fork}/bodies`). Those -track *consensus protocol* changes coupled to fork activations. -`/blobs/vN` tracks *engine-API blob protocol* changes that have -historically not aligned with fork activations. - All revisions use `POST` so that 128 versioned hashes (8 KiB hex) don't have to fit in the URL. All revisions return SSZ -`Optional[List[BlobEntry, MAX_BLOBS_REQUEST]]`, where the outer -`Optional` is the "all-or-nothing"/syncing channel (`None` = -"cannot serve this request, retry later or fall back") and each +`Optional[List[BlobEntry, MAX_BLOBS_REQUEST]]`, where each `BlobEntry` carries an `available: boolean` per-entry flag for per-blob misses on revisions that support partial responses. Revision-specific contents live inside `BlobEntry.contents`. @@ -429,19 +391,6 @@ we use only **two** of the RFC 7807 fields: Omitted when the EL has nothing more to say than the `type` already conveys (e.g. canned SSZ-decode failures). -We deliberately drop the other RFC 7807 fields: - -- `title` would just duplicate `type` (RFC 7807 says it SHOULD NOT - vary between occurrences of the same `type`); CLs can render their - own from a static `type → title` map. -- `status` duplicates the HTTP status line. -- `instance` adds a per-request URI; operators get correlation from - logs already. - -There is **no** legacy `engine_code` extension. CLs migrating from -the JSON-RPC API map old codes to new `type` strings via the table -below; after migration the codes are gone. - | HTTP status | `type` | Old JSON-RPC code | When | | - | - | - | - | | 400 Bad Request | `/engine-api/errors/parse-error` | -32700 | Body is not valid JSON / SSZ | @@ -458,10 +407,7 @@ below; after migration the codes are gone. | 500 Internal Server Error | `/engine-api/errors/internal` | -32603 / -32000 | Unrecoverable server error; `detail` carries the message | `type` URIs are written as **relative references** rooted at -`/engine-api/errors/...`. RFC 7807 allows relative URIs, and the -short form keeps error bodies small without losing identifier -stability. CLs MUST treat them as opaque strings — they MUST NOT -attempt to dereference them. +`/engine-api/errors/...`. Example error body: @@ -620,10 +566,12 @@ Endpoints that return data spanning multiple block-eras come in two flavours: 1. **Fork-scoped** (e.g. `/bodies`): the URL `{fork}` selects the - container schema. Within that schema, fields that didn't exist in - earlier block-eras are `Optional[T]` (= `[]` for those blocks). - The outer entry carries an explicit `available` flag so - "pruned / unavailable" stays distinct from "field-not-applicable": + container schema *and* limits the response to blocks from that + fork's time range. Every field in the fork's body container is + unconditionally present (no `Optional[T]` for cross-fork + nullability); blocks outside the fork's range come back as + `available=false` on the outer entry instead of as a + zero-valued body: ``` # /amsterdam/bodies/hash response @@ -632,25 +580,24 @@ flavours: body: ExecutionPayloadBody } + # Amsterdam ExecutionPayloadBody — every field always present ExecutionPayloadBody { transactions: List[Transaction, MAX_TXS] - withdrawals: Optional[List[Withdrawal, MAX_WITHDRAWALS]] # [] pre-Shanghai - block_access_list: Optional[ByteList[MAX_BAL_BYTES]] # [] pre-Amsterdam or if pruned + withdrawals: List[Withdrawal, MAX_WITHDRAWALS] + block_access_list: ByteList[MAX_BAL_BYTES] } ``` - A CL on the Cancun schema calls `/cancun/bodies/hash` and + A CL fetching a Cancun-era block calls `/cancun/bodies/hash` and receives the Cancun container (no `block_access_list` field at - all). Old CLs never see schemas they don't know. + all, and no `Optional` wrapper on `withdrawals`). Cross-fork + ranges require multiple requests, one per fork URL. 2. **Independently versioned** (e.g. `/blobs/vN`): each revision is its own container, no nullable optionals across revisions. Old CLs keep using `/blobs/v1`; new shapes ship as `/blobs/vN+1` alongside. -Per-entry fork tags (a `Union` of fork-shaped variants) were -rejected: every fork would bump the union and break old decoders. - --- ## Message ordering & idempotency @@ -669,11 +616,6 @@ ordering, so we pin two rules explicitly: across independent CL→EL flows is fine; the CL MUST NOT rely on the EL to reorder its own dependent requests. - This matches today's [`common.md`](./common.md) "Message ordering" - guarantee in spirit; it makes explicit that h2 multiplexing does - not relax it. There is **no sequence number on the wire** — the - protocol stays simple and CL bugs that break ordering are CL bugs. - - **Idempotency, narrowly defined.** Today's [`paris.md`](./paris.md) #4 specifies idempotency only with respect to `VALID | INVALID`: once a payload is decided one way, it cannot @@ -866,8 +808,9 @@ the summary exists for quick scanning. - **`MAX_*` constants** are defined in fork-scoped SSZ schema files; `MAX_ERROR_BYTES` is global. - **Cross-fork response containers** come in two flavours: - fork-scoped (`/bodies`) uses the URL `{fork}` to pick a schema, - with `Optional[T]` for fields absent in pre-fork blocks; + fork-scoped (`/bodies`) uses the URL `{fork}` to pick *both* the + schema and the era of returned blocks (every body field always + present; out-of-era blocks come back as `available=false`); independently versioned (`/blobs/vN`) gives each revision its own dedicated container. Both wrap their entries in `BodyEntry { available, body }` / `BlobEntry { available, contents }` From 9dbd160d1fbd6df01ddf0c29647c03a8df16cfa7 Mon Sep 17 00:00:00 2001 From: MariusVanDerWijden Date: Fri, 8 May 2026 11:39:19 +0200 Subject: [PATCH 3/6] engine: cleanup --- src/engine/refactor.md | 29 ++++++++++++++++------------- 1 file changed, 16 insertions(+), 13 deletions(-) diff --git a/src/engine/refactor.md b/src/engine/refactor.md index 52d9d7d99..03bc010a7 100644 --- a/src/engine/refactor.md +++ b/src/engine/refactor.md @@ -9,6 +9,7 @@ > **Target fork:** Amsterdam. The new API ships *as* the Amsterdam Engine > API; clients implement it instead of `engine_*` JSON-RPC at the > Amsterdam activation timestamp. + --- ## Table of contents @@ -54,8 +55,8 @@ table. Detail on each new endpoint follows in the sections below. | `engine_newPayloadV{1..5}` | `POST /{fork}/payloads` | `parentBeaconBlockRoot` and `executionRequests` folded into the SSZ envelope; `expectedBlobVersionedHashes` removed; `INVALID_BLOCK_HASH` removed from the status enum | | `engine_forkchoiceUpdatedV{1..4}` | `POST /{fork}/forkchoice` | one atomic call; carries forkchoice state, optional `payload_attributes`, and (Amsterdam+) optional `custody_columns` | | `engine_getPayloadV{1..6}` | `GET /{fork}/payloads/{id}` | poll-style, same semantics as today | -| `engine_getPayloadBodiesByHashV{1,2}` | `POST /{fork}/bodies/hash` | `{fork}` selects the response *schema* (not the era of requested blocks); `POST` because hash lists are too large for URLs | -| `engine_getPayloadBodiesByRangeV{1,2}` | `GET /{fork}/bodies?from=...&count=...` | `{fork}` selects the response schema | +| `engine_getPayloadBodiesByHashV{1,2}` | `POST /{fork}/bodies/hash` | `{fork}` selects both the response schema and the era of returned blocks; `POST` because hash lists are too large for URLs | +| `engine_getPayloadBodiesByRangeV{1,2}` | `GET /{fork}/bodies?from=...&count=...` | `{fork}` selects both the response schema and the era of returned blocks | | `engine_getBlobsV1` | `POST /blobs/v1` | independently versioned; legacy version numbers carry forward | | `engine_getBlobsV2` | `POST /blobs/v2` | all-or-nothing cell proofs | | `engine_getBlobsV3` | `POST /blobs/v3` | partial-response cell proofs | @@ -76,13 +77,14 @@ endpoints are unscoped. | Payload | `POST /engine/v2/{fork}/payloads` | Submit a payload received from the CL gossip network for the EL to validate / import. Replaces `engine_newPayload`. | | Payload | `GET /engine/v2/{fork}/payloads/{payloadId}` | Retrieve a built payload by id. Replaces `engine_getPayload`. CL polls when it wants a fresher snapshot. | | Forkchoice | `POST /engine/v2/{fork}/forkchoice` | Atomic forkchoice update: update head/safe/finalized, optionally start a payload build, optionally update custody set. Replaces `engine_forkchoiceUpdated`. | -| Bodies | `POST /engine/v2/{fork}/bodies/hash` | Replaces `engine_getPayloadBodiesByHash`. Fork-scoped: `{fork}` selects the *response schema*, not the fork of the requested blocks. | -| Bodies | `GET /engine/v2/{fork}/bodies?from=N&count=M` | Replaces `engine_getPayloadBodiesByRange`. Fork-scoped on response shape. | +| Bodies | `POST /engine/v2/{fork}/bodies/hash` | Replaces `engine_getPayloadBodiesByHash`. `{fork}` selects both the response schema *and* the era of returned blocks; out-of-era blocks come back as `available=false`. | +| Bodies | `GET /engine/v2/{fork}/bodies?from=N&count=M` | Replaces `engine_getPayloadBodiesByRange`. Same fork scoping as `/bodies/hash`. | | Blob pool | `POST /engine/v2/blobs/v{1..4}` | Replaces `engine_getBlobsV{1..4}`. The `vN` segment carries forward the legacy version numbers; `/v4` is the Amsterdam cell-range variant. | | Capabilities | `GET /engine/v2/capabilities` | Replaces `engine_exchangeCapabilities`. Unscoped; advertises supported forks, `/blobs/vN` revisions, and per-endpoint request-size limits. | | Identity | `GET /engine/v2/identity` | Replaces `engine_getClientVersion`. Unscoped. | Every hot-path body uses SSZ; every metadata endpoint uses JSON. + --- ## Endpoints @@ -281,12 +283,12 @@ suffix. ### Blob pool -The blob endpoint is **independently versioned**. -The new spec carries those legacy version numbers -forward onto the URL: `engine_getBlobsVN` becomes `POST /blobs/vN`. -ELs **MUST** serve at least the revision matching their current fork -(`/blobs/v4` for Amsterdam) and **MAY** serve any subset of older -revisions alongside; `GET /capabilities` advertises the actual list. +The blob endpoint is **independently versioned**: legacy +`engine_getBlobsVN` numbers carry forward onto the URL, so +`engine_getBlobsVN` becomes `POST /blobs/vN`. ELs **MUST** serve at +least the revision matching their current fork (`/blobs/v4` for +Amsterdam) and **MAY** serve any subset of older revisions alongside; +`GET /capabilities` advertises the actual list. All revisions use `POST` so that 128 versioned hashes (8 KiB hex) don't have to fit in the URL. All revisions return SSZ @@ -822,7 +824,6 @@ the summary exists for quick scanning. - **RFC 7807 with two fields:** `type` (required, relative URI rooted at `/engine-api/errors/...`) and `detail` (optional). Drop `title`, `status`, `instance`, `engine_code`. -- **CLs MUST NOT dereference `type`** — opaque strings. - **SSZ-decode failures** are a canned `400 Bad Request` with `type=/engine-api/errors/ssz-decode-error`, no `detail`. @@ -867,8 +868,10 @@ the summary exists for quick scanning. SSE / long-poll. - **`payload_id` is an opaque server-assigned token** issued by `POST /forkchoice`. CLs MUST NOT recompute or validate it. -- **`payload_id` TTL ≥ 10 minutes.** After expiry the EL MAY GC and - reuse the token namespace; within the TTL no collisions. +- **`payload_id` lifetime is build-bound, not time-bound.** A token + remains valid until either the payload was retrieved by + `GET /{fork}/payloads/{payloadId}` or another payload was built + via a forkchoice with payload attributes. - **`shouldOverrideBuilder`** lives inside the SSZ `BuiltPayload` body. From 094ec192d7a2571acdb36bb64936c06ef4d688cd Mon Sep 17 00:00:00 2001 From: MariusVanDerWijden Date: Fri, 8 May 2026 11:54:33 +0200 Subject: [PATCH 4/6] engine: cleanup --- src/engine/refactor-ssz.md | 15 ++++++---- src/engine/refactor.md | 60 +++++++++++++++++++++++++++++++++++++- 2 files changed, 68 insertions(+), 7 deletions(-) diff --git a/src/engine/refactor-ssz.md b/src/engine/refactor-ssz.md index f9c2d3dc4..24fdbc451 100644 --- a/src/engine/refactor-ssz.md +++ b/src/engine/refactor-ssz.md @@ -188,14 +188,17 @@ Status enum: | Value | Name | Used by | | - | - | - | -| `1` | `VALID` | both | -| `2` | `INVALID` | both | -| `3` | `SYNCING` | both | -| `4` | `ACCEPTED` | `POST /payloads` only | +| `0` | `VALID` | both | +| `1` | `INVALID` | both | +| `2` | `SYNCING` | both | +| `3` | `ACCEPTED` | `POST /payloads` only | + +Numbering starts at `0` so a default-initialised SSZ `PayloadStatus` +deserialises as `VALID` rather than as a reserved sentinel. `INVALID_BLOCK_HASH` is removed (already supplanted by `INVALID`). -`POST /forkchoice` MUST return `1`/`2`/`3` only; CLs MUST treat a -`4` from `/forkchoice` as a protocol error. +`POST /forkchoice` MUST return `0`/`1`/`2` only; CLs MUST treat a +`3` from `/forkchoice` as a protocol error. `Optional[String]` resolves to `List[List[byte, MAX_ERROR_BYTES], 1]`. diff --git a/src/engine/refactor.md b/src/engine/refactor.md index 03bc010a7..af62ab16b 100644 --- a/src/engine/refactor.md +++ b/src/engine/refactor.md @@ -29,6 +29,7 @@ - [Transport & framing](#transport--framing) - [SSZ encoding conventions](#ssz-encoding-conventions) - [Message ordering & idempotency](#message-ordering--idempotency) +- [Security considerations](#security-considerations) - [Motivation](#motivation) - [Goals & non-goals](#goals--non-goals) - [Why move away from JSON-RPC?](#why-move-away-from-json-rpc) @@ -115,7 +116,7 @@ Replaces `engine_newPayloadV{1..5}`. ``` PayloadStatus { - status: uint8 # VALID=1, INVALID=2, SYNCING=3, ACCEPTED=4 + status: uint8 # VALID=0, INVALID=1, SYNCING=2, ACCEPTED=3 latest_valid_hash: Optional[Hash32] validation_error: Optional[String] } @@ -228,6 +229,17 @@ available at the time of receipt; the EL MAY stop the build process after serving a call. `payloadId` values are opaque server-assigned tokens issued by `POST /forkchoice`. +The EL keeps optimising the payload until the slot deadline, so +successive `GET`s against the same `{payloadId}` may return different +bytes. The EL **MUST** include `Cache-Control: no-store` on the +response, and intermediaries **MUST NOT** cache or revalidate this +resource. CLs **MUST NOT** treat the response as cacheable. + +**Path validation.** `{payloadId}` is a path segment carrying a hex- +encoded `Bytes8`. The EL **MUST** validate that the path segment is +well-formed (8 bytes, hex) before dispatching to lookup logic; a +malformed segment returns `400 invalid-request`. + **Token TTL.** A `payloadId` is valid until either the payload was retrieved by `GET /{fork}/payloads/{payloadId}` or another payload was built via a forkchoice with payload attributes. @@ -562,6 +574,27 @@ discoverable in logs. `MAX_BAL_BYTES`, `MAX_VERSIONED_HASHES_PER_REQUEST`). `MAX_ERROR_BYTES` is global and pinned at `1024` here. +### JSON-RPC type → SSZ type mapping + +For implementers porting from the JSON-RPC API, the legacy openrpc +base types map onto SSZ as follows: + +| JSON-RPC type | SSZ type | +| - | - | +| `address` (20 bytes) | `Bytes20` | +| `hash32` (32 bytes) | `Bytes32` | +| `bytes8` (8 bytes) | `Bytes8` | +| `bytes32` (32 bytes) | `Bytes32` | +| `bytes48` (48 bytes) | `Bytes48` | +| `bytes256` (256 bytes) | `ByteVector[256]` | +| `bytesMax32` (0–32 bytes) | `ByteList[32]` | +| `bytes` (variable-length) | `ByteList[MAX_*]` (context-dependent) | +| `uint64` | `uint64` | +| `uint256` | `uint256` | +| `BOOLEAN` | `boolean` | +| `Array of T` | `List[T, MAX_*]` (context-dependent) | +| `T \| null` | `Optional[T]` (= `List[T, 1]`) | + ### Cross-fork response containers Endpoints that return data spanning multiple block-eras come in two @@ -631,6 +664,31 @@ ordering, so we pin two rules explicitly: --- +## Security considerations + +SSZ `MAX_*` constants bound *on-chain validity*, not per-request +resource use. A naive decoder facing a crafted `Content-Length`, +length prefix, or offset can be coerced into large allocations or +scans before any semantic rejection. ELs implementing this API +**MUST**: + +- **Cap by `Content-Length`** against an endpoint-specific maximum + *before* reading the body when the header is present, and cap the + bytes read from the body in all cases. +- **Validate SSZ length prefixes and offsets** against the remaining + buffer size *before* allocating backing storage for variable-length + fields. +- **Apply per-endpoint operational caps** (reverse proxy, + server config) in addition to library-level checks. The advertised + `limits.*` values in `GET /capabilities` are an upper bound, not a + target — operators are encouraged to reject earlier. + +ELs **SHOULD** use well-tested SSZ libraries and fuzz-test SSZ +parsing extensively. JWT authentication is unchanged from the legacy +JSON-RPC API; all existing requirements apply. + +--- + ## Motivation The remainder of this document is rationale and reference material: From b6a0dd2e7f8531dc2d67da2b3880b80312bf5834 Mon Sep 17 00:00:00 2001 From: MariusVanDerWijden Date: Fri, 8 May 2026 12:11:27 +0200 Subject: [PATCH 5/6] engine: cleanup --- src/engine/refactor-ssz.md | 340 ++++++++++++++++++++++++++++++------- src/engine/refactor.md | 178 +++++++++++++++++-- 2 files changed, 440 insertions(+), 78 deletions(-) diff --git a/src/engine/refactor-ssz.md b/src/engine/refactor-ssz.md index 24fdbc451..76dcfa485 100644 --- a/src/engine/refactor-ssz.md +++ b/src/engine/refactor-ssz.md @@ -27,6 +27,13 @@ - [`PayloadAttributes` (Amsterdam)](#payloadattributes-amsterdam) - [`ForkchoiceState`](#forkchoicestate) - [`PayloadStatus`](#payloadstatus) +- [Per-fork container catalogue](#per-fork-container-catalogue) + - [`ExecutionPayload` per fork](#executionpayload-per-fork) + - [`PayloadAttributes` per fork](#payloadattributes-per-fork) + - [`ExecutionPayloadBody` per fork](#executionpayloadbody-per-fork) + - [`BlobsBundle` per revision](#blobsbundle-per-revision) + - [`BlobAndProof` per revision](#blobandproof-per-revision) + - [Identification & capabilities](#identification--capabilities) - [Endpoint containers](#endpoint-containers) - [`POST /amsterdam/payloads`](#post-amsterdampayloads) - [`POST /amsterdam/forkchoice`](#post-amsterdamforkchoice) @@ -58,26 +65,32 @@ ## `MAX_*` constants -These are sketch values — final values come from a follow-up that -matches the consensus-specs `Amsterdam` preset. They are listed here -for completeness so readers can size the on-wire bounds. - -| Constant | Sketch value | Where it's used | +| Constant | Value | Source | | - | - | - | -| `MAX_TXS_PER_PAYLOAD` | `1048576` | `ExecutionPayload.transactions` | -| `MAX_BYTES_PER_TX` | `1073741824` | element bound inside `transactions` | -| `MAX_WITHDRAWALS_PER_PAYLOAD` | `16` | `ExecutionPayload.withdrawals`, `PayloadAttributes.withdrawals` | -| `MAX_EXTRA_DATA_BYTES` | `32` | `ExecutionPayload.extra_data` | -| `MAX_BAL_BYTES` | TBD (EIP-7928) | `ExecutionPayload.block_access_list` | -| `MAX_EXECUTION_REQUESTS_PER_PAYLOAD` | TBD (EIP-7685) | `ExecutionPayloadEnvelope.execution_requests` | -| `MAX_BYTES_PER_EXECUTION_REQUEST` | TBD | element bound inside `execution_requests` | -| `MAX_VERSIONED_HASHES_PER_REQUEST` | `128` | `BlobsRequest.versioned_hashes` | -| `MAX_BODIES_REQUEST` | `128` | bodies request and response lists | -| `MAX_BLOBS_REQUEST` | `128` | blobs request and response lists | -| `MAX_BLOBS_PER_PAYLOAD` | `MAX_VERSIONED_HASHES_PER_REQUEST` | `BlobsBundle.commitments`, `.blobs` | -| `CELLS_PER_EXT_BLOB` | `128` (EIP-7594) | cell-proof and custody bitvectors | -| `BYTES_PER_BLOB` | `131072` | one blob (`4096 * 32`) | -| `MAX_ERROR_BYTES` | `1024` | `validation_error`, JSON error `detail` | +| `MAX_BYTES_PER_TX` | `2**30` (1,073,741,824) | [EIP-4844](https://eips.ethereum.org/EIPS/eip-4844) | +| `MAX_TXS_PER_PAYLOAD` | `2**20` (1,048,576) | [Bellatrix](https://github.com/ethereum/consensus-specs/blob/dev/specs/bellatrix/beacon-chain.md) | +| `MAX_WITHDRAWALS_PER_PAYLOAD` | `2**4` (16) | [Capella](https://github.com/ethereum/consensus-specs/blob/dev/specs/capella/beacon-chain.md) | +| `BYTES_PER_LOGS_BLOOM` | `256` | [Bellatrix](https://github.com/ethereum/consensus-specs/blob/dev/specs/bellatrix/beacon-chain.md) | +| `MAX_EXTRA_DATA_BYTES` | `2**5` (32) | [Bellatrix](https://github.com/ethereum/consensus-specs/blob/dev/specs/bellatrix/beacon-chain.md) | +| `MAX_BLOB_COMMITMENTS_PER_BLOCK` | `2**12` (4,096) | [Deneb](https://github.com/ethereum/consensus-specs/blob/dev/specs/deneb/beacon-chain.md) | +| `FIELD_ELEMENTS_PER_BLOB` | `4096` | [EIP-4844](https://eips.ethereum.org/EIPS/eip-4844) | +| `BYTES_PER_FIELD_ELEMENT` | `32` | [EIP-4844](https://eips.ethereum.org/EIPS/eip-4844) | +| `BYTES_PER_BLOB` | `FIELD_ELEMENTS_PER_BLOB * BYTES_PER_FIELD_ELEMENT` (131,072) | derived | +| `CELLS_PER_EXT_BLOB` | `128` | [EIP-7594](https://eips.ethereum.org/EIPS/eip-7594) | +| `BYTES_PER_CELL` | `BYTES_PER_BLOB / CELLS_PER_EXT_BLOB` (1,024) | derived | +| `MAX_BAL_BYTES` | `MAX_BYTES_PER_TX` | [EIP-7928](https://eips.ethereum.org/EIPS/eip-7928) (placeholder until EIP pins a tighter bound) | +| `MAX_EXECUTION_REQUESTS_PER_PAYLOAD` | `2**8` (256) | [EIP-7685](https://eips.ethereum.org/EIPS/eip-7685) | +| `MAX_BYTES_PER_EXECUTION_REQUEST` | `MAX_BYTES_PER_TX` | this spec (placeholder; reuse the tx bound) | +| `MAX_VERSIONED_HASHES_PER_REQUEST` | `128` | [Osaka](./osaka.md#engine_getblobsv2) | +| `MAX_BLOBS_REQUEST` | `MAX_VERSIONED_HASHES_PER_REQUEST` (128) | derived | +| `MAX_BODIES_REQUEST` | `2**5` (32) | [Shanghai](./shanghai.md#engine_getpayloadbodiesbyhashv1) | +| `MAX_ERROR_BYTES` | `1024` | this spec | +| `MAX_CLIENT_CODE_LENGTH` | `2` | this spec | +| `MAX_CLIENT_NAME_LENGTH` | `64` | this spec | +| `MAX_CLIENT_VERSION_LENGTH` | `64` | this spec | +| `MAX_CLIENT_VERSIONS` | `4` | this spec | +| `MAX_CAPABILITY_NAME_LENGTH` | `64` | this spec | +| `MAX_CAPABILITIES` | `64` | this spec | --- @@ -204,6 +217,214 @@ deserialises as `VALID` rather than as a reserved sentinel. --- +## Per-fork container catalogue + +Each fork URL (`/{fork}/payloads`, `/{fork}/forkchoice`, +`/{fork}/bodies`) uses its own SSZ container shape. ELs serving +`/cancun/...` MUST use the Cancun containers; ELs serving +`/amsterdam/...` MUST use the Amsterdam containers; etc. This section +catalogues every fork-scoped variant. + +### `ExecutionPayload` per fork + +Used by `POST /{fork}/payloads` (the inner `payload` field of +`ExecutionPayloadEnvelope`) and `GET /{fork}/payloads/{payloadId}` +(the inner `payload` field of `BuiltPayload`). + +``` +# Paris +ExecutionPayloadParis { + parent_hash: Hash32 + fee_recipient: Address + state_root: Hash32 + receipts_root: Hash32 + logs_bloom: Bloom + prev_randao: Bytes32 + block_number: Uint64 + gas_limit: Uint64 + gas_used: Uint64 + timestamp: Uint64 + extra_data: ByteList[MAX_EXTRA_DATA_BYTES] + base_fee_per_gas: Uint256 + block_hash: Hash32 + transactions: List[ByteList[MAX_BYTES_PER_TX], MAX_TXS_PER_PAYLOAD] +} + +# Shanghai = Paris + withdrawals +ExecutionPayloadShanghai { + ...Paris fields... + withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] +} + +# Cancun = Shanghai + blob_gas_used + excess_blob_gas +ExecutionPayloadCancun { + ...Shanghai fields... + blob_gas_used: Uint64 + excess_blob_gas: Uint64 +} + +# Prague = Cancun (no payload-shape change; execution_requests is at the envelope level) +ExecutionPayloadPrague = ExecutionPayloadCancun + +# Osaka = Prague (no payload-shape change; blobs bundle moved to BlobsBundleV2) +ExecutionPayloadOsaka = ExecutionPayloadPrague + +# Amsterdam = Cancun + block_access_list + slot_number +ExecutionPayloadAmsterdam { + ...Cancun fields... + block_access_list: ByteList[MAX_BAL_BYTES] + slot_number: Uint64 +} +``` + +The Amsterdam variant is identical to the +[`ExecutionPayload` (Amsterdam)](#executionpayload-amsterdam) shape +above; this section just makes the progression explicit. + +### `PayloadAttributes` per fork + +Used by the `payload_attributes` field of `ForkchoiceUpdate` (the +request body of `POST /{fork}/forkchoice`). + +``` +# Paris +PayloadAttributesParis { + timestamp: Uint64 + prev_randao: Bytes32 + suggested_fee_recipient: Address +} + +# Shanghai = Paris + withdrawals +PayloadAttributesShanghai { + ...Paris fields... + withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] +} + +# Cancun = Shanghai + parent_beacon_block_root +PayloadAttributesCancun { + ...Shanghai fields... + parent_beacon_block_root: Root +} + +# Prague = Cancun (no shape change) +PayloadAttributesPrague = PayloadAttributesCancun + +# Osaka = Cancun (no shape change) +PayloadAttributesOsaka = PayloadAttributesCancun + +# Amsterdam = Cancun + slot_number +PayloadAttributesAmsterdam { + ...Cancun fields... + slot_number: Uint64 +} +``` + +### `ExecutionPayloadBody` per fork + +Used by the inner `body` field of `BodyEntry`. Each fork URL serves +only blocks from its own time range, so every field is +unconditionally present (no `Optional[T]`). + +``` +# Paris +ExecutionPayloadBodyParis { + transactions: List[ByteList[MAX_BYTES_PER_TX], MAX_TXS_PER_PAYLOAD] +} + +# Shanghai = Paris + withdrawals +ExecutionPayloadBodyShanghai { + ...Paris fields... + withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] +} + +# Cancun, Prague, Osaka = Shanghai (no shape change for the body) +ExecutionPayloadBodyCancun = ExecutionPayloadBodyShanghai +ExecutionPayloadBodyPrague = ExecutionPayloadBodyShanghai +ExecutionPayloadBodyOsaka = ExecutionPayloadBodyShanghai + +# Amsterdam = Shanghai + block_access_list +ExecutionPayloadBodyAmsterdam { + ...Shanghai fields... + block_access_list: ByteList[MAX_BAL_BYTES] +} +``` + +### `BlobsBundle` per revision + +Used by the `blobs_bundle` field of `BuiltPayload`. The bundle shape +follows the consensus-specs progression (V1 single proof, V2 cell +proofs). + +``` +# Cancun (V1) — one proof per blob +BlobsBundleV1 { + commitments: List[Bytes48, MAX_BLOB_COMMITMENTS_PER_BLOCK] + proofs: List[Bytes48, MAX_BLOB_COMMITMENTS_PER_BLOCK] + blobs: List[ByteVector[BYTES_PER_BLOB], MAX_BLOB_COMMITMENTS_PER_BLOCK] +} + +# Osaka+ (V2) — CELLS_PER_EXT_BLOB cell proofs per blob +BlobsBundleV2 { + commitments: List[Bytes48, MAX_BLOB_COMMITMENTS_PER_BLOCK] + proofs: List[Bytes48, MAX_BLOB_COMMITMENTS_PER_BLOCK * CELLS_PER_EXT_BLOB] + blobs: List[ByteVector[BYTES_PER_BLOB], MAX_BLOB_COMMITMENTS_PER_BLOCK] +} +``` + +`BuiltPayload` for Cancun / Prague carries `BlobsBundleV1`; +Osaka / Amsterdam carries `BlobsBundleV2`. + +### `BlobAndProof` per revision + +Used by `BlobEntry.contents` on the blob-pool endpoints (`/blobs/vN`). + +``` +# /blobs/v1 — Cancun whole-blob, single proof +BlobAndProofV1 { + blob: ByteVector[BYTES_PER_BLOB] + proof: Bytes48 +} + +# /blobs/v2 and /blobs/v3 — Osaka cell proofs +BlobAndProofV2 { + blob: ByteVector[BYTES_PER_BLOB] + proofs: List[Bytes48, CELLS_PER_EXT_BLOB] +} + +# /blobs/v4 — Amsterdam cell-range selection (per-cell nullable) +BlobCellsAndProofs { + blob_cells: List[Optional[ByteVector[BYTES_PER_CELL]], CELLS_PER_EXT_BLOB] + proofs: List[Optional[Bytes48], CELLS_PER_EXT_BLOB] +} +``` + +### Identification & capabilities + +Used by `GET /identity` and `GET /capabilities`. These are JSON on +the wire (see [refactor.md § Resource model](./refactor.md#resource-model-overview)), +but we list the SSZ shapes for completeness so future versions could +switch to SSZ if desired. + +``` +ClientVersion { + code: ByteList[MAX_CLIENT_CODE_LENGTH] + name: ByteList[MAX_CLIENT_NAME_LENGTH] + version: ByteList[MAX_CLIENT_VERSION_LENGTH] + commit: Bytes4 +} + +IdentityResponse { + versions: List[ClientVersion, MAX_CLIENT_VERSIONS] +} + +CapabilitiesResponse { + capabilities: List[ByteList[MAX_CAPABILITY_NAME_LENGTH], MAX_CAPABILITIES] + # ... plus the structured fields documented in refactor.md +} +``` + +--- + ## Endpoint containers ### `POST /amsterdam/payloads` @@ -225,7 +446,7 @@ from `payload.transactions`). #### Response -`PayloadStatus` (full enum, `1`/`2`/`3`/`4`). +`PayloadStatus` (full enum, `0`/`1`/`2`/`3`). ### `POST /amsterdam/forkchoice` @@ -359,8 +580,13 @@ BlobsV1Request { #### Response +`200 OK` carries the SSZ body below; `204 No Content` (with empty +body) signals "EL cannot serve this request" (e.g. syncing). + ``` -BlobsV1Response = Optional[List[BlobV1Entry, MAX_BLOBS_REQUEST]] +BlobsV1Response { + entries: List[BlobV1Entry, MAX_BLOBS_REQUEST] +} BlobV1Entry { available: Boolean @@ -374,8 +600,8 @@ BlobAndProofV1 { ``` When `available == false`, `contents` carries zero-valued bytes (a -`BYTES_PER_BLOB`-byte zero blob and a 48-byte zero proof). The outer -`Optional` returns `[]` when the EL cannot serve the request at all. +`BYTES_PER_BLOB`-byte zero blob and a 48-byte zero proof) and CLs +MUST ignore them. ### `POST /blobs/v2` @@ -391,8 +617,14 @@ BlobsV2Request { #### Response +`200 OK` carries the SSZ body below; `204 No Content` (with empty +body) signals either "EL cannot serve this request at all" or +"at least one requested blob is missing" (V2 is all-or-nothing). + ``` -BlobsV2Response = Optional[List[BlobV2Entry, MAX_BLOBS_REQUEST]] +BlobsV2Response { + entries: List[BlobV2Entry, MAX_BLOBS_REQUEST] +} BlobV2Entry { available: Boolean # always true for /v2 (all-or-nothing); included for shape symmetry @@ -405,9 +637,7 @@ BlobAndProofV2 { } ``` -All-or-nothing: if any requested blob is missing, the outer -`Optional` returns `[]` and no per-entry data is sent. CLs that need -partial responses use `/v3`. +CLs that need partial responses use `/v3`. ### `POST /blobs/v3` @@ -418,13 +648,14 @@ proofs). #### Response -Same shape as `/v2` (`BlobV2Entry` reused), but missing blobs -surface as `available=false` per entry rather than collapsing the -whole response to `[]`. Outer `Optional` returns `[]` only when the -EL cannot serve the request at all (e.g. syncing). +`200 OK` carries the SSZ body; missing blobs surface as +`available=false` per entry. `204 No Content` only when the EL +cannot serve the request at all (e.g. syncing). ``` -BlobsV3Response = Optional[List[BlobV2Entry, MAX_BLOBS_REQUEST]] +BlobsV3Response { + entries: List[BlobV2Entry, MAX_BLOBS_REQUEST] +} ``` ### `POST /blobs/v4` @@ -442,8 +673,13 @@ BlobsV4Request { #### Response +`200 OK` carries the SSZ body; `204 No Content` signals "EL cannot +serve this request at all." + ``` -BlobsV4Response = Optional[List[BlobV4Entry, MAX_BLOBS_REQUEST]] +BlobsV4Response { + entries: List[BlobV4Entry, MAX_BLOBS_REQUEST] +} BlobV4Entry { available: Boolean @@ -469,38 +705,22 @@ old spec). ## Open sketch questions -These are the items left to decide before promoting this sketch to -the canonical Amsterdam SSZ schema: - -1. **`MAX_*` placeholder values.** Several constants above are - `TBD` or sketch-only. They need to be pinned to the - consensus-specs `Amsterdam` preset values once those land. -2. **`MAX_BAL_BYTES`.** EIP-7928 defines the BAL but doesn't yet - pin a numeric upper bound that's friendly for SSZ. We need a - concrete number; otherwise the SSZ schema can't round-trip. +1. **`MAX_BAL_BYTES`.** EIP-7928 defines the BAL but hasn't pinned + a numeric upper bound yet. The catalogue currently uses + `MAX_BYTES_PER_TX` as a placeholder; this should be tightened + once the EIP lands. +2. **`MAX_BYTES_PER_EXECUTION_REQUEST`.** EIP-7685 hasn't pinned a + numeric per-element bound either. Same placeholder pattern as + `MAX_BAL_BYTES`; needs a concrete value. 3. **`Bitvector` SSZ encoding for `indices_bitarray` and `custody_columns`.** Both are `Bitvector[CELLS_PER_EXT_BLOB]` = `Bitvector[128]` = 16 bytes packed. Double-check that's the reading the Amsterdam spec wants (it currently describes it as "16 bytes interpreted as a bitarray"). -4. **`should_override_builder` typing.** SSZ has `bool` but it's - a 1-byte field. Keeping it inside `BuiltPayload` (rather than - moving to a header) was the [refactor.md](./refactor.md) - decision; this sketch follows that. -5. **`PayloadStatus` enum encoding.** A `uint8` with sentinel +4. **`PayloadStatus` enum encoding.** A `uint8` with sentinel values matches the JSON-RPC enum; SSZ has no native enum type so this is the cleanest mapping. Alternative: `Container { ... }` - wrapping a `uint8`. Open for discussion. -6. **`ExecutionPayloadBody` shared definition.** Today every fork - redefines `ExecutionPayloadBody` from scratch. The new spec - would benefit from a small set of fork-named containers - (`ExecutionPayloadBodyParis`, `ExecutionPayloadBodyShanghai`, - `ExecutionPayloadBodyAmsterdam`, …) with the URL `{fork}` - selecting which one. Not worked out here. -7. **Naming convention.** The legacy spec used `camelCase`; this + wrapping a `uint8`. +5. **Naming convention.** The legacy spec used `camelCase`; this sketch uses `snake_case` to match consensus-specs. Worth - confirming. -8. **`ByteVector[BYTES_PER_BLOB]` vs `ByteList[BYTES_PER_BLOB]`.** - A blob is fixed-size (131072 bytes), so `ByteVector` is the - correct typing. Verify against consensus-specs to keep - alignment. + confirming once before publication. diff --git a/src/engine/refactor.md b/src/engine/refactor.md index af62ab16b..b6d957b62 100644 --- a/src/engine/refactor.md +++ b/src/engine/refactor.md @@ -304,10 +304,13 @@ Amsterdam) and **MAY** serve any subset of older revisions alongside; All revisions use `POST` so that 128 versioned hashes (8 KiB hex) don't have to fit in the URL. All revisions return SSZ -`Optional[List[BlobEntry, MAX_BLOBS_REQUEST]]`, where each -`BlobEntry` carries an `available: boolean` per-entry flag for -per-blob misses on revisions that support partial responses. -Revision-specific contents live inside `BlobEntry.contents`. +`List[BlobEntry, MAX_BLOBS_REQUEST]` on `200 OK` and use HTTP +**`204 No Content`** to signal that the EL cannot serve the request +at all (syncing, blob pool unavailable, V2 all-or-nothing miss). +Within a `200` response, per-blob misses are reported via +`BlobEntry.available = false` on revisions that support partial +responses. Revision-specific contents live inside +`BlobEntry.contents`. #### `POST /engine/v2/blobs/v1` @@ -317,8 +320,8 @@ Replaces `engine_getBlobsV1` (Cancun, single-proof whole-blob). - **Response `BlobEntry.contents`:** `BlobAndProofV1 { blob, proof }` (one blob, one 48-byte KZG proof). - Partial responses supported: missing blobs surface as - `available=false` per entry. The outer `Optional` returns `None` - only if the EL cannot serve the request at all (e.g. syncing). + `available=false` per entry. `204 No Content` only when the EL + cannot serve the request at all (e.g. syncing). #### `POST /engine/v2/blobs/v2` @@ -327,8 +330,8 @@ Replaces `engine_getBlobsV2` (Osaka, all-or-nothing cell proofs). - **Request body:** SSZ `List[VersionedHash, MAX_BLOBS_REQUEST]`. - **Response `BlobEntry.contents`:** `BlobAndProofV2 { blob, proofs }` (one blob plus `CELLS_PER_EXT_BLOB` cell proofs). -- **All-or-nothing:** if any requested blob is missing, the outer - `Optional[List[...]]` returns `None`. Otherwise all entries have +- **All-or-nothing:** if any requested blob is missing, the EL + returns `204 No Content`. Otherwise `200 OK` and all entries have `available=true`. This matches today's V2 semantics. #### `POST /engine/v2/blobs/v3` @@ -339,8 +342,8 @@ proofs). - **Request body:** same as `/v2`. - **Response:** same `BlobEntry.contents` shape as `/v2`, but missing blobs surface as `available=false` per entry rather than collapsing - the whole response to `None`. The outer `Optional` returns `None` - only when the EL cannot serve the request at all. + the whole response. `204 No Content` only when the EL cannot serve + the request at all. #### `POST /engine/v2/blobs/v4` @@ -391,6 +394,66 @@ Returns JSON `ClientVersion[]` (same shape as today's `X-Engine-Client-Version` header on every request, removing the mutual-exchange handshake. +### Example: submit a payload + +```bash +curl -X POST http://localhost:8551/engine/v2/amsterdam/payloads \ + -H "Authorization: Bearer $JWT_TOKEN" \ + -H "Content-Type: application/octet-stream" \ + -H "Accept: application/octet-stream" \ + -H "X-Engine-Client-Version: LH/v6.2.1" \ + --data-binary @new_payload.ssz \ + -o payload_status.ssz +``` + +Request: + +``` +POST /engine/v2/amsterdam/payloads HTTP/2 +Host: localhost:8551 +Authorization: Bearer +Content-Type: application/octet-stream +Content-Length: 584 + +<584 bytes: SSZ(ExecutionPayloadEnvelope)> +``` + +Successful response (`status = VALID`): + +``` +HTTP/2 200 +Content-Type: application/octet-stream +Content-Length: 41 + +<41 bytes: SSZ(PayloadStatus)> +``` + +The 41 bytes break down as: `status` (1 byte = `0x00`, `VALID`) + +`latest_valid_hash` (4-byte offset + 32-byte hash = 36 bytes) ++ `validation_error` (4-byte offset + 0 bytes empty list). + +Error response (malformed body): + +``` +HTTP/2 400 +Content-Type: application/problem+json +Content-Length: 49 + +{ "type": "/engine-api/errors/ssz-decode-error" } +``` + +### Example: poll a built payload + +```bash +curl http://localhost:8551/engine/v2/amsterdam/payloads/0x1234567890abcdef \ + -H "Authorization: Bearer $JWT_TOKEN" \ + -H "Accept: application/octet-stream" \ + -o built_payload.ssz +``` + +Response carries `Cache-Control: no-store`; intermediaries MUST NOT +cache. See [Payload retrieval](#payload-retrieval). + --- ## Error model @@ -405,6 +468,15 @@ we use only **two** of the RFC 7807 fields: Omitted when the EL has nothing more to say than the `type` already conveys (e.g. canned SSZ-decode failures). +Success codes: + +| HTTP status | When | +| - | - | +| `200 OK` | SSZ-encoded response body | +| `204 No Content` | Null result (e.g. blob pool syncing on `/blobs/vN`); empty body | + +Error codes: + | HTTP status | `type` | Old JSON-RPC code | When | | - | - | - | - | | 400 Bad Request | `/engine-api/errors/parse-error` | -32700 | Body is not valid JSON / SSZ | @@ -416,6 +488,7 @@ we use only **two** of the RFC 7807 fields: | 409 Conflict | `/engine-api/errors/invalid-forkchoice` | -38002 | Forkchoice state is inconsistent (e.g. finalized not ancestor of head) | | 409 Conflict | `/engine-api/errors/reorg-too-deep` | -38006 | Reorg depth exceeds the EL's limit | | 413 Payload Too Large | `/engine-api/errors/request-too-large` | -38004 | Body exceeds an advertised `limits.*` value | +| 415 Unsupported Media Type | `/engine-api/errors/unsupported-media-type` | (new) | Request `Content-Type` does not match the endpoint's expected encoding (SSZ for hot-path, JSON for diagnostics) | | 422 Unprocessable Entity | `/engine-api/errors/invalid-body` | -32602 | Body decoded fine but has invalid values | | 422 Unprocessable Entity | `/engine-api/errors/invalid-attributes` | -38003 | `payload_attributes` validation failed | | 500 Internal Server Error | `/engine-api/errors/internal` | -32603 / -32000 | Unrecoverable server error; `detail` carries the message | @@ -467,6 +540,46 @@ understands via `GET /engine/v2/capabilities`. its supported fork schemas and endpoint set in a single JSON document at `/engine/v2/capabilities`. +### Capabilities format + +We considered advertising capabilities as a flat list of per-endpoint +strings (e.g. `"POST /amsterdam/payloads"`, the format used by the +existing `engine_exchangeCapabilities` method). The structured form +in `GET /capabilities` (separate `supported_forks`, +`fork_scoped_endpoints`, `independently_versioned`, +`unscoped_endpoints`, plus per-endpoint `limits`) is preferred +because: + +- Adding a fork doesn't multiply the capability list — one entry in + `supported_forks` covers every fork-scoped endpoint at once. +- The `limits.*` block can carry numeric per-endpoint bounds + (`bodies.max_count`, `blobs.max_versioned_hashes`, + `payload.max_bytes`) which a string-list form can't. +- It's easier to evolve: new fields land alongside, old CLs ignore + them. + +### Transition-window behavior + +During the rollout window, a CL upgraded to v2 may interact with an +EL still on the legacy JSON-RPC engine API. Two cases: + +- **EL doesn't expose `/engine/v2/...` at all.** The CL hits any v2 + URL and gets `404 Not Found` from the legacy server. The CL falls + back to JSON-RPC for the duration of that EL's lifetime — no + per-method retry dance. +- **EL exposes `/engine/v2/...` but doesn't know the URL fork.** The + CL hits `/{fork}/...` against an EL that only advertised + `supported_forks: [..., cancun]` while the CL is asking for + `amsterdam`. The EL returns + `400 /engine-api/errors/unsupported-fork`. The CL learns this once + from `GET /capabilities` and avoids issuing such requests; if it + doesn't, the per-request error is structured and explicit, not a + silent downgrade. + +There is **no per-method fallback ladder**. A CL either uses v2 or +JSON-RPC for the lifetime of an EL connection; mixing transports +within a connection is permitted but not required. + --- ## Authentication @@ -514,11 +627,19 @@ Unchanged in spirit: JWT (HS256, 256-bit shared secret). Differences: - **Trailing slashes are forbidden.** `/engine/v2/payloads` is the canonical form; `/engine/v2/payloads/` MUST return `404 method-not-found`. No automatic redirect. -- **Request body encoding:** `application/octet-stream` carrying SSZ - bytes for hot-path endpoints. JSON for diagnostic / metadata - endpoints (capabilities, identity, error bodies). -- **Response body encoding:** SSZ for hot-path data, JSON - (`application/json`) for diagnostics and error bodies. +- **Content-Type / Accept matrix:** + + | Channel | Header | Value | + | - | - | - | + | Hot-path request body (`/payloads`, `/forkchoice`, `/bodies`, `/blobs/vN`) | `Content-Type` | `application/octet-stream` (SSZ) | + | Hot-path request | `Accept` | `application/octet-stream` | + | Hot-path response success body | `Content-Type` | `application/octet-stream` (SSZ) | + | Diagnostic request / response (`/capabilities`, `/identity`) | `Content-Type` | `application/json` | + | Error response body (any endpoint) | `Content-Type` | `application/problem+json` | + + ELs MUST reject hot-path requests carrying any other `Content-Type` + with `415 Unsupported Media Type`. Diagnostic endpoints MUST be + served as JSON regardless of `Accept`. - **Compression:** Servers MAY support `Accept-Encoding: zstd, gzip`. Not required to implement; CLs MUST tolerate uncompressed responses. Blob bundles compress well, so operators are encouraged to enable @@ -771,6 +892,26 @@ We keep JSON available for **error bodies, capability discovery, and client identification** because those are ergonomic to debug with `curl` and not on the hot path. +#### Why not RLP? + +RLP is the EL's native encoding, so reusing it would cut one library +dependency on the EL side. We picked SSZ instead because: + +- **The CL natively serialises every payload field as SSZ today.** An + RLP transport would shift the conversion from "EL parses hex-JSON" + to "CL re-encodes SSZ as RLP" — same total work, just on a + different host. +- **SSZ pins fixed/variable lengths at the type level.** The + transport layer can enforce per-field size limits before + allocation, which RLP's recursive header structure makes harder. +- **`hash_tree_root` for free.** SSZ types come with a deterministic + Merkle root we can use for future content-addressed extensions + (e.g. payload identifiers, capability hashes). RLP would need a + separate hashing convention. +- **Alignment with the rest of the consensus stack.** Beacon API, + fork-choice store, gossip — all SSZ. Reusing the same encoding at + the EL/CL boundary keeps one mental model. + ### Simplifications & removed concepts 1. **`expectedBlobVersionedHashes`** — **removed**. The block-hash check @@ -873,9 +1014,10 @@ the summary exists for quick scanning. present; out-of-era blocks come back as `available=false`); independently versioned (`/blobs/vN`) gives each revision its own dedicated container. Both wrap their entries in - `BodyEntry { available, body }` / `BlobEntry { available, contents }` - with an outer `Optional[List[...]]` for the syncing / - all-or-nothing channel. Per-entry fork tags were rejected. + `BodyEntry { available, body }` / `BlobEntry { available, contents }`. + Whole-response "syncing / all-or-nothing miss" is signalled by + HTTP `204 No Content`, not an in-band SSZ sentinel. Per-entry fork + tags were rejected. #### Error model From 4e0fed12d3ebc9d1ca8829331a82b97b1d1bd154 Mon Sep 17 00:00:00 2001 From: MariusVanDerWijden Date: Thu, 28 May 2026 16:54:15 +0200 Subject: [PATCH 6/6] update to add TargetGasLimit --- src/engine/refactor-ssz.md | 6 ++++-- src/engine/refactor.md | 4 ++++ 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/src/engine/refactor-ssz.md b/src/engine/refactor-ssz.md index 76dcfa485..73f4664e7 100644 --- a/src/engine/refactor-ssz.md +++ b/src/engine/refactor-ssz.md @@ -169,6 +169,7 @@ PayloadAttributes { withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] parent_beacon_block_root: Root slot_number: Uint64 + target_gas_limit: Uint64 } ``` @@ -312,10 +313,11 @@ PayloadAttributesPrague = PayloadAttributesCancun # Osaka = Cancun (no shape change) PayloadAttributesOsaka = PayloadAttributesCancun -# Amsterdam = Cancun + slot_number +# Amsterdam = Cancun + slot_number + target_gas_limit PayloadAttributesAmsterdam { ...Cancun fields... - slot_number: Uint64 + slot_number: Uint64 + target_gas_limit: Uint64 } ``` diff --git a/src/engine/refactor.md b/src/engine/refactor.md index b6d957b62..1abafaf7b 100644 --- a/src/engine/refactor.md +++ b/src/engine/refactor.md @@ -150,6 +150,10 @@ Replaces `engine_forkchoiceUpdatedV{1..4}`. forkchoice update fails, no build is started and no custody change is applied. + When building a payload (Amsterdam+), the EL **MUST** use + `payload_attributes.target_gas_limit` as the target value for the + built block's `gas_limit`. + - **Response body:** SSZ-encoded `ForkchoiceUpdateResponse`: ```