eth/consensus : implement eccpow consensus engine#10
Open
mmingyeomm wants to merge 3500 commits into
Open
Conversation
In this PR, the Database interface in `core/state` has been extended with one more function: ```go // Iteratee returns a state iteratee associated with the specified state root, // through which the account iterator and storage iterator can be created. Iteratee(root common.Hash) (Iteratee, error) ``` With this additional abstraction layer, the implementation details can be hidden behind the interface. For example, state traversal can now operate directly on the flat state for Verkle or binary trees, which do not natively support traversal. Moreover, state dumping will now prefer using the flat state iterator as the primary option, offering better efficiency. Edit: this PR also fixes a tiny issue in the state dump, marshalling the next field in the correct way.
Implement the snap/2 wire protocol with BAL serving --------- Co-authored-by: Gary Rong <garyrong0905@gmail.com>
…34649) This PR adds Bytes field back to GetAccesListsPacket
The trienode history indexing progress is also exposed via an RPC endpoint and contributes to the eth_syncing status.
👋
This PR makes it possible to run "Amsterdam" in statetests. I'm aware
that they'll be failing and not in consensus with other clients, yet,
but it's nice to be able to run tests and see what works and what
doesn't
Before the change:
```
$ go run ./cmd/evm statetest ./amsterdam.json
[
{
"name": "00000019-mixed-1",
"pass": false,
"fork": "Amsterdam",
"error": "unexpected error: unsupported fork \"Amsterdam\""
}
]
```
After
```
$ go run ./cmd/evm statetest ./amsterdam.json
{"stateRoot": "0x25b78260b76493a783c77c513125c8b0c5d24e058b4e87130bbe06f1d8b9419e"}
[
{
"name": "00000019-mixed-1",
"pass": false,
"stateRoot": "0x25b78260b76493a783c77c513125c8b0c5d24e058b4e87130bbe06f1d8b9419e",
"fork": "Amsterdam",
"error": "post state root mismatch: got 25b78260b76493a783c77c513125c8b0c5d24e058b4e87130bbe06f1d8b9419e, want 0000000000000000000000000000000000000000000000000000000000000000"
}
]
```
…ared pipeline into triedb/internal (#34654) This PR adds `GenerateTrie(db, scheme, root)` to the `triedb` package, which rebuilds all tries from flat snapshot KV data. This is needed by snap/2 sync so it can rebuild the trie after downloading the flat state. The shared trie generation pipeline from `pathdb/verifier.go` was moved into `triedb/internal/conversion.go` so both `GenerateTrie` and `VerifyState` reuse the same code.
This PR refactors the encoding rules for `AccessListsPacket` in the wire protocol. Specifically: - The response is now encoded as a list of `rlp.RawValue` - `rlp.EmptyString` is used as a placeholder for unavailable BAL objects
PathDB keys diff layers by state root, not by block hash. That means a side-chain block can legitimately collide with an existing canonical diff layer when both blocks produce the same post-state (for example same parent, same coinbase, no txs). Today `layerTree.add` blindly inserts that second layer. If the root already exists, this overwrites `tree.layers[root]` and appends the same root to the mutation lookup again. Later account/storage lookups resolve that root to the wrong diff layer, which can corrupt reads for descendant canonical states. At runtime, the corruption is silent: no error is logged and no invariant check fires. State reads against affected descendants simply return stale data from the wrong diff layer (for example, an account balance that reflects one fewer block reward), which can propagate into RPC responses and block validation. This change makes duplicate-root inserts idempotent. A second layer with the same state root does not add any new retrievable state to a tree that is already keyed by root; keeping the original layer preserves the existing parent chain and avoids polluting the lookup history with duplicate roots. The regression test imports a canonical chain of two layers followed by a fork layer at height 1 with the same state root but a different block hash. Before the fix, account and storage lookups at the head resolve the fork layer instead of the canonical one. After the fix, the duplicate insert is skipped and lookups remain correct.
Co-authored-by: Felix Lange <fjl@twurst.com>
…33884) Changes JSON serialization of FilterCriteria to exclude "address" when it is empty.
ProcessBeaconBlockRoot (EIP-4788) and processRequestsSystemCall (EIP-7002/7251) do not merge the EVM access events into the state after execution. ProcessParentBlockHash (EIP-2935) already does this correctly at line 290-291. Without this merge, the Verkle witness will be missing the storage accesses from the beacon root and request system calls, leading to incomplete witnesses and potential consensus issues when Verkle activates.
Co-authored-by: Felix Lange <fjl@twurst.com>
This PR fixes #34623 by changing the `vm.StateDB` interface: Instead of `EmitLogsForBurnAccounts()` emitting burn logs, `LogsForBurnAccounts() []*types.Log` just returns these logs which are then emitted by the caller. This way when tracing is used, `hookedStateDB.AddLog` will be used automatically and there is no need to duplicate either the burn log logic or the `OnLog` tracing hook.
This is to fix a crasher in keeper.
This tool is designed for the offline translation of an MPT database to a binary trie. This is to be used for users who e.g. want to prove equivalence of a binary tree chain shadowing the MPT chain. It adds a `bintrie` command, cleanly separating the concerns.
already check on line 40 before.
`BinaryTrie.DeleteAccount` was a no-op, silently ignoring the caller's deletion request and leaving the old `BasicData` and `CodeHash` in the trie. Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>
…before returning values (#34690) Fix `GetAccount` returning **wrong account data** for non-existent addresses when the trie root is a `StemNode` (single-account trie) — the `StemNode` branch returned `r.Values` without verifying the queried address's stem matches. Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>
Pre-refactor PR to get 8037 upstreamed in chunks --------- Co-authored-by: Gary Rong <garyrong0905@gmail.com>
The spec has been changed during SIC #49, the offset is encoded as a big-endian number.
Two fixes for `testing_buildBlockV1`: 1. Add `omitempty` to `SlotNumber` in `ExecutableData` so it is omitted for pre-Amsterdam payloads. The spec defines the response as `ExecutionPayloadV3` which does not include `slotNumber`. 2. Pass `res.fees` instead of `new(big.Int)` in `BuildTestingPayload` so `blockValue` reflects actual priority fees instead of always being zero. Corresponding fixture update: ethereum/execution-apis#783
The comment formula showed (i+3) but the code multiplies by 9 (Lsh 3 + add = 8+1). This was a error when porting from upstream golang.org/x/crypto/bn256 where ξ=i+3. Go-ethereum changed the constant to ξ=i+9 but forgot to update the inner formula.
TestUpdatedKeyfileContents was intermittently failing with: - Emptying account file failed - wasn't notified of new accounts Root cause: waitForAccounts required the account list match and an immediately readable ks.changes notification in the same instant, creating a timing race between cache update visibility and channel delivery. This change keeps the same timeout window but waits until both conditions are observed, which preserves test intent while removing the flaky timing dependency. Validation: - go test ./accounts/keystore -run '^TestUpdatedKeyfileContents$' -count=100
Return ErrInvalidOpCode with the executing opcode and offending immediate for forbidden DUPN, SWAPN, and EXCHANGE operands. Extend TestEIP8024_Execution to assert both opcode and operand for all invalid-immediate paths.
# Summary
Replaces the inline `errors.New("event signature mismatch")` in
generated `UnpackXxxEvent` methods with per-event package-level sentinel
errors (e.g. `ErrTransferSignatureMismatch`,
`ErrApprovalSignatureMismatch`), allowing callers to reliably
distinguish a topic mismatch from a genuine decoding failure via
`errors.Is`.
Each event gets its own sentinel, generated via the abigen template:
```go
var ErrTransferSignatureMismatch = errors.New("event signature mismatch")
```
This scoping is intentional — it allows callers to be precise about
*which* event was mismatched, which is useful when routing logs across
multiple unpackers.
# Motivation
Previously, all errors returned from `UnpackXxxEvent` were
indistinguishable without string matching. This is especially
problematic when processing logs sourced from `eth_getBlockReceipts`,
where a caller receives the full set of logs for a block across all
contracts and event types. In that context, a signature mismatch is
expected and should be skipped, while any other error (malformed data,
topic parsing failure) indicates something is genuinely wrong and should
halt execution:
```go
for _, log := range blockLogs {
event, err := contract.UnpackTransferEvent(log)
if errors.Is(err, gen.ErrTransferSignatureMismatch) {
continue // not our event, expected
}
if err != nil {
return fmt.Errorf("unexpected decode failure: %w", err) // alert
}
// process event
}
```
**Changes:**
- `abigen` template: generates a `ErrXxxSignatureMismatch` sentinel per
event and returns it on topic mismatch instead of an inline error
- Existing generated bindings & testdata: regenerated to reflect the
update
Implements #34075
This fixes a truncation bug that results in an invalid serialization of
empty EIP712.
For example:
```json
{
"method": "eth_signTypedData_v4",
"request": {
"types": {
"EIP712Domain": [
{
"name": "version",
"type": "string"
}
],
"Empty": []
},
"primaryType": "Empty",
"domain": {
"version": "0"
},
"message": {}
}
}
```
When calculating the type-hash for the stuct-hash, it will incorrectly
use `Empty)` instead of `Empty()`
runtime.setDefaults was unconditionally assigning cfg.Random =
&common.Hash{}, which silently overwrote any caller-provided Random
value. This made it impossible to simulate a specific PREVRANDAO and
also forced post-merge rules whenever London was active, regardless of
the intended environment.
This change only initializes cfg.Random when it is nil, matching how
other fields in Config are defaulted. Existing callers that did not set
Random keep the same behavior (a non-nil zero hash still enables
post-merge semantics), while callers that explicitly set Random now get
their value respected.
…rrect variable usages in error strings (#35121) This PR is trying to stem further slop PRs by going over all incorrect strings and fixing them. --------- Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>
Co-authored-by: jwasinger <j-wasinger@hotmail.com>
Better error messages on the engine api
Implement a similar RPC as what reth offers paradigmxyz/reth#18539, to clear the tx pool.
Fixes an issue where we would falsely return 400 when we should return 200 because the request was served successfully --------- Co-authored-by: Felix Lange <fjl@twurst.com>
This is a PR that removes all correctly flagged typos, in order to stop an onslaught of slop PRs in its tracks. It should be followed by #34994 but the latter needs more configuration work and I want to limit the stem of PRs right now.
This adds a client option to configure trace context propagation via the `traceparent` HTTP header. I'm adding this so that prysm can enable distributed tracing on their engine API client.
… crash (#35140) Found while updating to go-ethereum master in prysm.
…5074) In old code, mu is struct not pointer, it caused create new mutex event with same writer. Change to use pointer to sync.Mutex, so that the mutex is shared between handler with same writer.
Adds snap/2 (EIP-8189), a block-access-list (BAL) based state sync, and wires it to run side by side with snap/1. It's opt-in (for now) behind a new --snap.v2 flag and chosen at startup. https://eips.ethereum.org/EIPS/eip-8189 --------- Co-authored-by: Toni Wahrstätter <info@toniwahrstaetter.com> Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This PR introduces a cache for GetBlobs request. The main purpose of this PR is to reduce the getBlobs latency by reading and decoding blobs from the pool in advance of the actual query. This is important especially in the context of a sparse blobpool, since it may be necessary to recover blobs from cells on a getBlobs request. Previously, the Engine API read and decoded blobs from the pool on every call. Now those calls check the cache and only fall back to the pool on a miss. The cache has two modes: - In topK mode (default), it wakes up periodically, picks the most profitable pending blob transactions up to the current fork's maxBlobsPerBlock, and loads their blobs. The selection logic is shared with the miner's block-building logic. The selection size is derived from eip4844.MaxBlobsPerBlock at the current head. - When the CL calls HasBlobs, the cache switches to hasBlobs mode and tries to pin the set it just reported as available. Cache updates (read, decode, and optionally conversion in the future) run in background goroutines. --------- Co-authored-by: Felix Lange <fjl@twurst.com>
This PR adds an optimization to the `findNodeByID` function in `p2p/discover`. There is already an open PR (#33205) for similar improvements, and I have further optimized the function to get better performance. I have attached the benchmark results comparing the current `main` branch with my `optimized version`, and the results show clear improvements. --------- Co-authored-by: Csaba Kiraly <csaba.kiraly@gmail.com>
Currently geth ignores the docker `--memory` directive and doesn't adjust its cache size downward when necessary, potentially running into OOM. while gopsutil has functions like `docker/CgroupMem()` they are rather for reading cgroup memory limit of a container from the host.
Implements https://eips.ethereum.org/EIPS/eip-8037 mainly done in order to judge the complexity of the EIP and to act as a jumping off point, since the eip will likely change. --------- Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This PR fixes an issue that when peers legitimately lack a requested BAL, empty (0x80) is delivered and this BAL entry will be refetched over and over again. A `refused` tracker is added and catchUp will fail if this BAL is unavailable against the entire peerset.
`TestTracingHTTPTimeout` still flakes in CI after #35101, failing at the POST: --- FAIL: TestTracingHTTPTimeout (0.26s) tracing_test.go:633: request: Post "http://127.0.0.1:43497": EOF The test sets a short server `WriteTimeout` and posts a blocking call. `ContextRequestTimeout` leaves a fixed 100ms for the server to write its timeout response before the HTTP write deadline cuts the connection. I can't repro it locally, but my theory is that under load that write can miss the window, so the connection is dropped and the client POST returns `EOF`, failing the test before it inspects the span. This is the only test exposed to it because it is the only one that configures a `WriteTimeout`. The EOF is benign: the server sets the timeout error on the SERVER span before attempting the write, independent of whether the client receives the response. Since that span status is all the test asserts, `tryPostJSONRPC` tolerates the transport error instead of failing on it.
The stack primitives pop by value: pop() returns the 32-byte value
itself, so every popped operand is copied out of the stack arena before
it is used. The result side was already in place, peek returns a pointer
and binary ops write into the new stack top. This PR fixes the operand
side: pointer-returning primitives (popPtr, popPtrPeek, etc), with the
handlers rewritten to read operands directly from their arena slots.
Every popped operand paid the copy, whatever the op went on to do with
it, so this optimization covers the arithmetic and comparison ops as
much as JUMP, MSTORE, SSTORE and RETURN.
The copy is visible in the assembly. On arm64, master's opLt spends four
instructions moving the popped value through the frame, and the
comparison then reads it back from there:
LDP (R5), (R6, R7) ; load words 0 and 1 of the popped value from the
arena
LDP 16(R5), (R5, R8) ; load words 2 and 3
STP (R6, R7), vm.~r0-64(SP) ; store words 0 and 1 into a frame slot
STP (R5, R8), vm.~r0-48(SP) ; store words 2 and 3
With popPtrPeek those four instructions are gone, the frame shrinks from
locals=0x58 to locals=0x18, and the function from 336 to 288 bytes. The
compiler cannot remove the copy itself: uint256.Int is a four-element
array, and Go's SSA does not promote arrays longer than one element to
registers, so a by-value pop pays this round trip no matter how far
inlining gets, for LT exactly as for ADD.
The CALL and CREATE families are deliberately not converted: a child
frame reuses the same stack arena, so parent pointers into popped slots
die when the child pushes. The rule is recorded on the primitives:
pointers stay valid until the next push or any sub call. Converting the
call family safely means materializing scalars before the child call,
left for later work with a call-heavy benchmark to justify it.
### Benchmarks
Measured with the benchmark suite from #35144 (the evm-bench contract
workloads and the block import benchmark), which is not part of this
PR's diff. Apple M4 Max, fixed iteration counts, n=10, all p=0.000. B/op
and allocs/op are statistically identical on every benchmark:
| benchmark | master | PR | vs master |
|---|---|---|---|
| Snailtracer | 60.0 ms | 54.1 ms | -9.8% |
| TenThousandHashes | 13.2 ms | 12.2 ms | -7.8% |
| ERC20Transfer | 11.7 ms | 11.0 ms | -5.5% |
| ERC20Mint | 7.49 ms | 7.02 ms | -6.2% |
| ERC20ApprovalTransfer | 8.92 ms | 8.44 ms | -5.4% |
This PR is independent of #35144 but plays nicely with it: the generated
dispatch there splices these handler bodies, so the in-place forms land
in its fast path too, where they measure larger.
### Testing
The rewritten handlers run on the interpreter's only execution path, so
correctness rests on references outside the change:
- **Consensus fixtures.** The full tests package passes: state tests,
the execution-spec families, blockchain tests.
- **Opcode testcases.** The JSON testcases compare individual opcode
results against committed expected values.
- **Tracer fixtures.** The tracetest reference files pin exact log and
return data shapes, covering the rewritten LOG and RETURN paths.
- **Cross-build differential.** A goevmlab campaign running this
branch's evm against master's evm over generated state tests across four
forks (Prague, Cancun, London, Osaka) with full trace comparison:
160,566 tests, zero divergences.
---------
Co-authored-by: MariusVanDerWijden <m.vanderwijden@live.de>
…#35170) sendInvalidTxs's *eth.TransactionsPacket case iterated `txs` — the locally-sent invalid transactions, every one of which is in `invalids` by construction — instead of the transactions actually carried by the received packet. As a result the loop returned "received bad tx" on the very first TransactionsPacket the peer sent, regardless of its contents, and never inspected what was really propagated. Iterate msg.Items() (the decoded contents of the received packet) so the "node must not propagate invalid txs" conformance check tests the real condition instead of producing a false negative. --------- Co-authored-by: Bosul Mun <bsbs8645@snu.ac.kr>
This PR improves the slot reservation logic in the context of snap/2. Geth has the mechanism to reserve roughly half the peer slots for peers supporting the snap protocol if snap syncing is needed by local node. With the context of snap/2, this mechanism should be changed that: we reserve the slot for the "usable snap peer", not blindly for peer with snap extension enabled (such as legacy snap/1, which can't serve the snap/2).
) This PR introduces a new condition that if the local node falls behind too much and the required BAL for catching up is very likely to be unavailable, the entire snap sync will be restarting from scratch. As the defined BAL retention window is weak-subjective-period which is calculated dynamically. A more conservative threshold is used (90K blocks) for robustness. Apart from that, the BAL catchup will be divided into several spans and apply one by one. It's essential to prevent the potential out-of-memory panic of placing the entire BAL set in memory.
This PR does two things: - Expose snap/2 specific sync progress fields - Seed the sync progress after `loadSyncStatus `
This PR fixes an issue where flat states are continuously persisted during downloadState, while the sync journal is only persisted at the end of Sync. As a result, an unclean shutdown can leave the on-disk flat state ahead of the journal markers. Some persisted entries may be stale (storage slots that should have been deleted), and these dangling entries are not detected or fixed by subsequent state downloads. To address this, this PR introduces a cleanup step before state downloading begins. It removes all state entries that are not covered by the persisted journal markers.
Adds `testing_commitBlockV1`. It is the write companion of `testing_buildBlockV1`: it builds a block from the provided payload attributes and transactions on top of the current canonical head, inserts it, and sets it as the new head, returning the new head hash. --------- Co-authored-by: MariusVanDerWijden <m.vanderwijden@live.de>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
implements eccpow consensus engine for Worldland Network