test(evm): silkworm sync regression#503
Open
starwarfan wants to merge 2 commits into
Open
Conversation
Add extracted fixtures for 254277:0 and 254297:0 plus a dedicated test target that replays them inside DTVM to keep the historical legacy stack-path accident reproducible without Silkworm snapshots. Document the fail/restore validation loop so disabling the legacy DUP/SWAP runtime-stack fix can be verified as a real regression.
Extend the extracted legacy CALL fixture package so repro validation can run directly through dtvmapi execute(), then document and verify the DTVM-only fail/pass loop when toggling the legacy DUP/SWAP runtime-stack fix.
⚡ Performance Regression Check Results
|
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a standalone regression repro for a historical legacy (pre–Tangerine Whistle) CALL gas divergence by introducing extracted mainnet fixtures plus a dedicated C++ test target, and documents/shell tooling to reproduce the issue via Silkworm or DTVM-only tests.
Changes:
- Add standalone fixture set (
block_254277 tx0accident +block_254297 tx0guard) with schema/README and a generator script. - Introduce
evmLegacyCallReproTeststo execute the fixtures in interpreter/multipass (and viadtvmapi). - Add Silkworm-based repro script and a doc note describing workflows.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/repro_legacy_call_254277.sh | Automates Silkworm run_single_tx repro and compares gas across interpreter/multipass. |
| tools/extract_legacy_call_repro.sh | Extracts self-contained JSON fixtures from RPC/trace data for the two historical cases. |
| tests/evm/fixtures/legacy_call_repro/schema.json | JSON schema for the legacy CALL repro fixture format. |
| tests/evm/fixtures/legacy_call_repro/README.md | Describes fixture contents and regeneration inputs. |
| tests/evm/fixtures/legacy_call_repro/cases.json | Index of included fixture cases. |
| tests/evm/fixtures/legacy_call_repro/block_254277_tx_0.json | Accident-path fixture (expected tx_gas=57956). |
| tests/evm/fixtures/legacy_call_repro/block_254297_tx_0.json | Guard-path fixture (expected tx_gas=94849). |
| src/tests/evm_legacy_call_repro_tests.cpp | New GTest suite that loads fixtures and runs them through DTVM runtime and dtvmapi. |
| src/tests/CMakeLists.txt | Adds evmLegacyCallReproTests target and registers it with CTest. |
| docs/legacy_multipass_call_repro.md | Documents repro commands, extraction, and test workflow. |
Comments suppressed due to low confidence (4)
src/tests/CMakeLists.txt:78
evmLegacyCallReproTestsis always added underZEN_ENABLE_EVM, but the test source includes/linksdtvmapi. Thedtvmapitarget is only created whenZEN_ENABLE_LIBEVMaddssrc/vm, so configurations withZEN_ENABLE_EVM=ONandZEN_ENABLE_LIBEVM=OFFwill fail at configure/link time. Gate this executable (and its link/test registration) behindZEN_ENABLE_LIBEVMorif(TARGET dtvmapi).
add_executable(evmLegacyCallReproTests evm_legacy_call_repro_tests.cpp)
# Only build evmFallbackExecutionTests if dtvmapi library is available
if(ZEN_ENABLE_LIBEVM)
add_executable(evmFallbackExecutionTests evm_fallback_execution_tests.cpp)
add_executable(evmModuleCacheTests evm_module_cache_tests.cpp)
src/tests/evm_legacy_call_repro_tests.cpp:97
- If opening the fixture file fails,
EXPECT_TRUE(File.is_open())will not abort, and the code continues to parse/index the document anyway. Switch toASSERT_TRUE(or return an error) so the test fails cleanly without operating on an invalid stream/document.
std::ifstream File(Path);
EXPECT_TRUE(File.is_open()) << "failed to open fixture: " << Path.string();
rapidjson::IStreamWrapper ISW(File);
rapidjson::Document Doc;
src/tests/evm_legacy_call_repro_tests.cpp:101
- After
Doc.ParseStream(ISW), the test usesEXPECT_FALSE(Doc.HasParseError())/EXPECT_TRUE(Doc.IsObject())but then continues even if they fail. UseASSERT_*(or early-return) so you don't proceed with an invalid RapidJSON document.
rapidjson::Document Doc;
Doc.ParseStream(ISW);
EXPECT_FALSE(Doc.HasParseError()) << "parse error in fixture: " << Path.string();
EXPECT_TRUE(Doc.IsObject()) << "fixture root must be object: " << Path.string();
src/tests/evm_legacy_call_repro_tests.cpp:106
- These direct
Doc["..."]accesses assume required members exist and are the right type. If the fixture schema changes or the file is corrupted, this can assert/crash instead of producing a readable test failure. AddHasMember/IsString(etc.) checks (preferablyASSERT_*) before indexing.
ParsedFixture Fixture;
Fixture.FixturePath = Path.string();
Fixture.CaseName = Doc["case_name"].GetString();
Fixture.Revision = parseRevision(Doc["revision"].GetString());
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| "meta": { | ||
| "source": "rpc+trace_replayBlockTransactions", | ||
| "rpc_url": rpc_url, | ||
| "trace_includes": ["stateDiff", "trace"], |
Comment on lines
+8
to
+10
| Fixture file: | ||
|
|
||
| - `tests/evm/fixtures/legacy_multipass_call/legacy_call_gas_cases.json` |
Comment on lines
+94
to
+109
| The regression test `LegacyRevisionDupSwapUseRuntimeStackPath` in | ||
| `src/tests/evm_jit_frontend_tests.cpp` is intentionally behavioral: | ||
|
|
||
| - It uses `EVMC_FRONTIER`. | ||
| - It asserts `DUP2` and `SWAP1` perform runtime stack access | ||
| (`stackGet`/`stackSet`) in legacy mode. | ||
|
|
||
| If legacy runtime-stack branches are disabled in the visitor, the same test | ||
| fails with: | ||
|
|
||
| - `DupStats.StackGetCount == 0` | ||
| - `SwapStats.StackGetCount == 0` | ||
| - `SwapStats.StackSetCount == 0` | ||
|
|
||
| That failure confirms the test is not cosmetic; it directly guards the legacy | ||
| operand provenance needed by CALL-family lowering. |
| if (Revision == "EVMC_SHANGHAI") | ||
| return EVMC_SHANGHAI; | ||
| if (Revision == "EVMC_CANCUN") | ||
| return EVMC_CANCUN; |
Comment on lines
+256
to
+257
| Vm->set_option(Vm, "mode", ModeValue); | ||
| Vm->set_option(Vm, "enable_gas_metering", "true"); |
Comment on lines
+341
to
+346
| auto Multi = runFixtureViaDTVMApi(Fixture, "multipass"); | ||
| ASSERT_TRUE(Interp.Success); | ||
| ASSERT_TRUE(Multi.Success); | ||
| EXPECT_EQ(Interp.Status, EVMC_SUCCESS); | ||
| EXPECT_EQ(Multi.Status, EVMC_SUCCESS); | ||
| EXPECT_EQ(Interp.GasCharged, Multi.GasCharged); |
Comment on lines
+29
to
+33
| SILKWORM_DIR="$1" | ||
| DTVM_BUILD_DIR="${DTVM_BUILD_DIR:-/mnt/data/zhonghao/DTVM/build}" | ||
| DATADIR_254277="${DATADIR_254277:-/mnt/erigon-snapshots/dtvm-repro-254277-b}" | ||
| DATADIR_254297="${DATADIR_254297:-/mnt/erigon-snapshots/dtvm-repro-254277-20260512}" | ||
| STAGED_PIPELINE_BIN="${STAGED_PIPELINE_BIN:-$SILKWORM_DIR/build/silkworm/node/cli/staged_pipeline}" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
1. Does this PR affect any open issues?(Y/N) and add issue references (e.g. "fix #123", "re #123".):
2. What is the scope of this PR (e.g. component or file name):
3. Provide a description of the PR(e.g. more details, effects, motivations or doc link):
4. Are there any breaking changes?(Y/N) and describe the breaking changes(e.g. more details, motivations or doc link):
5. Are there test cases for these changes?(Y/N) select and add more details, references or doc links:
6. Release note