Skip to content

test(evm): silkworm sync regression#503

Open
starwarfan wants to merge 2 commits into
DTVMStack:mainfrom
starwarfan:sync-fail
Open

test(evm): silkworm sync regression#503
starwarfan wants to merge 2 commits into
DTVMStack:mainfrom
starwarfan:sync-fail

Conversation

@starwarfan
Copy link
Copy Markdown
Contributor

1. Does this PR affect any open issues?(Y/N) and add issue references (e.g. "fix #123", "re #123".):

  • N
  • Y

2. What is the scope of this PR (e.g. component or file name):

3. Provide a description of the PR(e.g. more details, effects, motivations or doc link):

  • Affects user behaviors
  • Contains CI/CD configuration changes
  • Contains documentation changes
  • Contains experimental features
  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Other

4. Are there any breaking changes?(Y/N) and describe the breaking changes(e.g. more details, motivations or doc link):

  • N
  • Y

5. Are there test cases for these changes?(Y/N) select and add more details, references or doc links:

  • Unit test
  • Integration test
  • Benchmark (add benchmark stats below)
  • Manual test (add detailed scripts or steps below)
  • Other

6. Release note

None

Add extracted fixtures for 254277:0 and 254297:0 plus a dedicated test target that replays them inside DTVM to keep the historical legacy stack-path accident reproducible without Silkworm snapshots. Document the fail/restore validation loop so disabling the legacy DUP/SWAP runtime-stack fix can be verified as a real regression.
Extend the extracted legacy CALL fixture package so repro validation can run directly through dtvmapi execute(), then document and verify the DTVM-only fail/pass loop when toggling the legacy DUP/SWAP runtime-stack fix.
@github-actions
Copy link
Copy Markdown

⚡ Performance Regression Check Results

⚠️ Performance Regression Detected (interpreter)

No benchmark summary available.


⚠️ Performance Regression Detected (multipass)

No benchmark summary available.


Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a standalone regression repro for a historical legacy (pre–Tangerine Whistle) CALL gas divergence by introducing extracted mainnet fixtures plus a dedicated C++ test target, and documents/shell tooling to reproduce the issue via Silkworm or DTVM-only tests.

Changes:

  • Add standalone fixture set (block_254277 tx0 accident + block_254297 tx0 guard) with schema/README and a generator script.
  • Introduce evmLegacyCallReproTests to execute the fixtures in interpreter/multipass (and via dtvmapi).
  • Add Silkworm-based repro script and a doc note describing workflows.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
tools/repro_legacy_call_254277.sh Automates Silkworm run_single_tx repro and compares gas across interpreter/multipass.
tools/extract_legacy_call_repro.sh Extracts self-contained JSON fixtures from RPC/trace data for the two historical cases.
tests/evm/fixtures/legacy_call_repro/schema.json JSON schema for the legacy CALL repro fixture format.
tests/evm/fixtures/legacy_call_repro/README.md Describes fixture contents and regeneration inputs.
tests/evm/fixtures/legacy_call_repro/cases.json Index of included fixture cases.
tests/evm/fixtures/legacy_call_repro/block_254277_tx_0.json Accident-path fixture (expected tx_gas=57956).
tests/evm/fixtures/legacy_call_repro/block_254297_tx_0.json Guard-path fixture (expected tx_gas=94849).
src/tests/evm_legacy_call_repro_tests.cpp New GTest suite that loads fixtures and runs them through DTVM runtime and dtvmapi.
src/tests/CMakeLists.txt Adds evmLegacyCallReproTests target and registers it with CTest.
docs/legacy_multipass_call_repro.md Documents repro commands, extraction, and test workflow.
Comments suppressed due to low confidence (4)

src/tests/CMakeLists.txt:78

  • evmLegacyCallReproTests is always added under ZEN_ENABLE_EVM, but the test source includes/links dtvmapi. The dtvmapi target is only created when ZEN_ENABLE_LIBEVM adds src/vm, so configurations with ZEN_ENABLE_EVM=ON and ZEN_ENABLE_LIBEVM=OFF will fail at configure/link time. Gate this executable (and its link/test registration) behind ZEN_ENABLE_LIBEVM or if(TARGET dtvmapi).
    add_executable(evmLegacyCallReproTests evm_legacy_call_repro_tests.cpp)
    # Only build evmFallbackExecutionTests if dtvmapi library is available
    if(ZEN_ENABLE_LIBEVM)
      add_executable(evmFallbackExecutionTests evm_fallback_execution_tests.cpp)
      add_executable(evmModuleCacheTests evm_module_cache_tests.cpp)

src/tests/evm_legacy_call_repro_tests.cpp:97

  • If opening the fixture file fails, EXPECT_TRUE(File.is_open()) will not abort, and the code continues to parse/index the document anyway. Switch to ASSERT_TRUE (or return an error) so the test fails cleanly without operating on an invalid stream/document.
  std::ifstream File(Path);
  EXPECT_TRUE(File.is_open()) << "failed to open fixture: " << Path.string();

  rapidjson::IStreamWrapper ISW(File);
  rapidjson::Document Doc;

src/tests/evm_legacy_call_repro_tests.cpp:101

  • After Doc.ParseStream(ISW), the test uses EXPECT_FALSE(Doc.HasParseError()) / EXPECT_TRUE(Doc.IsObject()) but then continues even if they fail. Use ASSERT_* (or early-return) so you don't proceed with an invalid RapidJSON document.
  rapidjson::Document Doc;
  Doc.ParseStream(ISW);
  EXPECT_FALSE(Doc.HasParseError()) << "parse error in fixture: " << Path.string();
  EXPECT_TRUE(Doc.IsObject()) << "fixture root must be object: " << Path.string();

src/tests/evm_legacy_call_repro_tests.cpp:106

  • These direct Doc["..."] accesses assume required members exist and are the right type. If the fixture schema changes or the file is corrupted, this can assert/crash instead of producing a readable test failure. Add HasMember/IsString (etc.) checks (preferably ASSERT_*) before indexing.
  ParsedFixture Fixture;
  Fixture.FixturePath = Path.string();
  Fixture.CaseName = Doc["case_name"].GetString();
  Fixture.Revision = parseRevision(Doc["revision"].GetString());


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

"meta": {
"source": "rpc+trace_replayBlockTransactions",
"rpc_url": rpc_url,
"trace_includes": ["stateDiff", "trace"],
Comment on lines +8 to +10
Fixture file:

- `tests/evm/fixtures/legacy_multipass_call/legacy_call_gas_cases.json`
Comment on lines +94 to +109
The regression test `LegacyRevisionDupSwapUseRuntimeStackPath` in
`src/tests/evm_jit_frontend_tests.cpp` is intentionally behavioral:

- It uses `EVMC_FRONTIER`.
- It asserts `DUP2` and `SWAP1` perform runtime stack access
(`stackGet`/`stackSet`) in legacy mode.

If legacy runtime-stack branches are disabled in the visitor, the same test
fails with:

- `DupStats.StackGetCount == 0`
- `SwapStats.StackGetCount == 0`
- `SwapStats.StackSetCount == 0`

That failure confirms the test is not cosmetic; it directly guards the legacy
operand provenance needed by CALL-family lowering.
if (Revision == "EVMC_SHANGHAI")
return EVMC_SHANGHAI;
if (Revision == "EVMC_CANCUN")
return EVMC_CANCUN;
Comment on lines +256 to +257
Vm->set_option(Vm, "mode", ModeValue);
Vm->set_option(Vm, "enable_gas_metering", "true");
Comment on lines +341 to +346
auto Multi = runFixtureViaDTVMApi(Fixture, "multipass");
ASSERT_TRUE(Interp.Success);
ASSERT_TRUE(Multi.Success);
EXPECT_EQ(Interp.Status, EVMC_SUCCESS);
EXPECT_EQ(Multi.Status, EVMC_SUCCESS);
EXPECT_EQ(Interp.GasCharged, Multi.GasCharged);
Comment on lines +29 to +33
SILKWORM_DIR="$1"
DTVM_BUILD_DIR="${DTVM_BUILD_DIR:-/mnt/data/zhonghao/DTVM/build}"
DATADIR_254277="${DATADIR_254277:-/mnt/erigon-snapshots/dtvm-repro-254277-b}"
DATADIR_254297="${DATADIR_254297:-/mnt/erigon-snapshots/dtvm-repro-254277-20260512}"
STAGED_PIPELINE_BIN="${STAGED_PIPELINE_BIN:-$SILKWORM_DIR/build/silkworm/node/cli/staged_pipeline}"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants