feat(evm): add profile-guided JIT with thread pool and sliding window profiling#481
Open
ys8888john wants to merge 5 commits into
Open
feat(evm): add profile-guided JIT with thread pool and sliding window profiling#481ys8888john wants to merge 5 commits into
ys8888john wants to merge 5 commits into
Conversation
⚡ Performance Regression Check Results✅ Performance Check Passed (interpreter)Performance Benchmark Results (threshold: 25%)
Summary: 194 benchmarks, 3 regressions ✅ Performance Check Passed (multipass)Performance Benchmark Results (threshold: 25%)
Summary: 194 benchmarks, 0 regressions |
ys8888john
force-pushed
the
feature/profile-guided-jit
branch
4 times, most recently
from
d4d1692 to
5b51502
Compare
16 tasks
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an optional profile-guided JIT (PGJIT) mode for EVM execution: contracts start in the interpreter, a sliding-window profiler detects “hot” contracts, and background compilation is dispatched via a small fixed thread pool. This is integrated into the EVMC VM interface and exposed via config/CLI flags.
Changes:
- Add sliding-window profiling + background JIT compilation pool to trigger EVM JIT compilation on hot contracts.
- Add runtime configuration + CLI/test flags for enabling PGJIT and controlling compile-thread count.
- Make EVM JIT code publication safer for cross-thread visibility (atomic publish), and adjust eager compiler error handling/logging.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| src/vm/dt_evmc_vm.cpp | Implements PGJIT profiling, trigger logic, and a background compile pool; integrates interpreter/JIT switching in execute(). |
| src/runtime/config.h | Adds EnableProfileGuidedJIT and NumJITCompileThreads runtime config knobs. |
| src/runtime/evm_module.cpp | Skips eager compilation when PGJIT is enabled; waits for background compilation in destructor. |
| src/runtime/evm_module.h | Publishes JIT code pointer atomically; stores a future for background compilation completion. |
| src/compiler/evm_compiler.cpp | Wraps EVM JIT compilation in try/catch and publishes JIT code after mprotect. |
| src/runtime/runtime.cpp | Chooses JIT only when JIT code is actually available; otherwise falls back to interpreter. |
| src/cli/dtvm.cpp | Adds --enable-profile-guided-jit CLI flag. |
| src/tests/spec_unit_tests.cpp | Adds --enable-profile-guided-jit flag to spec test runner. |
| src/tests/solidity_contract_tests.cpp | Adds --enable-profile-guided-jit flag (but removes --enable-multipass-lazy). |
| src/action/compiler.cpp | Always runs EVM eager compiler in multipass mode (removes lazy-flag gating for EVM). |
| src/compiler/evm_frontend/evm_mir_compiler.cpp | Minor comment addition in MIR builder init. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+42
to
+48
| // Enable profile-guided JIT: | ||
| // contracts start in interpreter mode; runtime profiling (sliding window | ||
| // of recent calls) determines which contracts are hot enough to trigger | ||
| // background JIT compilation. Modules are NOT compiled at load time. | ||
| bool EnableProfileGuidedJIT = false; | ||
| // Maximum number of concurrent background JIT compilation threads. | ||
| uint32_t NumJITCompileThreads = 10; |
There was a problem hiding this comment.
NumJITCompileThreads can be set to 0 programmatically, but JITCompilePool(0) will accept tasks that never run, and shutdown/destruction can then block forever waiting for unfinished futures. Add validation (e.g., in RuntimeConfig::validate) to ensure NumJITCompileThreads >= 1 (and ideally cap it consistently with set_option’s limit) when profile-guided JIT is enabled.
| @@ -247,8 +247,9 @@ GTEST_API_ int main(int argc, char **argv) { | |||
| "Number of threads for multipass JIT(set 0 for automatic " | |||
| "determination)") | |||
| ->excludes(DMMOption); | |||
There was a problem hiding this comment.
This test binary previously exposed --enable-multipass-lazy like the other CLIs/tests, but that option is removed here. If existing CI scripts or local workflows pass --enable-multipass-lazy to this binary, they'll now fail argument parsing. Consider keeping the flag for parity/backward-compatibility (even if unused for solidity tests) alongside the new --enable-profile-guided-jit option.
Suggested change
| ->excludes(DMMOption); | |
| ->excludes(DMMOption); | |
| bool EnableMultipassLazyCompatibility = false; | |
| CLIParser.add_flag( | |
| "--enable-multipass-lazy", EnableMultipassLazyCompatibility, | |
| "Deprecated compatibility flag accepted for parity with other " | |
| "multipass JIT CLIs; ignored by solidity contract tests"); |
Comment on lines
22
to
+30
|
|
||
| #include <condition_variable> | ||
| #include <cstdlib> | ||
| #include <cstring> | ||
| #include <future> | ||
| #include <mutex> | ||
| #include <optional> | ||
| #include <queue> | ||
| #include <string> |
There was a problem hiding this comment.
This file now uses std::thread, std::function, and std::chrono (e.g., in JITCompilePool and updateProfileAndMaybeTriggerJIT), but the corresponding standard headers are not included here. Add the missing includes (, , ) to avoid non-portable reliance on transitive includes and potential build failures on stricter toolchains.
Suggested change
| #include <condition_variable> | |
| #include <cstdlib> | |
| #include <cstring> | |
| #include <future> | |
| #include <mutex> | |
| #include <optional> | |
| #include <queue> | |
| #include <string> | |
| #include <chrono> | |
| #include <condition_variable> | |
| #include <cstdlib> | |
| #include <cstring> | |
| #include <functional> | |
| #include <future> | |
| #include <mutex> | |
| #include <optional> | |
| #include <queue> | |
| #include <string> | |
| #include <thread> |
Comment on lines
+759
to
+761
| COMPILER::EVMAnalyzer Analyzer(Mod->getRevision()); | ||
| Analyzer.analyze(reinterpret_cast<const uint8_t *>(Mod->Code), Mod->CodeSize); | ||
| if (Analyzer.getJITSuitability().ShouldFallback) { |
There was a problem hiding this comment.
The JIT suitability gate here is weaker than the one used at module creation: it only checks Analyzer.getJITSuitability().ShouldFallback and ignores hasUnresolvedCompatibleDynamicReturnTrampoline(Analyzer). That can allow contracts that should be forced to interpreter mode to be JIT-compiled in profile-guided mode. Reuse the exact same predicate as EVMModule::newEVMModule() and persist the decision on the module (e.g., set Mod->ShouldFallbackToInterp=true when rejecting) so it won't keep re-evaluating/attempting JIT later.
Suggested change
| COMPILER::EVMAnalyzer Analyzer(Mod->getRevision()); | |
| Analyzer.analyze(reinterpret_cast<const uint8_t *>(Mod->Code), Mod->CodeSize); | |
| if (Analyzer.getJITSuitability().ShouldFallback) { | |
| // Reuse the same suitability predicate as module creation and persist | |
| // fallback on the module so we do not keep re-evaluating/re-attempting JIT. | |
| if (Mod->ShouldFallbackToInterp) { | |
| CurrentProfile.JITRejected = true; | |
| return; | |
| } | |
| COMPILER::EVMAnalyzer Analyzer(Mod->getRevision()); | |
| Analyzer.analyze(reinterpret_cast<const uint8_t *>(Mod->Code), Mod->CodeSize); | |
| if (Analyzer.getJITSuitability().ShouldFallback || | |
| hasUnresolvedCompatibleDynamicReturnTrampoline(Analyzer)) { | |
| Mod->ShouldFallbackToInterp = true; |
Comment on lines
+894
to
+921
| // JIT fast path: used by both non-PGJ mode and PGJ mode after JIT is | ||
| // ready. Module and instance are already resolved above. | ||
| evmc_message MsgWithCode = *Msg; | ||
| MsgWithCode.code = reinterpret_cast<uint8_t *>(Mod->Code); | ||
| MsgWithCode.code_size = Mod->CodeSize; | ||
| TheInst->setExeResult(evmc::Result{EVMC_SUCCESS, 0, 0}); | ||
| TheInst->pushMessage(&MsgWithCode); | ||
|
|
||
| evmc::Result Result; | ||
|
|
||
| #ifdef ZEN_ENABLE_VIRTUAL_STACK | ||
| if (Msg->depth == 0) { | ||
| // depth==0: set up virtual stack for stack overflow protection via guard | ||
| // pages. The virtual stack switches RSP to a separate mmap'd region. | ||
| zen::utils::VirtualStackInfo StackInfo; | ||
| StackInfo.SavedPtr1 = TheInst; | ||
| StackInfo.SavedPtr2 = &MsgWithCode; | ||
| StackInfo.SavedPtr3 = &Result; | ||
| TheInst->pushVirtualStack(&StackInfo); | ||
| StackInfo.runInVirtualStack(&callJITFromVirtualStack); | ||
| TheInst->popVirtualStack(); | ||
| } else { | ||
| // depth>0: re-entered via EVMC host callback, already on physical stack | ||
| VM->RT->callEVMInJITMode(*TheInst, MsgWithCode, Result); | ||
| } | ||
| #else | ||
| VM->RT->callEVMInJITMode(*TheInst, MsgWithCode, Result); | ||
| #endif // ZEN_ENABLE_VIRTUAL_STACK |
There was a problem hiding this comment.
In non-profile-guided mode, this path unconditionally calls callEVMInJITMode(), but JIT compilation can now fail silently (EagerEVMJITCompiler::compile catches exceptions and only logs), leaving Mod->getJITCode()==nullptr. That would result in calling a null function pointer. Guard this fast path by checking Mod->getJITCode() and falling back to the interpreter (or returning an error) if JIT code is unavailable.
Comment on lines
107
to
+114
| if (RT.getConfig().Mode != common::RunMode::InterpMode) { | ||
| #ifdef ZEN_ENABLE_MULTIPASS_JIT | ||
| if (RT.getConfig().EnableProfileGuidedJIT) { | ||
| // Profile-guided JIT: skip JIT compilation at load time. | ||
| // JIT will be triggered later by the profiling logic in execute(). | ||
| // Eagerly init bytecode cache for interpreter use. | ||
| (void)Mod->getBytecodeCache(); | ||
| } else |
There was a problem hiding this comment.
When EnableProfileGuidedJIT is true, module creation skips the EVMAnalyzer and never sets ShouldFallbackToInterp. That means contracts that should permanently fall back to interpreter mode can still get JIT-triggered later, and the execute() hot path comment about the flag being set at creation becomes false. Consider running the analyzer even in profile-guided mode to set ShouldFallbackToInterp (without compiling), or otherwise persist the rejection decision onto the module so execute() can reliably avoid JIT for unsupported contracts.
ys8888john
force-pushed
the
feature/profile-guided-jit
branch
2 times, most recently
from
09c69c1 to
8ccc5fa
Compare
Comment on lines
+108
to
+113
| size_t CodeSize = CodeMPool.getMemEnd() - JITCode; | ||
| platform::mprotect(JITCode, TO_MPROTECT_CODE_SIZE(CodeSize), | ||
| PROT_READ | PROT_EXEC); | ||
| // Publish JITCode only after mprotect — atomic release ensures the | ||
| // interpreter thread sees fully executable code. | ||
| EVMMod->setJITCodeAndSize(JITFuncPtr, CodeSize); |
| const evmc_result &Result, | ||
| EVMModule *Mod) { | ||
| std::string ModName = getStableModName(Msg); | ||
| uint64_t GasUsed = static_cast<uint64_t>(Msg->gas - Result.gas_left); |
Comment on lines
+89
to
+99
| std::string getStableModName(const evmc_message *Msg) { | ||
| static const char HexChars[] = "0123456789abcdef"; | ||
| std::string Name = "mod_0x"; | ||
| Name.reserve(6 + 40); | ||
| for (size_t I = 0; I < sizeof(Msg->code_address.bytes); ++I) { | ||
| uint8_t Byte = Msg->code_address.bytes[I]; | ||
| Name.push_back(HexChars[Byte >> 4]); | ||
| Name.push_back(HexChars[Byte & 0x0F]); | ||
| } | ||
| return Name; | ||
| } |
Comment on lines
+854
to
+889
| // JIT not ready: run interpreter with profiling | ||
| (void)Mod->getBytecodeCache(); | ||
|
|
||
| evmc_message MsgWithCode = *Msg; | ||
| MsgWithCode.code = reinterpret_cast<uint8_t *>(Mod->Code); | ||
| MsgWithCode.code_size = Mod->CodeSize; | ||
| TheInst->setExeResult(evmc::Result{EVMC_SUCCESS, 0, 0}); | ||
| TheInst->pushMessage(&MsgWithCode); | ||
|
|
||
| const bool ReuseCachedInstance = !IsTransientMod && Msg->depth == 0; | ||
|
|
||
| std::unique_ptr<zen::evm::InterpreterExecContext> TempCtx; | ||
| zen::evm::InterpreterExecContext *CtxPtr = nullptr; | ||
| if (!ReuseCachedInstance) { | ||
| TempCtx = std::make_unique<zen::evm::InterpreterExecContext>(TheInst); | ||
| CtxPtr = TempCtx.get(); | ||
| } else { | ||
| if (!VM->CachedCtx) { | ||
| VM->CachedCtx = | ||
| std::make_unique<zen::evm::InterpreterExecContext>(TheInst); | ||
| } else { | ||
| VM->CachedCtx->resetForNewCall(TheInst); | ||
| } | ||
| CtxPtr = VM->CachedCtx.get(); | ||
| } | ||
|
|
||
| auto &Ctx = *CtxPtr; | ||
| zen::evm::BaseInterpreter Interpreter(Ctx); | ||
| Ctx.allocTopFrame(&MsgWithCode); | ||
| Interpreter.interpret(); | ||
|
|
||
| evmc::Result Result = | ||
| std::move(const_cast<evmc::Result &>(Ctx.getExeResult())); | ||
| Result.gas_left = TheInst->getGas(); | ||
|
|
||
| evmc_result RawResult = Result.release_raw(); |
Comment on lines
58
to
61
| void EagerEVMJITCompiler::compile() { | ||
| auto Timer = Stats.startRecord(zen::utils::StatisticPhase::JITCompilation); | ||
| try { | ||
| auto Timer = Stats.startRecord(zen::utils::StatisticPhase::JITCompilation); | ||
|
|
Comment on lines
+851
to
+856
| // Profile-guided JIT: use interpreter with profiling when JIT not ready, | ||
| // then switch to JIT once compiled. | ||
| if (!Mod->getJITCode() && VM->Config.EnableProfileGuidedJIT) { | ||
| // JIT not ready: run interpreter with profiling | ||
| (void)Mod->getBytecodeCache(); | ||
|
|
ys8888john
added a commit
to ys8888john/DTVM
that referenced
this pull request
May 8, 2026
ys8888john
force-pushed
the
feature/profile-guided-jit
branch
4 times, most recently
from
a1bbf4d to
69323aa
Compare
starwarfan
reviewed
May 11, 2026
| // - not yet evaluated (no JIT decision made), or | ||
| // - already JIT-compiled/rejected (profile data served its purpose). | ||
| if (OldProfile.WindowCallCount == 0) { | ||
| VM->ProfileStore.erase(EvictIt); |
Contributor
There was a problem hiding this comment.
What if the evicted profile has module cache?
Contributor
Author
There was a problem hiding this comment.
nothing at all, here just rm built jit module in ProfileStore.
starwarfan
reviewed
May 11, 2026
| TaskQueue.push( | ||
| [Task = std::move(Task), Promise = std::move(Promise)]() mutable { | ||
| try { | ||
| Task(); |
Contributor
There was a problem hiding this comment.
What if taskqueue size keeps increasing?
Contributor
Author
There was a problem hiding this comment.
Just have to wait. Do you have a better idea?
starwarfan
reviewed
May 11, 2026
| EVMModule::~EVMModule() { | ||
| #ifdef ZEN_ENABLE_JIT | ||
| if (JITCompileFuture.valid()) { | ||
| JITCompileFuture.wait(); |
Contributor
Author
There was a problem hiding this comment.
Add a timeout check?
starwarfan
reviewed
May 11, 2026
Contributor
starwarfan
left a comment
starwarfan
left a comment
There was a problem hiding this comment.
could you separate the profilestore and ringbuffer from dt_evmc_vm.cpp, the dt_evmc_vm.cpp contains too many data types
ys8888john
force-pushed
the
feature/profile-guided-jit
branch
2 times, most recently
from
aae9437 to
eb9bc1f
Compare
ys8888john
force-pushed
the
feature/profile-guided-jit
branch
from
eb9bc1f to
0b89021
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
… profiling
1. Does this PR affect any open issues?(Y/N) and add issue references (e.g. "fix #123", "re #123".):
2. What is the scope of this PR (e.g. component or file name):
3. Provide a description of the PR(e.g. more details, effects, motivations or doc link):
4. Are there any breaking changes?(Y/N) and describe the breaking changes(e.g. more details, motivations or doc link):
5. Are there test cases for these changes?(Y/N) select and add more details, references or doc links:
6. Release note