Skip to content

fix(evaluator): Honor inc-build flag in warmup prebuilds#207

Merged
fuyu0425 merged 2 commits into
mainfrom
fix/evaluator-warmup-inc-build
Apr 25, 2026
Merged

fix(evaluator): Honor inc-build flag in warmup prebuilds#207
fuyu0425 merged 2 commits into
mainfrom
fix/evaluator-warmup-inc-build

Conversation

@fuyu0425

@fuyu0425 fuyu0425 commented Apr 25, 2026

Copy link
Copy Markdown
Member

Summary

  • keep evaluator warmup/eager prebuild enabled for configured benchmarks instead of disabling it when inc_build_enabled=false
  • propagate inc_build_enabled through dispatcher warmup, config-mode prebuild, and registration-backed prebuild so disabled configs prebuild clean variants rather than incremental-image work
  • add regressions for dispatcher mode, warmup planning, registration serialization, and both prebuild helpers; update the nearest distributed evaluator docs

Testing

  • pytest -q tests/test_evaluator_warmup.py tests/test_registry.py tests/test_evaluator_dual_queue.py::TestRunEvaluatorMain::test_dispatcher_mode_keeps_warmup_when_inc_build_disabled tests/test_evaluator_dual_queue.py::TestConfiglessEvaluator::test_enqueue_pre_builds_from_registration_propagates_inc_image_settings tests/test_evaluator_dual_queue.py::TestConfiglessEvaluator::test_enqueue_pre_builds_from_registration_honors_inc_build_disabled tests/test_evaluator_dual_queue.py::TestConfiglessEvaluator::test_enqueue_pre_builds_propagates_inc_image_settings tests/test_evaluator_dual_queue.py::TestConfiglessEvaluator::test_enqueue_pre_builds_honors_inc_build_disabled
  • scripts/ci-tests/run-local.sh checks

Notes

  • this keeps eager verifier-side cache priming for all benchmarks in the experiment config, but switches that priming from inc-image prep to clean builds when inc_build_enabled is off

@fuyu0425 fuyu0425 changed the title fix(evaluator): Skip dispatcher warmup when inc-build is disabled fix(evaluator): Honor inc-build flag in warmup prebuilds Apr 25, 2026
@fuyu0425 fuyu0425 merged commit 1eb1f35 into main Apr 25, 2026
5 checks passed
@fuyu0425 fuyu0425 deleted the fix/evaluator-warmup-inc-build branch April 25, 2026 08:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant