Skip to content

fix(plan): respect explicit COLI_CUDA_MTP=1 - skip the planner's DRAFT=0 export (opt-in was silently inert)#470

Merged
JustVugg merged 1 commit into
JustVugg:devfrom
mohamedmastouri2000-boop:fix/planner-respect-cudamtp
Jul 20, 2026
Merged

fix(plan): respect explicit COLI_CUDA_MTP=1 - skip the planner's DRAFT=0 export (opt-in was silently inert)#470
JustVugg merged 1 commit into
JustVugg:devfrom
mohamedmastouri2000-boop:fix/planner-respect-cudamtp

Conversation

@mohamedmastouri2000-boop

Copy link
Copy Markdown
Contributor

What

Per the request in #467: when COLI_CUDA_MTP=1 is explicitly in the environment, _auto_tune skips its DRAFT=0 export so the engine's DRAFT=-1 auto path (colibri.cg_draft = (has_mtp && (!cuda || cuda_mtp)) ? 3 : 0) can actually see the opt-in. Plus the ENVIRONMENT.md line next to COLI_CUDA_MTP.

Why

The documented opt-in was silently inert on the Windows bare-run and --auto-tier flows: resource_plan.py _auto_tune exported DRAFT=0 (compute-bound, or disk-bound with projected hit < 0.90), which preempts the engine's auto path — the only place COLI_CUDA_MTP is consulted. Anyone A/B-ing speculation under CUDA got a silent no-op unless they also knew to set DRAFT=3 (my first A/B pass in #467 measured exactly baseline-vs-baseline noise because of this).

Behaviour

Verification

On the #467 box: COLI_CUDA_MTP=1 alone (no DRAFT) through the bare-run Windows flow now loads with MTP ACTIVE (draft=3) and speculates (2.00 tok/forward on a 2-token smoke); without the var, draft=0 as before. The doc line also records the measured trade-off so nobody reads the opt-in as free speed: acceptance holds (49% on sm_120) but at ~85% hit the widened S=4 expert union costs −32% net; the opt-in pays near full residency.

Scope

c/resource_plan.py (one guard + comment in _auto_tune), docs/ENVIRONMENT.md (one row). No engine changes.

… so the engine's auto path can engage

The documented CUDA-MTP opt-in was silently inert through the wrapper: _auto_tune exported DRAFT=0 (compute-bound, or disk-bound with projected hit < 0.90), which preempts the engine's DRAFT=-1 auto path - the only place COLI_CUDA_MTP is consulted (colibri.c). Found while running the JustVugg#467 A/B: COLI_CUDA_MTP=1 alone measured pure baseline noise until DRAFT=3 was also set. Now an explicit COLI_CUDA_MTP=1 makes the planner leave DRAFT unset (engine resolves draft=3); unset keeps DRAFT=0 -> MTP off under CUDA, the measured-correct default (JustVugg#389 -42 streaming-bound). ENVIRONMENT.md documents the behaviour and the measured trade-off.
@JustVugg

Copy link
Copy Markdown
Owner

Merged into dev — thank you @mohamedmastouri2000-boop. Verified locally beyond CI: unit-tested _auto_tune both ways — COLI_CUDA_MTP=1 at disk/0.85 hit now leaves DRAFT unset (opt-in engages draft=3), and unset still exports DRAFT=0 (the measured-correct default). import os present, CI 8/8.

This closes the third-and-last wrapper gotcha from #467 — the documented opt-in now actually works, so anyone can reproduce your MTP-under-CUDA A/B (49% acceptance, −32% at 85% hit) without the DRAFT=3 incantation. One tiny non-blocking note for a future pass: COLI_CUDA_MTP=1 set on a no-GPU box would now also skip the DRAFT=0 export (it's a CUDA flag, so that's user-error territory) — if you ever want to tighten it, gating on has_gpu would scope it to where it's meant to apply. Not worth a PR on its own; just noting it.

With this, #467 is fully closed: findings 1/2/4 shipped (#468/#469), finding 3 measured with the default confirmed, and the opt-in now honoured (#470). Genuinely one of the best-run issues in the repo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants