Skip to content

coli: measured Windows launcher defaults — OMP tuning parity + DIRECT/PIPE/PILOT_REAL (opt-out)#279

Merged
JustVugg merged 2 commits into
JustVugg:mainfrom
KingIcyCreamProjects:pr/win32-launcher-defaults
Jul 16, 2026
Merged

coli: measured Windows launcher defaults — OMP tuning parity + DIRECT/PIPE/PILOT_REAL (opt-out)#279
JustVugg merged 2 commits into
JustVugg:mainfrom
KingIcyCreamProjects:pr/win32-launcher-defaults

Conversation

@KingIcyCreamProjects

Copy link
Copy Markdown
Contributor

On Windows a stock run leaves large measured wins on the table: the glm.c self-exec OMP tuning is Linux/FreeBSD-only, and the PREFETCH/readahead path is a posix_fadvise no-op under _WIN32. This teaches the launcher to setdefault, on win32 only:

  • the OMP block (OMP_WAIT_POLICY=active, GOMP_SPINCOUNT, OMP_DYNAMIC=FALSE, OMP_NUM_THREADS=<physical>), same scope + presence-based kill-switch (COLI_NO_OMP_TUNE) as the engine's own tuner; OMP_PROC_BIND/OMP_PLACES are deliberately omitted (and no longer emitted by --auto-tier on win32) because MinGW libgomp rejects affinity outright;
  • DIRECT=1, PIPE=1, PILOT_REAL=1 — all lossless (byte-identical output), all overridable by simply setting the variable (DIRECT=0 wins).

Measured on a 9950X3D + Samsung 9100 PRO (Gen5) + RTX 5090, Windows 11 native, 96-token greedy REPLAY, one lever per step, medians of 3–4 runs:

step tok/s note
stock 0.45 disk-bound, buffered reads ran ~3 GB/s
+OMP block 0.48 expert-matmul −15%
+DIRECT=1 1.02 +113% (iobench: 10.68 GB/s O_DIRECT vs 9.03 buffered warm; consistent with #162's 1.47×)
+PIPE=1 1.10 workers 4/8/16 flat → default 8 untouched
+PILOT_REAL=1 1.22 hit rate 58→78%

Full ladder (including the negatives: PIN_GB=40 regression, CUDA-tier-alone ≈0% replicating #101, MTP DRAFT=3 losing 0.65 vs 1.05 disk-bound) is written up in the fork's tuning doc: https://github.com/KingIcyCreamProjects/colibri/blob/polish/ui-docs/docs/tuning-9950x3d-5090.md

tests/test_env_defaults.py covers the defaults, explicit-override-wins, kill-switch scope, and the non-win32 no-op. If you'd rather ship these as opt-in (e.g. behind COLI_WIN_TUNE=1) than opt-out, happy to flip the gate — the measurement stands either way.

Note: includes PR 1's commit (physical-core count) as its base — env_for uses physical_cpu_count().

KingIcyCreamProjects and others added 2 commits July 15, 2026 12:05
…formationEx)

os.cpu_count() returns logical processors, so on SMT machines the plan
sets OMP_NUM_THREADS to 2 threads/core, which thrashes the AVX-512 units
during expert matmul (9950X3D: 32 logical vs 16 physical). Count
RelationProcessorCore records instead, with the existing lscpu/cpu_count
fallbacks intact.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…/PIPE/PILOT_REAL

On Windows the engine self-exec OMP tuning never runs (Linux/FreeBSD-only)
and posix_fadvise readahead is a compat.h no-op, so a stock Windows run
leaves large measured wins on the table. The launcher now setdefaults, on
win32 only, each independently overridable by setting the variable:

- OMP_WAIT_POLICY=active, GOMP_SPINCOUNT=200000, OMP_DYNAMIC=FALSE,
  OMP_NUM_THREADS=<physical cores> (parity with the glm.c self-exec block;
  COLI_NO_OMP_TUNE disables exactly this block, presence-based like the
  engine). OMP_PROC_BIND/OMP_PLACES deliberately omitted and also removed
  from environment_for_plan on win32: MinGW libgomp has no affinity support
  ("Affinity not supported on this configuration").
- DIRECT=1: unbuffered expert reads. Measured on a 9950X3D + Samsung 9100
  PRO Gen5 + Win11: iobench 10.68 GB/s O_DIRECT vs 9.03 buffered (warm);
  end-to-end REPLAY 0.48 -> 1.02 tok/s. Matches JustVugg#162 (1.47x same class).
- PIPE=1: load/matmul overlap, byte-identical output; +8% on top of DIRECT
  (PIPE_WORKERS untouched at 8 - 4/8/16 swept flat on Gen5).
- PILOT_REAL=1: real cross-layer prefetch, the only working prefetch on
  Windows; +11% and expert hit rate +19 points.

Full ladder methodology and numbers: 96-token greedy REPLAY, one lever per
step, medians of 3-4 runs (see the fork tuning doc referenced in the PR).
tests/test_env_defaults.py covers the defaults, explicit-override-wins,
the kill-switch scope, and the non-win32 no-op.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@JustVugg
JustVugg merged commit 3074c7d into JustVugg:main Jul 16, 2026
@JustVugg

Copy link
Copy Markdown
Owner

Merged to main. Verified: clean merge + all 14 tests pass (test_env_defaults + test_resource_plan). The physical_cpu_count win32 block is identical to #275 (merged just before this) so git folded it cleanly — no duplication. Good call making the I/O defaults setdefault (explicit override always wins) and scoping COLI_NO_OMP_TUNE to the OMP block only; DIRECT/PIPE/PILOT_REAL are all lossless so opt-out defaults are safe. Skipping OMP_PROC_BIND/OMP_PLACES on Windows (MinGW libgomp has no affinity) is the right fix too. Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants