coli: measured Windows launcher defaults — OMP tuning parity + DIRECT/PIPE/PILOT_REAL (opt-out)#279
Merged
JustVugg merged 2 commits intoJul 16, 2026
Conversation
…formationEx) os.cpu_count() returns logical processors, so on SMT machines the plan sets OMP_NUM_THREADS to 2 threads/core, which thrashes the AVX-512 units during expert matmul (9950X3D: 32 logical vs 16 physical). Count RelationProcessorCore records instead, with the existing lscpu/cpu_count fallbacks intact. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…/PIPE/PILOT_REAL
On Windows the engine self-exec OMP tuning never runs (Linux/FreeBSD-only)
and posix_fadvise readahead is a compat.h no-op, so a stock Windows run
leaves large measured wins on the table. The launcher now setdefaults, on
win32 only, each independently overridable by setting the variable:
- OMP_WAIT_POLICY=active, GOMP_SPINCOUNT=200000, OMP_DYNAMIC=FALSE,
OMP_NUM_THREADS=<physical cores> (parity with the glm.c self-exec block;
COLI_NO_OMP_TUNE disables exactly this block, presence-based like the
engine). OMP_PROC_BIND/OMP_PLACES deliberately omitted and also removed
from environment_for_plan on win32: MinGW libgomp has no affinity support
("Affinity not supported on this configuration").
- DIRECT=1: unbuffered expert reads. Measured on a 9950X3D + Samsung 9100
PRO Gen5 + Win11: iobench 10.68 GB/s O_DIRECT vs 9.03 buffered (warm);
end-to-end REPLAY 0.48 -> 1.02 tok/s. Matches JustVugg#162 (1.47x same class).
- PIPE=1: load/matmul overlap, byte-identical output; +8% on top of DIRECT
(PIPE_WORKERS untouched at 8 - 4/8/16 swept flat on Gen5).
- PILOT_REAL=1: real cross-layer prefetch, the only working prefetch on
Windows; +11% and expert hit rate +19 points.
Full ladder methodology and numbers: 96-token greedy REPLAY, one lever per
step, medians of 3-4 runs (see the fork tuning doc referenced in the PR).
tests/test_env_defaults.py covers the defaults, explicit-override-wins,
the kill-switch scope, and the non-win32 no-op.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner
|
Merged to |
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On Windows a stock run leaves large measured wins on the table: the glm.c self-exec OMP tuning is Linux/FreeBSD-only, and the
PREFETCH/readahead path is aposix_fadviseno-op under_WIN32. This teaches the launcher tosetdefault, on win32 only:OMP_WAIT_POLICY=active,GOMP_SPINCOUNT,OMP_DYNAMIC=FALSE,OMP_NUM_THREADS=<physical>), same scope + presence-based kill-switch (COLI_NO_OMP_TUNE) as the engine's own tuner;OMP_PROC_BIND/OMP_PLACESare deliberately omitted (and no longer emitted by--auto-tieron win32) because MinGW libgomp rejects affinity outright;DIRECT=1,PIPE=1,PILOT_REAL=1— all lossless (byte-identical output), all overridable by simply setting the variable (DIRECT=0wins).Measured on a 9950X3D + Samsung 9100 PRO (Gen5) + RTX 5090, Windows 11 native, 96-token greedy REPLAY, one lever per step, medians of 3–4 runs:
Full ladder (including the negatives: PIN_GB=40 regression, CUDA-tier-alone ≈0% replicating #101, MTP DRAFT=3 losing 0.65 vs 1.05 disk-bound) is written up in the fork's tuning doc: https://github.com/KingIcyCreamProjects/colibri/blob/polish/ui-docs/docs/tuning-9950x3d-5090.md
tests/test_env_defaults.pycovers the defaults, explicit-override-wins, kill-switch scope, and the non-win32 no-op. If you'd rather ship these as opt-in (e.g. behindCOLI_WIN_TUNE=1) than opt-out, happy to flip the gate — the measurement stands either way.Note: includes PR 1's commit (physical-core count) as its base —
env_forusesphysical_cpu_count().