You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This PR adds a progressive local-cluster execution path for consumer Macs and browser/WebGPU devices:
coordinator/worker registration and distributed MoE expert routing
native batch-union expert RPC for disk-backed workers
contiguous dense MLP activation sharding across local nodes
browser/mobile WebGPU expert workers over WebSocket
native TCP-to-WebSocket activation proxy
WebGPU expert weight exporter for readable floating-point checkpoints
browser worker UI and protocol/dispatch tests
The coordinator keeps token generation, routing, attention, and KV state local. Remote workers execute assigned expert FFNs or dense MLP ranges.
Validation
native make test
73 Python tests
17 web tests
TypeScript/Vite production build
Python and JavaScript syntax checks
Scope and follow-ups
The WebGPU path currently uses f32 buffers and is intended for a trusted LAN. TLS/authentication, WebRTC transport, quantized WebGPU weight formats, and broader dense/tensor-parallel sharding can follow as separate changes.
Not yet tested (sorry for being gpu poor) :
Real C coordinator → dense worker COLIDN01 end-to-end
Dense shard output equivalence against local computation
Real C → WebGPU browser integration
WebGPU FFN numerical correctness against a CPU reference
Exporter tests
Actual iPhone/Android/WebGPU hardware smoke test
Heads-up: c/glm.c was just split into c/colibri.c + header modules (#391, now merged into dev). This PR touches the old glm.c, so it will need a rebase onto current dev with its changes moved into colibri.c (or the relevant extracted header: quant.h, sample.h, kv_persist.h, telemetry.h, grammar.h). The make glm target still works (alias of make colibri), so no build scripts break. Apologies for the churn — ping me if the rebase gets thorny and I can help place the hunks.
Rebase needed: this has been in conflict against dev for a while and today's merges (#298, #192) widened it. It bundles three big features (local-cluster MoE, dense sharding, WebGPU workers) — honestly it would land much more easily as three separate PRs, each rebased on current dev. Would you be up for splitting it? The cluster piece especially has overlap with the direction in #431/#432 worth reconciling.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds a progressive local-cluster execution path for consumer Macs and browser/WebGPU devices:
The coordinator keeps token generation, routing, attention, and KV state local. Remote workers execute assigned expert FFNs or dense MLP ranges.
Validation
make testScope and follow-ups
The WebGPU path currently uses f32 buffers and is intended for a trusted LAN. TLS/authentication, WebRTC transport, quantized WebGPU weight formats, and broader dense/tensor-parallel sharding can follow as separate changes.
Not yet tested (sorry for being gpu poor) :
Real C coordinator → dense worker COLIDN01 end-to-end
Dense shard output equivalence against local computation
Real C → WebGPU browser integration
WebGPU FFN numerical correctness against a CPU reference
Exporter tests
Actual iPhone/Android/WebGPU hardware smoke test