WebGPU backend (M10, phases 0-4)#1
Merged
Merged
Conversation
yhirose
force-pushed
the
worktree-webgpu-spike
branch
5 times, most recently
from
July 20, 2026 14:23
01f5741 to
f23019a
Compare
The rest of the repo comments in English; ci.yml was the exception. No behaviour change — comments only.
roadmap.md, architecture.md, performance-notes.md and the x86 runbook are development notes for my own use, not documentation for users of this library — but they sat in docs/, which is where a reader looks for the latter. Move them to docs/_internal/ on disk and ignore docs/_*, leaving docs/ free for user-facing documentation to be written into. Nothing is lost: the files stay where they were being read from, one directory down, and record.md was already ignored on the same reasoning.
Standalone spike under spike/webgpu/, not wired into include/. Answers the two questions that would otherwise have been guessed at. The async boundary is not the obstacle it looked like. Device acquisition needs no async on the C++ side (JS passes a ready device via Module.preinitializedWebGPUDevice), and blocking waits are supported upstream via wgpuInstanceWaitAny(timeout>0) with the TimedWaitAny instance feature. gpu::flush() and sync_to_host() keep their signatures; the suspend surface is two call sites, not the interpreter. A 64x64-tile WGSL sgemm (the shape of sgemm_64_, minus MMA intrinsics WGSL lacks) runs ~75x the scalar wasm CPU path at n=1024, 580-630 GF/s on an M1 Pro, matching the CPU oracle to 5e-06. Two findings that change the design: dispatch+wait has a ~0.6-1.0ms fixed floor, so auto_threshold_() needs its own arm rather than Metal's; and Asyncify vs JSPI is a build-shape fork (JSPI is smaller and avoids the -fwasm-exceptions conflict but is Chrome-only, so it means two builds). See spike/webgpu/README.md for measurements and what is still uncovered.
The alias and gpu_available() sat at the bottom of cuda.h only because that was the one place both tl::metal and tl::cuda were visible. With a third backend coming (M10, WebGPU) that would mean editing the CUDA header to add a browser GPU, so the choice moves to its own header. No behaviour change: gpu.h includes both backend headers and holds the same #if chain, and storage.h includes <gpu.h> instead of <cuda.h>. The cuda benches that include "cuda.h" use tl::cuda:: directly and are unaffected.
… 1-2) The first vertical slice: gemm runs on the GPU under wasm and agrees with the ref oracle. include/webgpu.h follows cuda.h, not metal.h. A WebGPU storage buffer has no CPU-dereferenceable pointer, but gpu::alloc must return one, so each allocation is a device buffer paired with a malloc'd host buffer and a per-allocation dirty state drives lazy H2D/D2H — exactly CUDA's mirror. That is what lets array.h and storage.h stay untouched. flush() and sync_to_host() keep their synchronous signatures, as the spike predicted: the instance is created with TimedWaitAny and the only suspend points are OnSubmittedWorkDone and MapAsync. Dispatches accumulate in one encoder and flush() submits once, which matters more here than on Metal given the ~1ms dispatch floor. View offsets resolved against the plan's open question: binding offsets need 256-byte alignment that view offsets do not have, so bindings cover whole buffers and offsets ride in the params block as element offsets, folded into the indexing in WGSL. The uniform ring does use a dynamic offset, hence the explicit bind group layout. test/wasm/ relinks test_array.cpp unchanged — doctest builds under emcc, and matches_gpu_oracle is already backend-agnostic. 24 cases / 335 assertions green in gpu mode on Chrome, with 149 GEMM dispatches confirmed reaching the GPU (a temporarily instrumented run; the suite passes whether or not the backend engages, so green alone proves nothing). Native ctest unaffected. Everything not yet ported returns false and falls to CPU, which is also the Safari path when JSPI or a device is missing.
Ports the remaining kernel families from metal_kernels.metal: the five contiguous binaries, the six unaries, the five rank-2 broadcast binaries, softmax and the row sum/max reductions. gemv_*, attn_decode and rope stay false stubs, as on Metal. WGSL has no preprocessor, so the per-op variants Metal generates from a macro would have to be copy-pasted — the edge-tile bug class that file's header warns about. Instead each family is one entry point switching on an op code in the uniform block. The branch is uniform across a dispatch and these kernels are memory-bound. That in turn lets one bind group layout and one params struct cover the backend; one-input kernels bind their input twice, which is legal because both bindings are read-only and the output is always a fresh allocation. Two failure modes made this harder than it should have been, and both presented identically — every op returning zeros, including GEMM, which had been working: - CreateShaderModule returns an invalid object rather than null when the WGSL does not compile, as does every pipeline built from it, so the backend reported itself available and quietly did nothing. It now checks GetCompilationInfo and logs the error. The trigger was a literal: WGSL has no -inf, and the decimal spelling of f32::lowest rounds out of range. - Because dispatches batch into one encoder, one invalid dispatch discards the entire command buffer, so a new kernel's mistake breaks unrelated ops. The harness now installs onuncapturederror and surfaces it. The per-entry-point dispatch census is kept rather than removed after debugging: this backend has no test runner that fails when it is absent (unported ops fall back to CPU and the suite passes either way), so the counts are the only evidence the GPU did the work. Browser: 24 cases / 335 assertions green in gpu mode on Chrome, dispatching sgemm 149, ew_unary 50, ew_binary 36, row_reduce 18, ew_bcast 16, softmax 8. Native ctest unaffected.
Port misc/census.cpp to the browser as its own wasm and page, measure the CPU/GPU crossovers, and fill in the TENSORLIB_WEBGPU arms of auto_threshold_() and batch_matmul_bias_threshold_() — until now the wasm build was using the Metal numbers, which are calibrated against an AMX CPU rival that does not exist here. Measured on Chrome / M1 Pro: matmul 4e6 in num_elements*k units (two orders below Metal's 5e8), elementwise 2e6, reduction 2e5, batch bias 8e6. The elementwise and reduction numbers coincide with Metal's because WebGPU's much worse dispatch floor and the much weaker scalar-wasm CPU baseline cancel; matmul is where they do not. The census gets a whole-block section so the batch bias is measured directly rather than inferred, and the test harness gains an auto-mode pass — native ctest cannot reach this arm, so that pass is the only coverage the thresholds have. Both stay green, with auto dispatching 57 of gpu mode's 149 sgemms.
The WebGPU backend had no CI presence at all: ci.yml did not mention wasm, so a break in the TENSORLIB_WEBGPU path would go unnoticed until someone opened a browser. Give it the cuda-build treatment — a job that builds both wasms — since it has the same constraint of no device on hosted runners. That makes the manual browser run the real regression test, as it is for CUDA, so it has to be trustworthy. It was not: the suite is green even if the backend never dispatches, because declined ops fall to CPU and pass anyway. run_tests now fails when any kernel family dispatched zero times in gpu mode. Confirmed by injecting a family that cannot dispatch: doctest reported 335/335 SUCCESS and the run still came back FAIL. auto mode is exempt, where a zero is a threshold decision rather than a regression. build.sh no longer dies when emsdk_env.sh is absent (setup-emsdk puts emcc straight on PATH) and reports a clear error when emcc is missing entirely. One gap remains, documented rather than closed: WGSL is compiled by the browser at runtime, so unlike nvcc -ptx the build cannot catch a shader error.
The backend ships to browsers, so CI had been trying to be a browser — and hosted runners will not give headless Chrome a WebGPU adapter. But what CI has to verify is the WGSL and the C++, not Chrome, and any runtime with navigator.gpu and JSPI will do. Deno has both and reaches an adapter through lavapipe on a GPU-less runner. Notably that is the same lavapipe Chrome ignored, so the obstacle was Chrome rather than the runner. The suite now runs on every push: gpu and auto against the ref oracle, on the same unmodified wasm the page loads, in about 8 seconds. Dispatch counts match Chrome's exactly, so it exercises the same paths. This turns the one place WebGPU was weaker than CUDA into a place it is stronger. nvcc -ptx checks CUDA kernels at build time; WGSL is compiled at run time, so a shader error used to be invisible to everything except a human opening a page. Running the suite compiles them. The build-only job is subsumed and removed. The browser page stays for browser-specific questions (JSPI linkage, device handoff) but is no longer the only line of defence. Getting here took three failed attempts at headless Chrome; they are not kept as commits since they cancel out, but what they settled is recorded in docs/webgpu-plan.md so the next person does not repeat them.
… pass sync_to_host set the mirror to BOTH even when GetConstMappedRange returned null, so a failed readback left the host copy permanently believed current while holding whatever it held before — stale data handed to array::raw(), never retried. Only a completed memcpy updates the state now, and a failure says so on stderr like the WGSL compile path does. encode_ opened and ended a compute pass per dispatch, which contradicts this file's own note about batching like metal.h does: a pass boundary is the coarsest sync WebGPU has, so independent ops in a batch were serialized for nothing. One pass now spans the batch — ordering and write visibility within a pass are guaranteed, so chained ops are unaffected. The uniform ring grows to 4096 slots (1 MB) for the same reason: at 256 it forced a blocking flush mid-graph, chosen by ring capacity rather than by any data dependency. The workgroup tree reduction was copy-pasted three times (softmax's max and sum phases, row_reduce); barrier placement is the worst thing to duplicate, since a misplaced one is a silent race, not a compile error. Folded into tree_reduce() over the existing reduce_op. The list of WGSL entry points was likewise written out twice, and each copy is load-bearing in a different direction — missing one loses the fail-loud pipeline prebuild, missing the other loses the census assertion — so it is now kEntryPoints. Also: build.sh reported wasm sizes with BSD `stat -f%z`, which GNU stat rejects, so the ubuntu CI job this script was just wired into printed no number at all; the failure hid inside a command substitution, so `set -e` never saw it. Verified: deno suite green in gpu and auto modes, 24 cases / 335 assertions, dispatch counts identical to before. Native ctest 5/5.
yhirose
force-pushed
the
worktree-webgpu-spike
branch
from
July 20, 2026 16:30
f23019a to
cf1e906
Compare
The headline, the cross-platform bullet, the use_gpu() example comment and the backends table all still described two GPU backends on three desktop platforms. A third one has been in the tree — and in CI — since the M10 phases landed. Op coverage is stated once, for all three backends together, because it is the same: WebGPU implements exactly the entry points Metal does, and the ones it declines (GEMV, attention, RoPE, int4) are the ones Metal declines too. They are CUDA-only, not a WebGPU gap. What is specific to WebGPU is the JSPI requirement — Chrome in practice, CPU fallback elsewhere — and that its SGEMM has no STEEL-equivalent tiling behind it. Also notes the wasm build line, that CI covers it headless via Deno, and that the WGSL sidesteps the #embed requirement the Metal backend carries by committing a generated C string.
The README showed one six-line example and then talked about backends, dependencies and CI. Nothing told a reader what the library can actually compute, so the only way to find out was to read array.h. Organized by what you are trying to do rather than by header order: creating, inspecting, operating, evaluating, choosing a backend, storage dtypes. Constraints are stated inline where they are surprising rather than left to be discovered — dot() is rank 1-2 only, softmax() is last-axis only, slice() is axis 0 only, data() needs contiguity where raw() does not, and element access needs f32. Ends with the aliasing and threading rules, which are the ones that produce wrong answers rather than exceptions. Deliberately omitted: realize(), install_runtime_hooks() and the threshold internals. They are levers for embedders, and advertising them invites use where eval() is meant. Every signature and every snippet was compile-checked against the headers at C++17, not transcribed by eye.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a third GPU backend so culebra's WASM playground can run the Tensor path on the GPU — under wasm32 neither
__APPLE__norTENSORLIB_CUDAholds, so it has always fallen back to CPU.Plan and measurements live in
docs/webgpu-plan.md; the spike that preceded it is inspike/webgpu/.What lands
gpufacade moved out ofcuda.hinto a newinclude/gpu.h. Pure refactor; adding a browser backend no longer means editing the CUDA header.include/webgpu.hhost layer (host/device mirror copied fromcuda.h, soarray.h/storage.hare untouched), WGSL sgemm, and thetest/wasm/harness.gemv_*,attn_decodeandroperemainreturn falsestubs, as on Metal.Thresholds
Measured on Chrome / M1 Pro with a browser port of
misc/census.cpp. matmul crosses over at 4e6 innum_elements * kunits — two orders of magnitude below the Metal arm's 5e8, because the wasm CPU rival is scalar (~8 GF/s) and has no AMX. Elementwise (2e6) and reduction (2e5) land on exactly the Metal numbers: WebGPU's much worse dispatch floor and the much weaker CPU baseline cancel for memory-bound kernels.Calibration trap worth knowing about: a backgrounded tab throttles the CPU thread while GPU work is barely affected, moving every crossover in the GPU's favour. Two of four runs were degraded that way. The tell is absolute CPU throughput — discard runs that do not reach ~8 GF/s.
Testing
The suite runs in CI on every push, on a runner with no GPU, in about 8 seconds.
This works because CI does not need to be a browser — it needs to verify the WGSL and the C++. Any runtime with
navigator.gpuand JSPI will do, and Deno has both, reaching an adapter through Mesa's lavapipe. Headless Chrome refuses to expose WebGPU at all on hosted runners (three attempts, documented in the plan); notably it ignored the same lavapipe Deno uses, so the obstacle was Chrome rather than the runner.That makes WebGPU better covered than CUDA rather than worse.
nvcc -ptxchecks CUDA kernels at build time but CI never runs them; WGSL is compiled at run time, so running the suite is what checks the shaders.Two things make the green meaningful:
TENSORLIB_WEBGPUarm ofauto_threshold_()is unreachable from native ctest. It dispatches 57 of gpu mode's 149 sgemms, so it is routing by size rather than doing all-or-nothing.run_testsnow fails if any kernel family dispatched zero times. Verified by injecting a family that cannot dispatch: doctest reported 335/335 SUCCESS and the run still came back FAIL.Native
cteststays 5/5; no existing backend's numbers change.Not included
Phase 5 (culebra playground integration) is deliberately out of scope — it only touches culebra's repo, and the submodule pin does not need to move for this.
🤖 Generated with Claude Code