docs(examples): add GB10/DGX Spark NVFP4 optimized sweep specs

examples/gb10-dgx-spark-nvfp4-optimized/PROFILES-old-vs-new.txt

@@ -0,0 +1,24 @@
+########## qwen27b ##########
+--- OLD serve (4k-reference profile) ---
+  CUTE_DSL_ARCH=sm_121a FLASHINFER_DISABLE_VERSION_CHECK=1 VLLM_SERVER_DEV_MODE=1 VLLM_USE_FLASHINFER_SAMPLER=0 /home/bob/runtimes/vllm/versions/vllm-0.24.0-qwen36-sm121/.venv/bin/vllm serve nvidia/Qwen3.6-27B-NVFP4 --host 127.0.0.1 --port 9104 --max-model-len 4096 --max-num-seqs 32 --max-num-batched-tokens 2048 --gpu-memory-utilization 0.4 --kv-cache-dtype fp8 --attention-backend flashinfer --moe-backend marlin --no-enable-prefix-caching --quantization modelopt --trust-remote-code --load-format fastsafetensors --enable-chunked-prefill --no-async-scheduling --no-enable-prefix-caching --mamba-cache-mode none --language-model-only --mm-processor-cache-gb 0 --enforce-eager --default-chat-template-kwargs '{"enable_thinking":false}' --no-enable-log-requests --no-enable-flashinfer-autotune --reasoning-parser qwen3 --tool-call-parser qwen3_xml --enable-auto-tool-choice
+--- NEW serve (4k-reference profile) ---
+  CUTE_DSL_ARCH=sm_121a FLASHINFER_DISABLE_VERSION_CHECK=1 VLLM_SERVER_DEV_MODE=1 VLLM_USE_FLASHINFER_SAMPLER=0 /home/bob/runtimes/vllm/versions/vllm-0.24.0-qwen36-sm121/.venv/bin/vllm serve nvidia/Qwen3.6-27B-NVFP4 --host 127.0.0.1 --port 9104 --max-model-len 4096 --max-num-seqs 32 --max-num-batched-tokens 16384 --gpu-memory-utilization 0.4 --kv-cache-dtype fp8 --attention-backend flashinfer --moe-backend marlin --no-enable-prefix-caching --quantization modelopt --trust-remote-code --load-format fastsafetensors --enable-chunked-prefill --no-enable-prefix-caching --language-model-only --mm-processor-cache-gb 0 --default-chat-template-kwargs '{"enable_thinking":false}' --no-enable-log-requests --no-enable-flashinfer-autotune --reasoning-parser qwen3 --tool-call-parser qwen3_xml --enable-auto-tool-choice
+
+########## qwen35b ##########
+--- OLD serve (4k-reference profile) ---
+  CUTE_DSL_ARCH=sm_121a FLASHINFER_DISABLE_VERSION_CHECK=1 VLLM_SERVER_DEV_MODE=1 VLLM_USE_FLASHINFER_SAMPLER=0 /home/bob/runtimes/vllm/versions/vllm-0.24.0-qwen36-sm121/.venv/bin/vllm serve nvidia/Qwen3.6-35B-A3B-NVFP4 --host 127.0.0.1 --port 9204 --max-model-len 4096 --max-num-seqs 32 --max-num-batched-tokens 2048 --gpu-memory-utilization 0.4 --kv-cache-dtype fp8 --attention-backend flashinfer --moe-backend marlin --no-enable-prefix-caching --quantization modelopt --trust-remote-code --load-format fastsafetensors --enable-chunked-prefill --no-async-scheduling --no-enable-prefix-caching --mamba-cache-mode none --language-model-only --mm-processor-cache-gb 0 --enforce-eager --default-chat-template-kwargs '{"enable_thinking":false}' --no-enable-log-requests --no-enable-flashinfer-autotune --reasoning-parser qwen3 --tool-call-parser qwen3_xml --enable-auto-tool-choice
+--- NEW serve (4k-reference profile) ---
+  CUTE_DSL_ARCH=sm_121a FLASHINFER_DISABLE_VERSION_CHECK=1 VLLM_SERVER_DEV_MODE=1 VLLM_USE_FLASHINFER_SAMPLER=0 /home/bob/runtimes/vllm/versions/vllm-0.24.0-qwen36-sm121/.venv/bin/vllm serve nvidia/Qwen3.6-35B-A3B-NVFP4 --host 127.0.0.1 --port 9204 --max-model-len 4096 --max-num-seqs 32 --max-num-batched-tokens 16384 --gpu-memory-utilization 0.4 --kv-cache-dtype fp8 --attention-backend flashinfer --moe-backend marlin --no-enable-prefix-caching --quantization modelopt --trust-remote-code --load-format fastsafetensors --enable-chunked-prefill --no-enable-prefix-caching --language-model-only --mm-processor-cache-gb 0 --default-chat-template-kwargs '{"enable_thinking":false}' --no-enable-log-requests --no-enable-flashinfer-autotune --reasoning-parser qwen3 --tool-call-parser qwen3_xml --enable-auto-tool-choice
+
+########## gemma4-26b ##########
+--- OLD serve (4k-reference profile) ---
+  CUTE_DSL_ARCH=sm_121a FLASHINFER_DISABLE_VERSION_CHECK=1 VLLM_SERVER_DEV_MODE=1 VLLM_USE_V2_MODEL_RUNNER=1 /home/bob/runtimes/vllm/current/.venv/bin/vllm serve nvidia/Gemma-4-26B-A4B-NVFP4 --host 127.0.0.1 --port 9304 --max-model-len 4096 --max-num-seqs 32 --max-num-batched-tokens 8192 --gpu-memory-utilization 0.3 --kv-cache-dtype fp8 --attention-backend TRITON_ATTN --moe-backend cutlass --no-enable-prefix-caching --trust-remote-code --enable-chunked-prefill --reasoning-parser gemma4 --enable-auto-tool-choice --tool-call-parser gemma4 --language-model-only --mm-processor-cache-gb 0 --default-chat-template-kwargs '{"enable_thinking":false}' --no-enable-log-requests --no-enable-flashinfer-autotune --override-generation-config '{"max_new_tokens":null}' --kv-cache-memory-bytes 12884901888
+--- NEW serve (4k-reference profile) ---
+  CUTE_DSL_ARCH=sm_121a FLASHINFER_DISABLE_VERSION_CHECK=1 VLLM_SERVER_DEV_MODE=1 VLLM_USE_V2_MODEL_RUNNER=1 /home/bob/runtimes/vllm/current/.venv/bin/vllm serve nvidia/Gemma-4-26B-A4B-NVFP4 --host 127.0.0.1 --port 9304 --max-model-len 4096 --max-num-seqs 32 --max-num-batched-tokens 16384 --gpu-memory-utilization 0.3 --kv-cache-dtype fp8 --attention-backend TRITON_ATTN --moe-backend cutlass --no-enable-prefix-caching --trust-remote-code --enable-chunked-prefill --reasoning-parser gemma4 --enable-auto-tool-choice --tool-call-parser gemma4 --language-model-only --mm-processor-cache-gb 0 --default-chat-template-kwargs '{"enable_thinking":false}' --no-enable-log-requests --no-enable-flashinfer-autotune --override-generation-config '{"max_new_tokens":null}' --kv-cache-memory-bytes 12884901888
+
+########## gemma4-31b ##########
+--- OLD serve (4k-reference profile) ---
+  CUTE_DSL_ARCH=sm_121a FLASHINFER_DISABLE_VERSION_CHECK=1 VLLM_SERVER_DEV_MODE=1 VLLM_USE_V2_MODEL_RUNNER=1 /home/bob/runtimes/vllm/current/.venv/bin/vllm serve nvidia/Gemma-4-31B-IT-NVFP4 --host 127.0.0.1 --port 9304 --max-model-len 4096 --max-num-seqs 32 --max-num-batched-tokens 8192 --gpu-memory-utilization 0.6 --kv-cache-dtype fp8 --attention-backend TRITON_ATTN --no-enable-prefix-caching --trust-remote-code --enable-chunked-prefill --reasoning-parser gemma4 --enable-auto-tool-choice --tool-call-parser gemma4 --language-model-only --mm-processor-cache-gb 0 --default-chat-template-kwargs '{"enable_thinking":false}' --no-enable-log-requests --no-enable-flashinfer-autotune --override-generation-config '{"max_new_tokens":null}' --kv-cache-memory-bytes 10737418240
+--- NEW serve (4k-reference profile) ---
+  CUTE_DSL_ARCH=sm_121a FLASHINFER_DISABLE_VERSION_CHECK=1 VLLM_SERVER_DEV_MODE=1 VLLM_USE_V2_MODEL_RUNNER=1 /home/bob/runtimes/vllm/current/.venv/bin/vllm serve nvidia/Gemma-4-31B-IT-NVFP4 --host 127.0.0.1 --port 9304 --max-model-len 4096 --max-num-seqs 32 --max-num-batched-tokens 16384 --gpu-memory-utilization 0.6 --kv-cache-dtype fp8 --attention-backend TRITON_ATTN --no-enable-prefix-caching --trust-remote-code --enable-chunked-prefill --reasoning-parser gemma4 --enable-auto-tool-choice --tool-call-parser gemma4 --language-model-only --mm-processor-cache-gb 0 --default-chat-template-kwargs '{"enable_thinking":false}' --no-enable-log-requests --no-enable-flashinfer-autotune --override-generation-config '{"max_new_tokens":null}' --kv-cache-memory-bytes 10737418240
+

examples/gb10-dgx-spark-nvfp4-optimized/README.md

@@ -0,0 +1,58 @@
+---
+title: GB10 / DGX Spark NVFP4 Optimized Sweep Specs
+author: Bob <dutifulbob@gmail.com>
+date: 2026-07-07
+---
+
+# GB10 / DGX Spark NVFP4 Optimized Sweep Specs
+
+Throughput-tuned default-sweep specs for four NVFP4 models on a GB10
+(DGX Spark, aarch64, ~122 GiB unified memory), vLLM 0.24.0:
+
+- `qwen27b.json` — `nvidia/Qwen3.6-27B-NVFP4` (hybrid linear-attention)
+- `qwen35b.json` — `nvidia/Qwen3.6-35B-A3B-NVFP4` (hybrid linear-attention MoE)
+- `gemma4-26b.json` — `nvidia/Gemma-4-26B-A4B-NVFP4`
+- `gemma4-31b.json` — `nvidia/Gemma-4-31B-IT-NVFP4` (dense)
+
+These are machine-specific: the `runner.vllm_command`, env
+(`CUTE_DSL_ARCH=sm_121a`, etc.), and ports point at a particular runtime.
+Edit them before use elsewhere. `PROFILES-old-vs-new.txt` records each
+model's old serve command vs the tuned one.
+
+## What was tuned (and why)
+
+Measured against a public DGX Spark benchmark, the original hand-authored
+specs served vLLM in a decode-hostile way. The tuned specs drop three flags
+and raise one:
+
+- **Drop `--mamba-cache-mode none`** — the Qwen3.6 models are hybrid
+  linear-attention (only every 4th layer is full attention; the rest keep a
+  recurrent SSM state). `none` disables caching that state, so every decode
+  step recomputes it over the whole sequence and decode throughput falls with
+  context. Default (cache on) keeps it flat.
+- **Drop `--enforce-eager`** — enables CUDA graphs.
+- **Drop `--no-async-scheduling`** — enables scheduler/compute overlap.
+- **Raise `--max-num-batched-tokens` to 16384** — large-context prefill was
+  step-throttled at 2048.
+
+Result on qwen27b: aggregate output throughput at concurrency rose ~20–30%
+(4k max-throughput reference c32: 157 → 186 tok/s; decode-8k c32: 61 → 80;
+decode-16k c16: 31 → 40). The Gemma specs were already free of the bad flags.
+
+## Notes on other levers (tested, not adopted here)
+
+- **MTP / speculative decoding** (`--speculative-config '{"method":"mtp",
+  "num_speculative_tokens":1}'`) roughly 1.6x's *single-stream* decode on the
+  Qwen models (they ship an MTP head), but adds overhead and memory pressure
+  under concurrency, so it is left off for max throughput. Enable it when
+  single-request latency matters more than aggregate throughput.
+- **flashinfer autotune + sampler** boot fine on sm_121 but gave no aggregate
+  throughput gain here, so they stay disabled.
+
+## Usage
+
+```sh
+localperf bench run --spec qwen27b.json \
+  --run-dir runs/qwen27b --artifact runs/qwen27b.sqlite --timeout 6h
+localperf artifact render runs/qwen27b.sqlite
+```