feat(glm-dsa): GLM-5.2 sparse-MLA construct-time determinism (deep-align)#1695
Open
yhl-amd wants to merge 4 commits into
Open
feat(glm-dsa): GLM-5.2 sparse-MLA construct-time determinism (deep-align)#1695yhl-amd wants to merge 4 commits into
yhl-amd wants to merge 4 commits into
Conversation
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire frequency_penalty/presence_penalty through sampling_params -> sequence token_counts -> scheduler sparse penalty triples -> model_runner _apply_penalties (bf16-cast to match logits). Adds opt-in _check_logits_finite (ATOM_DEBUG_LOGIT_NANCHECK) and tests/test_sampler.py. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…+ sampling defaults Port onto latest mainline (673e6e0): bring the GLM-5.2 <tool_call>...<arg_key> tool-call parser (mainline's tool_parser lacks the arg_key format), add vLLM-style sliding-window incremental detokenization (no U+FFFD split of CJK/emoji during streaming), and ANTHROPIC_DEFAULT_TOP_P=0.9 + temperature handling for /v1/messages. abort-on-disconnect is already in mainline (skipped). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e 1/3/4) Port cause 1/3/4 onto mainline (673e6e0) via native's own unfused paths, NOT the old branch's dead-code deletion: - cause 1 (linear.py): small-M bf16 projections use deterministic F.linear instead of nondeterministic aiter tgemm.mm. - cause 3: ENABLE_ALLREDUCE_RMSNORM_FUSION=False (separate all-reduce + RMSNorm). - cause 4: ENABLE_DS_INDEXER_QK_ROPE_CACHE_FUSION=False + ENABLE_GLM_FUSED_INDEXER =False (mainline already has the unfused branches; just flip the flags). cause 2 (deterministic top-k tie-break) is handled in aiter's topk kernel + TOPK_FORCE_PATH=o, so no deepseek_v2 change is needed for it here. Deliberately does NOT disable QKNORM/INPUT_RMSNORM fusions (their unfused path is buggier). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
🏷️ CI GuideRuns automatically on every eligible PR before approval:
Heavy model tests:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Port of the GLM-5.2 sparse-MLA determinism work onto current
main(673e6e0). Makes the sparse-MLA indexer pipeline construct-time deterministic (NV-aligned), which eliminates cross-rank / run-to-run KV-subset divergence that corrupts long generations on gfx950.Commits:
deepseek_v2.py,linear.py): small-M bf16 projections use deterministicF.linear(not non-deterministictgemm.mm);ENABLE_ALLREDUCE_RMSNORM_FUSION/ENABLE_DS_INDEXER_QK_ROPE_CACHE_FUSION/ENABLE_GLM_FUSED_INDEXERforced off (mainline's own unfused branches). Deliberately does NOT disable QKNORM/INPUT_RMSNORM fusions (their unfused path is buggier).<tool_call>...<arg_key>parser + vLLM-style streaming UTF-8 detok (no U+FFFD split of CJK/emoji) +ANTHROPIC_DEFAULT_TOP_P=0.9.cause 2 (deterministic top-k tie-break) is in the companion aiter PR (ROCm/aiter#4376) +
TOPK_FORCE_PATH=o.Verification (GLM-5.2-MXFP4, tp8, gfx950)
Caveats for review
tool_parser.pyis ported wholesale from the GLM-5.2 branch and overwrites mainline's current tool_parser; it needs reconciliation with any newer mainline tool-parser handlers before merge.abort-on-client-disconnectcommits are already in mainline and were intentionally excluded.🤖 Generated with Claude Code