feat: support GLM DSA sharing top-k and MTP export.#1823
Conversation
sanlio36
changed the title
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces support for GLM-5/5.1/5.2 models, adds W4A8 dynamic quantization preprocessing, implements sticky prefix routing in the block manager pool, and adds an MTP top-k fallback mechanism for DeepSeek V32. The review feedback highlights critical issues, including potential crashes from calling .sizes() on undefined tensors in deepseek_v32_decoder_loader.cpp. Additionally, several style guide violations were identified: C++ helper functions in deepseek_v32_decoder_loader.cpp should use snake_case instead of PascalCase, file-local Python functions in export_mtp.py must be prefixed with a leading underscore, and file-local C++ functions in hf_model_loader.cpp and mtp_worker_impl.cpp should be placed in anonymous namespaces.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
sanlio36
changed the title
sanlio36
force-pushed
the
preview/glm-5.2_merge
branch
from
a6a2503 to
6a3b860
Compare
sanlio36
force-pushed
the
preview/glm-5.2_merge
branch
2 times, most recently
from
09b47f5 to
df96c63
Compare
sanlio36
requested review from
Clement-Wang26,
DongheJin,
DragonFive,
JimHsiung,
Kang-Meng,
RobbieLeung,
liujinguang0125,
liutongxuan,
walsonyang,
xiao-yu-chen,
yingxudeng,
yq33victor and
zhang-minchao
as code owners
| return worker.execute_no_sync_on_stream(processed_input, compute_stream); | ||
| std::optional<ForwardOutput> output = | ||
| worker.execute_no_sync_on_stream(processed_input, compute_stream); | ||
| const int32_t ret = compute_stream.synchronize(); |
There was a problem hiding this comment.
请问这里为什么要加同步？Qwen3.5这里加了后mtp会增加空泡导致性能下降
There was a problem hiding this comment.
猜测应该是遇到茂军老师一直反馈的一下报错，之后这么改的。但是这么改是否合理有待考量：
F20260627 17:21:22.615046 3253981 mtp_worker_impl.cpp:318] Check failed: state.token_id == token_id (539 vs. 19) MTP decode target state token mismatch, request_id=chatcmpl-2551906557222686169-xaLgCSacqP7RJtXV2rzbQv
*** Check failure stack trace: ***
@ 0x6f6310 google::LogMessage::SendToLog()
@ 0x6f2e0c google::LogMessage::Flush()
@ 0x6f68e0 google::LogMessageFatal::~LogMessageFatal()
@ 0xc770f0 xllm::MTPWorkerImpl::step_decode()
@ 0xcbd35c xllm::SpeculativeWorkerImpl::step()
@ 0xc02388 _ZZN4xllm10WorkerImpl10step_asyncERKNS_12ForwardInputEENUlvE_clEv
@ 0x31e3720 xllm::ThreadPool::internal_loop()
@ 0x31e4410 _ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJZN4xllm10ThreadPoolC4EmN5folly8FunctionIFvvEEEbRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEUlvE_EEEEE6_M_runEv
@ 0xffff878c9da4 (unknown)
@ 0xffff7825fbb4 (unknown)
@ 0xffff782c7f4c (unknown)
@ (nil) (unknown)
There was a problem hiding this comment.
当前改动为规避dp+mtp的精度bug，可在非dp场景下不走这个同步，或进一步定位乱序的问题根因。
| return topk_indices.index_select(/*dim=*/0, index); | ||
| } | ||
|
|
||
| std::optional<ForwardOutput> run_llm_no_sync_impl( |
There was a problem hiding this comment.
这里函数名是no_sync，里面却加了sync，需要看一下是修改函数名还是把sync去掉
6 participants
Description
Related Issues
Change Type
Pull Request Checklist
Thank you for contributing to xLLM. Before requesting review, please make sure the following items are complete.
PR Title and Commit Messages
<type>: <subject>.Pre-commit Checks
pre-commitby runningpip install pre-commitor an equivalent command.pre-commit install.pre-commit run --all-filesand fixed any reported issues.Self Review
.agents/skills/code-review/references/custom-code-style.md, especially code written or assisted by AI.mainbranch.Build and Test Coverage
python setup.py build testhas passed on a CUDA machine.python setup.py build testhas passed on an NPU machine.python setup.py build testhas passed on an MLU machine.Reviewer Notes