SM120 (RTX PRO 6000) MoE backend gaps: MoeConfig.backend schema missing MegaMoE entries, and no EP-capable path for this hardware #16360
ItamarTerem
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi TensorRT-LLM team,
Deploying a large MoE model (Kimi-K2.7-Code-NVFP4, 384 experts) on 8x RTX PRO 6000 (SM120) via the pytorch backend, trtllm-serve, version 1.3.0rc20. Two related findings we wanted to raise:
Net result: there is currently no correct + EP-capable MoE path for SM120 in this stack, which limits per-GPU memory headroom for large-expert-count models on this hardware class.
Questions:
Happy to share full repro steps/configs if helpful. Thanks for your work on this — appreciate the pointers.
Beta Was this translation helpful? Give feedback.
All reactions