Add MiniMax M2.5 MXFP4 benchmark for MI355x vLLM v0.17.1 (TP=2,4)#827
Open
functionstackx wants to merge 5 commits intomainfrom
Open
Add MiniMax M2.5 MXFP4 benchmark for MI355x vLLM v0.17.1 (TP=2,4)#827functionstackx wants to merge 5 commits intomainfrom
functionstackx wants to merge 5 commits intomainfrom
Conversation
functionstackx
requested review from
billishyahao and
chunfangamd
as code owners
functionstackx
changed the title
Contributor
Author
|
@claude can u comment out TP=4 and only do TP=2 for now since TP=4 is bugged vllm-project/vllm#35637 |
Collaborator
|
Claude finished @functionstackx's task in 53s —— View job Commented out TP=4 for MiniMax M2.1 MXFP4Due to vllm-project/vllm#35637, TP=4 has been disabled. Only TP=2 will run.
Changes made:
|
Contributor
Author
|
@chunfangamd i am seeing good mxfp4 performance when comparing to FP8. just waiting for AMD to release mxfp4 version of MiniMax M2.5 now. currently AMD only have mxfp4 version of MiniMax M2.1. 
|
chunfangamd
changed the title
functionstackx
changed the title
functionstackx
changed the title
Add MiniMax M2.1 MXFP4 benchmark config for MI355x with vLLM v0.16.0. - Model: amd/MiniMax-M2.1-MXFP4 - TP=2 and TP=4 (matching MiniMax M2.5 FP8 pattern) - Only VLLM_ROCM_USE_AITER=1 env var (per Andy Luo recipe) - Seq lengths: 1k1k, 1k8k, 8k1k (conc 4-64) Closes #826 Co-authored-by: functionstackx <functionstackx@users.noreply.github.com>
TP=4 is bugged for this model per vllm-project/vllm#35637. Comment out TP=4 search-space entries, keeping only TP=2. Co-authored-by: functionstackx <functionstackx@users.noreply.github.com>
Author: hongxiayang - Keep AITER for attention but disable it specifically for MoE, so the fused MoE falls back to triton kernels that can handle N=384, when TP=4 and N=192 when TP=8. - Install the amd-quark library to fix the crash when TP=4 with VLLM_ROCM_USE_AITER_MOE=0.
functionstackx
force-pushed
the
claude/issue-826-20260301-0409
branch
2 times, most recently
from
bd10495 to
e849d65
Compare
functionstackx
force-pushed
the
claude/issue-826-20260301-0409
branch
from
e849d65 to
86cc700
Compare
functionstackx
force-pushed
the
claude/issue-826-20260301-0409
branch
from
86cc700 to
b82116b
Compare
functionstackx
force-pushed
the
claude/issue-826-20260301-0409
branch
from
b82116b to
7dd6063
Compare
- Model: amd/MiniMax-M2.1-MXFP4 → amd/MiniMax-M2.5-MXFP4 - Image: vllm/vllm-openai-rocm v0.16.0 → v0.17.1 - Rename config key and script from m2.1 to m2.5 - Update perf-changelog entry Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
functionstackx
force-pushed
the
claude/issue-826-20260301-0409
branch
from
7dd6063 to
5bc40e6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add MiniMax M2.5 MXFP4 benchmark config for MI355x with vLLM v0.17.1, now that AMD's MXFP4 checkpoint is out: https://huggingface.co/amd/MiniMax-M2.5-MXFP4
Closes #826
Generated with Claude Code