Conversation
Add comprehensive documentation for ATOM's vLLM OOT plugin mode covering architecture, supported models, configuration translation, attention integration, model wrapper, environment variables, and troubleshooting. Signed-off-by: wuhuikx <hattie.wu@amd.com>
| > | Component | Import / Entry Point | Purpose | | ||
| > |-----------|---------------------|---------| | ||
| > | `register_platform` | `atom.plugin.vllm.register:register_platform` | vLLM platform plugin — returns `ATOMPlatform` | | ||
| > | `register_model` | `atom.plugin.vllm.register:register_model` | vLLM general plugin — overrides model registry | |
Contributor
There was a problem hiding this comment.
For the function register_platform and register_model, we need to add more descriptions for both functions because they are the entrance of the ATOM OOT plugin of vLLM. https://github.com/ROCm/ATOM/blob/main/pyproject.toml#L33-L39
| ## 1. Overview | ||
|
|
||
| When ATOM is installed alongside vLLM, it automatically registers itself via | ||
| Python entry points. vLLM discovers ATOM at startup and delegates model |
Contributor
There was a problem hiding this comment.
Suggested change
| Python entry points. vLLM discovers ATOM at startup and delegates model | |
| Python entry points when launching the vLLM server. vLLM discovers ATOM at startup and delegates model |
| | `Qwen3MoeForCausalLM` | `atom.models.qwen3_moe.Qwen3MoeForCausalLM` | `ATOMMoEForCausalLM` | MoE | | ||
| | `DeepseekV3ForCausalLM` | `atom.models.deepseek_v2.DeepseekV3ForCausalLM` | `ATOMMoEForCausalLM` | MoE | | ||
| | `GptOssForCausalLM` | `atom.models.gpt_oss.GptOssForCausalLM` | `ATOMMoEForCausalLM` | MoE | | ||
| | `Glm4MoeForCausalLM` | `atom.models.glm4_moe.Glm4MoeForCausalLM` | `ATOMMoEForCausalLM` | MoE | |
Contributor
There was a problem hiding this comment.
Kimi is also supported but shares the same architecture with DS. Here we can tell to users that Kimi-K2 is supported.
| If you see `Fallback to original vLLM attention backend`, the attention plugin | ||
| is disabled. | ||
|
|
||
| ### Illegal memory access in MoE |
Contributor
There was a problem hiding this comment.
We can remove this section about illegal mem access issue because the issue is gone
Contributor
|
Do we need to add the SVG illustration in doc? |
Contributor
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Add comprehensive documentation for ATOM's vLLM OOT plugin mode covering
architecture, supported models, configuration translation, attention
integration, model wrapper, environment variables, and troubleshooting.