diff --git a/astro.config.mjs b/astro.config.mjs index e4889ca..464a1ae 100644 --- a/astro.config.mjs +++ b/astro.config.mjs @@ -93,6 +93,11 @@ export default defineConfig({ translations: { 'zh-CN': '沐曦 MACA' }, slug: 'hardware/metax_maca', }, + { + label: 'Mthreads MUSA', + translations: { 'zh-CN': '摩尔线程 MUSA' }, + slug: 'hardware/musa', + }, ], }, { diff --git a/src/content/docs/en/getting_started/launch_xllm.md b/src/content/docs/en/getting_started/launch_xllm.md index c4ce578..028a639 100644 --- a/src/content/docs/en/getting_started/launch_xllm.md +++ b/src/content/docs/en/getting_started/launch_xllm.md @@ -199,3 +199,45 @@ do --node_rank=$i \ > $LOG_FILE 2>&1 & done ``` + +## Mthreads MUSA + +```bash +#!/bin/bash +set -e + +rm -rf core.* + +export MUSA_VISIBLE_DEVICES=0 + +MODEL_PATH="/path/to/model/Qwen3.5-27B" +MASTER_NODE_ADDR="127.0.0.1:9748" +START_PORT=18000 +START_DEVICE=0 +LOG_DIR="log" +NNODES=1 + +mkdir -p $LOG_DIR + +for (( i=0; i<$NNODES; i++ )) +do + PORT=$((START_PORT + i)) + DEVICE=$((START_DEVICE + i)) + LOG_FILE="$LOG_DIR/node_$i.log" + xllm \ + --model $MODEL_PATH \ + --devices="musa:$DEVICE" \ + --port $PORT \ + --master_node_addr=$MASTER_NODE_ADDR \ + --nnodes=$NNODES \ + --block_size=64 \ + --max_memory_utilization=0.8 \ + --enable_prefix_cache=false \ + --enable_chunked_prefill=true \ + --enable_schedule_overlap=true \ + --enable_graph=true \ + --node_rank=$i \ > $LOG_FILE 2>&1 & +done +``` + +See [Mthreads MUSA](/en/hardware/musa/) for details. diff --git a/src/content/docs/en/getting_started/quick_start.md b/src/content/docs/en/getting_started/quick_start.md index 3281d63..cd1d534 100644 --- a/src/content/docs/en/getting_started/quick_start.md +++ b/src/content/docs/en/getting_started/quick_start.md @@ -142,6 +142,35 @@ docker run -it \ /bin/bash ``` +### Mthreads MUSA + +Image pull: + +```bash +docker pull registry.mthreads.com/presale/devtech/xllm:0710 +``` + +Container startup: + +```bash +docker run -it \ + --ipc=host \ + --network=host \ + --privileged \ + --shm-size=128g \ + --name xllm-musa \ + --device=/dev/mtgpu0 \ + --device=/dev/dri \ + --group-add video \ + --ulimit memlock=-1 \ + -v $HOME:$HOME \ + -w $HOME \ + registry.mthreads.com/presale/devtech/xllm:0710 \ + /bin/bash +``` + +See [Mthreads MUSA](/en/hardware/musa/) for full details. + ## Build xllm If you download a release image, i.e., an image with a version number in the tag, you can skip this step because the release image comes with a pre-compiled xllm binary, and call `xllm` directly. diff --git a/src/content/docs/en/hardware/musa.md b/src/content/docs/en/hardware/musa.md new file mode 100644 index 0000000..5471398 --- /dev/null +++ b/src/content/docs/en/hardware/musa.md @@ -0,0 +1,84 @@ +--- +title: "Mthreads MUSA" +description: "Run xLLM on Mthreads MUSA GPUs." +sidebar: + order: 7 +--- + +Use the MUSA device backend (`--devices=musa:`) when running xLLM on Mthreads MUSA hardware. + +## Image and Container Startup + +Pull the Mthreads xLLM image: + +```bash +docker pull registry.mthreads.com/presale/devtech/xllm:0710 +``` + +Start the container: + +```bash +docker run -it \ + --ipc=host \ + --network=host \ + --privileged \ + --shm-size=128g \ + --name xllm-musa \ + --device=/dev/mtgpu0 \ + --device=/dev/dri \ + --group-add video \ + --ulimit memlock=-1 \ + -v $HOME:$HOME \ + -w $HOME \ + registry.mthreads.com/presale/devtech/xllm:0710 \ + /bin/bash +``` + +Add `--device=/dev/mtgpuN` for additional GPUs. Select the physical GPU with `export MUSA_VISIBLE_DEVICES=0`. + +## Server Startup Command + +```bash +#!/bin/bash +set -e + +rm -rf core.* + +export MUSA_VISIBLE_DEVICES=0 + +MODEL_PATH="/path/to/model/Qwen3.5-27B" +MASTER_NODE_ADDR="127.0.0.1:9748" +START_PORT=18000 +START_DEVICE=0 +LOG_DIR="log" +NNODES=1 + +mkdir -p $LOG_DIR + +for (( i=0; i<$NNODES; i++ )) +do + PORT=$((START_PORT + i)) + DEVICE=$((START_DEVICE + i)) + LOG_FILE="$LOG_DIR/node_$i.log" + xllm \ + --model $MODEL_PATH \ + --devices="musa:$DEVICE" \ + --port $PORT \ + --master_node_addr=$MASTER_NODE_ADDR \ + --nnodes=$NNODES \ + --block_size=64 \ + --max_memory_utilization=0.8 \ + --enable_prefix_cache=false \ + --enable_chunked_prefill=true \ + --enable_schedule_overlap=true \ + --enable_graph=true \ + --node_rank=$i \ > $LOG_FILE 2>&1 & +done +``` + +For a single-device run, the logical device id usually starts at `0`. For multi-GPU or multi-node deployments, keep device ids, `--node_rank`, `--nnodes`, and service ports aligned with your topology. + +## Notes + +- Image and container startup are also listed in [Quick Start](/en/getting_started/quick_start/). +- Launch flags and multi-node setup are covered in [Launch xllm](/en/getting_started/launch_xllm/). diff --git a/src/content/docs/en/hardware/overview.md b/src/content/docs/en/hardware/overview.md index ee7ca08..417011e 100644 --- a/src/content/docs/en/hardware/overview.md +++ b/src/content/docs/en/hardware/overview.md @@ -14,6 +14,7 @@ xLLM supports multiple accelerator backends for large-scale model inference. Thi - [Cambricon MLU](/en/hardware/cambricon_mlu/) - MLU backend setup and launch entry points. - [Hygon DCU](/en/hardware/dcu/) - Hygon DCU backend setup and launch entry points. - [MetaX MACA](/en/hardware/metax_maca/) - MetaX MACA backend setup and launch entry points. +- [Mthreads MUSA](/en/hardware/musa/) - Mthreads MUSA GPU image, build, and `--devices=musa` launch entry points. ## Common Workflow diff --git a/src/content/docs/zh/getting_started/launch_xllm.md b/src/content/docs/zh/getting_started/launch_xllm.md index ed5c15e..2855124 100644 --- a/src/content/docs/zh/getting_started/launch_xllm.md +++ b/src/content/docs/zh/getting_started/launch_xllm.md @@ -199,3 +199,45 @@ do --node_rank=$i \ > $LOG_FILE 2>&1 & done ``` + +## 摩尔线程 MUSA + +```bash +#!/bin/bash +set -e + +rm -rf core.* + +export MUSA_VISIBLE_DEVICES=0 + +MODEL_PATH="/path/to/model/Qwen3.5-27B" +MASTER_NODE_ADDR="127.0.0.1:9748" +START_PORT=18000 +START_DEVICE=0 +LOG_DIR="log" +NNODES=1 + +mkdir -p $LOG_DIR + +for (( i=0; i<$NNODES; i++ )) +do + PORT=$((START_PORT + i)) + DEVICE=$((START_DEVICE + i)) + LOG_FILE="$LOG_DIR/node_$i.log" + xllm \ + --model $MODEL_PATH \ + --devices="musa:$DEVICE" \ + --port $PORT \ + --master_node_addr=$MASTER_NODE_ADDR \ + --nnodes=$NNODES \ + --block_size=64 \ + --max_memory_utilization=0.8 \ + --enable_prefix_cache=false \ + --enable_chunked_prefill=true \ + --enable_schedule_overlap=true \ + --enable_graph=true \ + --node_rank=$i \ > $LOG_FILE 2>&1 & +done +``` + +详见 [摩尔线程 MUSA](/zh/hardware/musa/)。 diff --git a/src/content/docs/zh/getting_started/quick_start.md b/src/content/docs/zh/getting_started/quick_start.md index e463e9d..970601a 100644 --- a/src/content/docs/zh/getting_started/quick_start.md +++ b/src/content/docs/zh/getting_started/quick_start.md @@ -142,6 +142,35 @@ docker run -it \ /bin/bash ``` +### 摩尔线程 MUSA + +镜像拉取命令： + +```bash +docker pull registry.mthreads.com/presale/devtech/xllm:0710 +``` + +容器启动命令： + +```bash +docker run -it \ + --ipc=host \ + --network=host \ + --privileged \ + --shm-size=128g \ + --name xllm-musa \ + --device=/dev/mtgpu0 \ + --device=/dev/dri \ + --group-add video \ + --ulimit memlock=-1 \ + -v $HOME:$HOME \ + -w $HOME \ + registry.mthreads.com/presale/devtech/xllm:0710 \ + /bin/bash +``` + +更多细节见 [摩尔线程 MUSA](/zh/hardware/musa/)。 + ## 编译xllm 如果下载的是release镜像，即tag中带有版本号的镜像，可以跳过此步，因为release镜像自带编译好的xllm二进制文件，可以直接调用`xllm`。 diff --git a/src/content/docs/zh/hardware/musa.md b/src/content/docs/zh/hardware/musa.md new file mode 100644 index 0000000..2a2fae5 --- /dev/null +++ b/src/content/docs/zh/hardware/musa.md @@ -0,0 +1,84 @@ +--- +title: "摩尔线程 MUSA" +description: "在摩尔线程 MUSA GPU 上使用 xLLM 进行大模型推理部署。" +sidebar: + order: 7 +--- + +在摩尔线程（Mthreads）MUSA GPU 上部署 xLLM 时使用 MUSA 设备后端（`--devices=musa:`）。 + +## 镜像和容器启动命令 + +拉取摩尔线程 xLLM 镜像： + +```bash +docker pull registry.mthreads.com/presale/devtech/xllm:0710 +``` + +启动容器： + +```bash +docker run -it \ + --ipc=host \ + --network=host \ + --privileged \ + --shm-size=128g \ + --name xllm-musa \ + --device=/dev/mtgpu0 \ + --device=/dev/dri \ + --group-add video \ + --ulimit memlock=-1 \ + -v $HOME:$HOME \ + -w $HOME \ + registry.mthreads.com/presale/devtech/xllm:0710 \ + /bin/bash +``` + +按实际 GPU 数量增加 `--device=/dev/mtgpuN`；选择物理卡使用 `export MUSA_VISIBLE_DEVICES=0`。 + +## 服务启动命令 + +```bash +#!/bin/bash +set -e + +rm -rf core.* + +export MUSA_VISIBLE_DEVICES=0 + +MODEL_PATH="/path/to/model/Qwen3.5-27B" +MASTER_NODE_ADDR="127.0.0.1:9748" +START_PORT=18000 +START_DEVICE=0 +LOG_DIR="log" +NNODES=1 + +mkdir -p $LOG_DIR + +for (( i=0; i<$NNODES; i++ )) +do + PORT=$((START_PORT + i)) + DEVICE=$((START_DEVICE + i)) + LOG_FILE="$LOG_DIR/node_$i.log" + xllm \ + --model $MODEL_PATH \ + --devices="musa:$DEVICE" \ + --port $PORT \ + --master_node_addr=$MASTER_NODE_ADDR \ + --nnodes=$NNODES \ + --block_size=64 \ + --max_memory_utilization=0.8 \ + --enable_prefix_cache=false \ + --enable_chunked_prefill=true \ + --enable_schedule_overlap=true \ + --enable_graph=true \ + --node_rank=$i \ > $LOG_FILE 2>&1 & +done +``` + +单卡部署时逻辑设备号通常从 `0` 开始。多卡或多机场景中，需要让设备编号、`--node_rank`、`--nnodes` 和服务端口与部署拓扑保持一致。 + +## 注意事项 + +- 镜像与容器启动说明亦见 [快速开始](/zh/getting_started/quick_start/)。 +- 启动参数与多机部署见 [启动 xllm](/zh/getting_started/launch_xllm/)。 diff --git a/src/content/docs/zh/hardware/overview.md b/src/content/docs/zh/hardware/overview.md index 99f2930..39b5990 100644 --- a/src/content/docs/zh/hardware/overview.md +++ b/src/content/docs/zh/hardware/overview.md @@ -14,6 +14,7 @@ xLLM 支持多种加速器后端，用于大模型推理部署。本章节汇总 - [寒武纪 MLU](/zh/hardware/cambricon_mlu/) - MLU 后端环境和启动入口。 - [海光 DCU](/zh/hardware/dcu/) - 海光 DCU 后端环境和启动入口。 - [沐曦 MACA](/zh/hardware/metax_maca/) - 沐曦 MACA 后端环境和启动入口。 +- [摩尔线程 MUSA](/zh/hardware/musa/) - 摩尔线程 MUSA GPU 镜像启动入口。 ## 通用流程