Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions astro.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,11 @@ export default defineConfig({
translations: { 'zh-CN': '沐曦 MACA' },
slug: 'hardware/metax_maca',
},
{
label: 'Mthreads MUSA',
translations: { 'zh-CN': '摩尔线程 MUSA' },
slug: 'hardware/musa',
},
],
},
{
Expand Down
42 changes: 42 additions & 0 deletions src/content/docs/en/getting_started/launch_xllm.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,3 +199,45 @@ do
--node_rank=$i \ > $LOG_FILE 2>&1 &
done
```

## Mthreads MUSA

```bash
#!/bin/bash
set -e

rm -rf core.*

export MUSA_VISIBLE_DEVICES=0

MODEL_PATH="/path/to/model/Qwen3.5-27B"
MASTER_NODE_ADDR="127.0.0.1:9748"
START_PORT=18000
START_DEVICE=0
LOG_DIR="log"
NNODES=1

mkdir -p $LOG_DIR

for (( i=0; i<$NNODES; i++ ))
do
PORT=$((START_PORT + i))
DEVICE=$((START_DEVICE + i))
LOG_FILE="$LOG_DIR/node_$i.log"
xllm \
--model $MODEL_PATH \
--devices="musa:$DEVICE" \
--port $PORT \
--master_node_addr=$MASTER_NODE_ADDR \
--nnodes=$NNODES \
--block_size=64 \
--max_memory_utilization=0.8 \
--enable_prefix_cache=false \
--enable_chunked_prefill=true \
--enable_schedule_overlap=true \
--enable_graph=true \
--node_rank=$i \ > $LOG_FILE 2>&1 &
done
```

See [Mthreads MUSA](/en/hardware/musa/) for details.
29 changes: 29 additions & 0 deletions src/content/docs/en/getting_started/quick_start.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,35 @@ docker run -it \
/bin/bash
```

### Mthreads MUSA

Image pull:

```bash
docker pull registry.mthreads.com/presale/devtech/xllm:0710
```

Container startup:

```bash
docker run -it \
--ipc=host \
--network=host \
--privileged \
--shm-size=128g \
--name xllm-musa \
--device=/dev/mtgpu0 \
--device=/dev/dri \
--group-add video \
--ulimit memlock=-1 \
-v $HOME:$HOME \
-w $HOME \
registry.mthreads.com/presale/devtech/xllm:0710 \
/bin/bash
```

See [Mthreads MUSA](/en/hardware/musa/) for full details.

## Build xllm

If you download a release image, i.e., an image with a version number in the tag, you can skip this step because the release image comes with a pre-compiled xllm binary, and call `xllm` directly.
Expand Down
84 changes: 84 additions & 0 deletions src/content/docs/en/hardware/musa.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
---
title: "Mthreads MUSA"
description: "Run xLLM on Mthreads MUSA GPUs."
sidebar:
order: 7
---

Use the MUSA device backend (`--devices=musa:<id>`) when running xLLM on Mthreads MUSA hardware.

## Image and Container Startup

Pull the Mthreads xLLM image:

```bash
docker pull registry.mthreads.com/presale/devtech/xllm:0710
```

Start the container:

```bash
docker run -it \
--ipc=host \
--network=host \
--privileged \
--shm-size=128g \
--name xllm-musa \
--device=/dev/mtgpu0 \
--device=/dev/dri \
--group-add video \
--ulimit memlock=-1 \
-v $HOME:$HOME \
-w $HOME \
registry.mthreads.com/presale/devtech/xllm:0710 \
/bin/bash
```

Add `--device=/dev/mtgpuN` for additional GPUs. Select the physical GPU with `export MUSA_VISIBLE_DEVICES=0`.

## Server Startup Command

```bash
#!/bin/bash
set -e

rm -rf core.*

export MUSA_VISIBLE_DEVICES=0

MODEL_PATH="/path/to/model/Qwen3.5-27B"
MASTER_NODE_ADDR="127.0.0.1:9748"
START_PORT=18000
START_DEVICE=0
LOG_DIR="log"
NNODES=1

mkdir -p $LOG_DIR

for (( i=0; i<$NNODES; i++ ))
do
PORT=$((START_PORT + i))
DEVICE=$((START_DEVICE + i))
LOG_FILE="$LOG_DIR/node_$i.log"
xllm \
--model $MODEL_PATH \
--devices="musa:$DEVICE" \
--port $PORT \
--master_node_addr=$MASTER_NODE_ADDR \
--nnodes=$NNODES \
--block_size=64 \
--max_memory_utilization=0.8 \
--enable_prefix_cache=false \
--enable_chunked_prefill=true \
--enable_schedule_overlap=true \
--enable_graph=true \
--node_rank=$i \ > $LOG_FILE 2>&1 &
done
```

For a single-device run, the logical device id usually starts at `0`. For multi-GPU or multi-node deployments, keep device ids, `--node_rank`, `--nnodes`, and service ports aligned with your topology.

## Notes

- Image and container startup are also listed in [Quick Start](/en/getting_started/quick_start/).
- Launch flags and multi-node setup are covered in [Launch xllm](/en/getting_started/launch_xllm/).
1 change: 1 addition & 0 deletions src/content/docs/en/hardware/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ xLLM supports multiple accelerator backends for large-scale model inference. Thi
- [Cambricon MLU](/en/hardware/cambricon_mlu/) - MLU backend setup and launch entry points.
- [Hygon DCU](/en/hardware/dcu/) - Hygon DCU backend setup and launch entry points.
- [MetaX MACA](/en/hardware/metax_maca/) - MetaX MACA backend setup and launch entry points.
- [Mthreads MUSA](/en/hardware/musa/) - Mthreads MUSA GPU image, build, and `--devices=musa` launch entry points.

## Common Workflow

Expand Down
42 changes: 42 additions & 0 deletions src/content/docs/zh/getting_started/launch_xllm.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,3 +199,45 @@ do
--node_rank=$i \ > $LOG_FILE 2>&1 &
done
```

## 摩尔线程 MUSA

```bash
#!/bin/bash
set -e

rm -rf core.*

export MUSA_VISIBLE_DEVICES=0

MODEL_PATH="/path/to/model/Qwen3.5-27B"
MASTER_NODE_ADDR="127.0.0.1:9748"
START_PORT=18000
START_DEVICE=0
LOG_DIR="log"
NNODES=1

mkdir -p $LOG_DIR

for (( i=0; i<$NNODES; i++ ))
do
PORT=$((START_PORT + i))
DEVICE=$((START_DEVICE + i))
LOG_FILE="$LOG_DIR/node_$i.log"
xllm \
--model $MODEL_PATH \
--devices="musa:$DEVICE" \
--port $PORT \
--master_node_addr=$MASTER_NODE_ADDR \
--nnodes=$NNODES \
--block_size=64 \
--max_memory_utilization=0.8 \
--enable_prefix_cache=false \
--enable_chunked_prefill=true \
--enable_schedule_overlap=true \
--enable_graph=true \
--node_rank=$i \ > $LOG_FILE 2>&1 &
done
```

详见 [摩尔线程 MUSA](/zh/hardware/musa/)。
29 changes: 29 additions & 0 deletions src/content/docs/zh/getting_started/quick_start.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,35 @@ docker run -it \
/bin/bash
```

### 摩尔线程 MUSA

镜像拉取命令:

```bash
docker pull registry.mthreads.com/presale/devtech/xllm:0710
```

容器启动命令:

```bash
docker run -it \
--ipc=host \
--network=host \
--privileged \
--shm-size=128g \
--name xllm-musa \
--device=/dev/mtgpu0 \
--device=/dev/dri \
--group-add video \
--ulimit memlock=-1 \
-v $HOME:$HOME \
-w $HOME \
registry.mthreads.com/presale/devtech/xllm:0710 \
/bin/bash
```

更多细节见 [摩尔线程 MUSA](/zh/hardware/musa/)。

## 编译xllm

如果下载的是release镜像,即tag中带有版本号的镜像,可以跳过此步,因为release镜像自带编译好的xllm二进制文件,可以直接调用`xllm`。
Expand Down
84 changes: 84 additions & 0 deletions src/content/docs/zh/hardware/musa.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
---
title: "摩尔线程 MUSA"
description: "在摩尔线程 MUSA GPU 上使用 xLLM 进行大模型推理部署。"
sidebar:
order: 7
---

在摩尔线程(Mthreads)MUSA GPU 上部署 xLLM 时使用 MUSA 设备后端(`--devices=musa:<id>`)。

## 镜像和容器启动命令

拉取摩尔线程 xLLM 镜像:

```bash
docker pull registry.mthreads.com/presale/devtech/xllm:0710
```

启动容器:

```bash
docker run -it \
--ipc=host \
--network=host \
--privileged \
--shm-size=128g \
--name xllm-musa \
--device=/dev/mtgpu0 \
--device=/dev/dri \
--group-add video \
--ulimit memlock=-1 \
-v $HOME:$HOME \
-w $HOME \
registry.mthreads.com/presale/devtech/xllm:0710 \
/bin/bash
```

按实际 GPU 数量增加 `--device=/dev/mtgpuN`;选择物理卡使用 `export MUSA_VISIBLE_DEVICES=0`。

## 服务启动命令

```bash
#!/bin/bash
set -e

rm -rf core.*

export MUSA_VISIBLE_DEVICES=0

MODEL_PATH="/path/to/model/Qwen3.5-27B"
MASTER_NODE_ADDR="127.0.0.1:9748"
START_PORT=18000
START_DEVICE=0
LOG_DIR="log"
NNODES=1

mkdir -p $LOG_DIR

for (( i=0; i<$NNODES; i++ ))
do
PORT=$((START_PORT + i))
DEVICE=$((START_DEVICE + i))
LOG_FILE="$LOG_DIR/node_$i.log"
xllm \
--model $MODEL_PATH \
--devices="musa:$DEVICE" \
--port $PORT \
--master_node_addr=$MASTER_NODE_ADDR \
--nnodes=$NNODES \
--block_size=64 \
--max_memory_utilization=0.8 \
--enable_prefix_cache=false \
--enable_chunked_prefill=true \
--enable_schedule_overlap=true \
--enable_graph=true \
--node_rank=$i \ > $LOG_FILE 2>&1 &
done
```

单卡部署时逻辑设备号通常从 `0` 开始。多卡或多机场景中,需要让设备编号、`--node_rank`、`--nnodes` 和服务端口与部署拓扑保持一致。

## 注意事项

- 镜像与容器启动说明亦见 [快速开始](/zh/getting_started/quick_start/)。
- 启动参数与多机部署见 [启动 xllm](/zh/getting_started/launch_xllm/)。
1 change: 1 addition & 0 deletions src/content/docs/zh/hardware/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ xLLM 支持多种加速器后端,用于大模型推理部署。本章节汇总
- [寒武纪 MLU](/zh/hardware/cambricon_mlu/) - MLU 后端环境和启动入口。
- [海光 DCU](/zh/hardware/dcu/) - 海光 DCU 后端环境和启动入口。
- [沐曦 MACA](/zh/hardware/metax_maca/) - 沐曦 MACA 后端环境和启动入口。
- [摩尔线程 MUSA](/zh/hardware/musa/) - 摩尔线程 MUSA GPU 镜像启动入口。

## 通用流程

Expand Down
Loading