Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions astro.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,11 @@ export default defineConfig({
translations: { 'zh-CN': '海光 DCU' },
slug: 'hardware/dcu',
},
{
label: 'MetaX MACA',
translations: { 'zh-CN': '沐曦 MACA' },
slug: 'hardware/metax_maca',
},
],
},
{
Expand Down
40 changes: 40 additions & 0 deletions src/content/docs/en/getting_started/launch_xllm.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,3 +159,43 @@ do
--node_rank=$i \ > $LOG_FILE 2>&1 &
done
```

## MetaX MACA

```bash
#!/bin/bash
set -e

rm -rf core.*

export CUDA_VISIBLE_DEVICES=0
export FLASHINFER_OPS_PATH=/opt/conda/lib/python3.10/site-packages/flashinfer/data/aot/

MODEL_PATH="/path/to/model/Qwen3-8B"
MASTER_NODE_ADDR="127.0.0.1:9748"
START_PORT=18000
START_DEVICE=0
LOG_DIR="log"
NNODES=1

mkdir -p $LOG_DIR

for (( i=0; i<$NNODES; i++ ))
do
PORT=$((START_PORT + i))
DEVICE=$((START_DEVICE + i))
LOG_FILE="$LOG_DIR/node_$i.log"
xllm \
--model $MODEL_PATH \
--devices="cuda:$DEVICE" \
--port $PORT \
--nnodes=$NNODES \
--master_node_addr=$MASTER_NODE_ADDR \
--block_size=128 \
--max_memory_utilization=0.86 \
--enable_prefix_cache=false \
--enable_chunked_prefill=false \
--enable_schedule_overlap=true \
--node_rank=$i \ > $LOG_FILE 2>&1 &
done
```
30 changes: 30 additions & 0 deletions src/content/docs/en/getting_started/quick_start.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,36 @@ docker run -it \
/bin/bash
```

### MetaX MACA

Below are our pre-built dev image.
```bash
docker pull pub-registry1.metax-tech.com/dev-m01421/xllm-maca3.7.1.9:v1
```

Container startup command:
```bash
docker run -it \
--ipc=host \
-u 0 \
--name xllm-maca \
--network=host \
--privileged=true \
--shm-size 100gb \
--device=/dev/mxcd \
--device=/dev/dri \
--device=/dev/infiniband \
--security-opt seccomp=unconfined \
--security-opt apparmor=unconfined \
--group-add video \
--ulimit memlock=-1 \
-v /opt/maca:/opt/maca \
-v $HOME:$HOME \
-w $HOME \
<docker_image_name> \
/bin/bash
```

## Build xllm

If you download a release image, i.e., an image with a version number in the tag, you can skip this step because the release image comes with a pre-compiled xllm binary, and call `xllm` directly.
Expand Down
88 changes: 88 additions & 0 deletions src/content/docs/en/hardware/metax_maca.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
---
title: "MetaX MACA"
description: "Run xLLM on MetaX MACA devices with the MetaX MACA backend."
sidebar:
order: 6
---

Use the MetaX MACA backend when running xLLM on MetaX MACA hardware.

## Image and Container Startup

Pull the MetaX MACA development image:

```bash
docker pull pub-registry1.metax-tech.com/dev-m01421/xllm-maca3.7.1.9:v1
```

Start the container:

```bash
docker run -it \
--ipc=host \
-u 0 \
--name xllm-maca \
--network=host \
--privileged=true \
--shm-size 100gb \
--device=/dev/mxcd \
--device=/dev/dri \
--device=/dev/infiniband \
--security-opt seccomp=unconfined \
--security-opt apparmor=unconfined \
--group-add video \
--ulimit memlock=-1 \
-v /opt/maca:/opt/maca \
-v $HOME:$HOME \
-w $HOME \
<docker_image_name> \
/bin/bash
```

## Server Startup Command

```bash
#!/bin/bash
set -e

rm -rf core.*

export CUDA_VISIBLE_DEVICES=0
export FLASHINFER_OPS_PATH=/opt/conda/lib/python3.10/site-packages/flashinfer/data/aot/

MODEL_PATH="/path/to/model/Qwen3-8B"
MASTER_NODE_ADDR="127.0.0.1:9748"
START_PORT=18000
START_DEVICE=0
LOG_DIR="log"
NNODES=1

mkdir -p $LOG_DIR

for (( i=0; i<$NNODES; i++ ))
do
PORT=$((START_PORT + i))
DEVICE=$((START_DEVICE + i))
LOG_FILE="$LOG_DIR/node_$i.log"
xllm \
--model $MODEL_PATH \
--devices="cuda:$DEVICE" \
--port $PORT \
--nnodes=$NNODES \
--master_node_addr=$MASTER_NODE_ADDR \
--block_size=128 \
--max_memory_utilization=0.86 \
--enable_prefix_cache=false \
--enable_chunked_prefill=false \
--enable_schedule_overlap=true \
--node_rank=$i \ > $LOG_FILE 2>&1 &
done
```

For a single-device run, `<device-id>` usually starts from `0`. For multi-worker deployments, keep device ids, `--node_rank`, `--nnodes`, and service ports aligned.

## Notes

- The current docs list a pre-built MetaX MACA development image in [Quick Start](/en/getting_started/quick_start/).
- The MetaX MACA container startup command requires device mounts such as `/dev/mxcd`, `/dev/dri`, and `/dev/infiniband`; the command above includes these mounts.
- Build xllm with MetaX MACA: python setup.py build --device maca
1 change: 1 addition & 0 deletions src/content/docs/en/hardware/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ xLLM supports multiple accelerator backends for large-scale model inference. Thi
- [Ascend NPU](/en/hardware/ascend_npu/) - Ascend NPU setup, runtime environment, and HCCL launch notes.
- [Cambricon MLU](/en/hardware/cambricon_mlu/) - MLU backend setup and launch entry points.
- [Hygon DCU](/en/hardware/dcu/) - Hygon DCU backend setup and launch entry points.
- [MetaX MACA](/en/hardware/metax_maca/) - MetaX MACA backend setup and launch entry points.

## Common Workflow

Expand Down
40 changes: 40 additions & 0 deletions src/content/docs/zh/getting_started/launch_xllm.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,3 +159,43 @@ do
--node_rank=$i \ > $LOG_FILE 2>&1 &
done
```

## 沐曦 MACA

```bash
#!/bin/bash
set -e

rm -rf core.*

export CUDA_VISIBLE_DEVICES=0
export FLASHINFER_OPS_PATH=/opt/conda/lib/python3.10/site-packages/flashinfer/data/aot/

MODEL_PATH="/path/to/model/Qwen3-8B"
MASTER_NODE_ADDR="127.0.0.1:9748"
START_PORT=18000
START_DEVICE=0
LOG_DIR="log"
NNODES=1

mkdir -p $LOG_DIR

for (( i=0; i<$NNODES; i++ ))
do
PORT=$((START_PORT + i))
DEVICE=$((START_DEVICE + i))
LOG_FILE="$LOG_DIR/node_$i.log"
xllm \
--model $MODEL_PATH \
--devices="cuda:$DEVICE" \
--port $PORT \
--nnodes=$NNODES \
--master_node_addr=$MASTER_NODE_ADDR \
--block_size=128 \
--max_memory_utilization=0.86 \
--enable_prefix_cache=false \
--enable_chunked_prefill=false \
--enable_schedule_overlap=true \
--node_rank=$i \ > $LOG_FILE 2>&1 &
done
```
30 changes: 30 additions & 0 deletions src/content/docs/zh/getting_started/quick_start.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,36 @@ docker run -it \
/bin/bash
```

### 沐曦 MACA

下面是我们构建好的开发镜像。
```bash
docker pull pub-registry1.metax-tech.com/dev-m01421/xllm-maca3.7.1.9:v1
```

容器启动命令如下:
```bash
docker run -it \
--ipc=host \
-u 0 \
--name xllm-maca \
--network=host \
--privileged=true \
--shm-size 100gb \
--device=/dev/mxcd \
--device=/dev/dri \
--device=/dev/infiniband \
--security-opt seccomp=unconfined \
--security-opt apparmor=unconfined \
--group-add video \
--ulimit memlock=-1 \
-v /opt/maca:/opt/maca \
-v $HOME:$HOME \
-w $HOME \
<docker_image_name> \
/bin/bash
```

## 编译xllm

如果下载的是release镜像,即tag中带有版本号的镜像,可以跳过此步,因为release镜像自带编译好的xllm二进制文件,可以直接调用`xllm`。
Expand Down
88 changes: 88 additions & 0 deletions src/content/docs/zh/hardware/metax_maca.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
---
title: "沐曦 MACA"
description: "使用沐曦 MACA 后端在沐曦 MACA 硬件上运行 xLLM。"
sidebar:
order: 6
---

在沐曦 MACA 硬件上部署 xLLM 时使用沐曦 MACA 后端。

## 镜像和容器启动命令

拉取沐曦 MACA 开发镜像:

```bash
docker pull pub-registry1.metax-tech.com/dev-m01421/xllm-maca3.7.1.9:v1
```

启动容器:

```bash
docker run -it \
--ipc=host \
-u 0 \
--name xllm-maca \
--network=host \
--privileged=true \
--shm-size 100gb \
--device=/dev/mxcd \
--device=/dev/dri \
--device=/dev/infiniband \
--security-opt seccomp=unconfined \
--security-opt apparmor=unconfined \
--group-add video \
--ulimit memlock=-1 \
-v /opt/maca:/opt/maca \
-v $HOME:$HOME \
-w $HOME \
<docker_image_name> \
/bin/bash
```

## 服务启动命令

```bash
#!/bin/bash
set -e

rm -rf core.*

export CUDA_VISIBLE_DEVICES=0
export FLASHINFER_OPS_PATH=/opt/conda/lib/python3.10/site-packages/flashinfer/data/aot/

MODEL_PATH="/path/to/model/Qwen3-8B"
MASTER_NODE_ADDR="127.0.0.1:9748"
START_PORT=18000
START_DEVICE=0
LOG_DIR="log"
NNODES=1

mkdir -p $LOG_DIR

for (( i=0; i<$NNODES; i++ ))
do
PORT=$((START_PORT + i))
DEVICE=$((START_DEVICE + i))
LOG_FILE="$LOG_DIR/node_$i.log"
xllm \
--model $MODEL_PATH \
--devices="cuda:$DEVICE" \
--port $PORT \
--nnodes=$NNODES \
--master_node_addr=$MASTER_NODE_ADDR \
--block_size=128 \
--max_memory_utilization=0.86 \
--enable_prefix_cache=false \
--enable_chunked_prefill=false \
--enable_schedule_overlap=true \
--node_rank=$i \ > $LOG_FILE 2>&1 &
done
```

单卡部署时 `<device-id>` 通常从 `0` 开始。多 worker 部署中,需要让设备编号、`--node_rank`、`--nnodes` 和服务端口保持一致。

## 注意事项

- 当前文档在 [快速开始](/zh/getting_started/quick_start/) 中列出了沐曦 MACA 开发镜像。
- 沐曦MACA 容器启动需要挂载 `/dev/mxcd`、`/dev/dri`、`/dev/infiniband` 等设备;上面的命令已包含这些挂载。
- 在MetaX MACA容器中编译XLLM命令: python setup.py build --device maca
1 change: 1 addition & 0 deletions src/content/docs/zh/hardware/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ xLLM 支持多种加速器后端,用于大模型推理部署。本章节汇总
- [昇腾 NPU](/zh/hardware/ascend_npu/) - 昇腾 NPU 环境、运行时变量和 HCCL 启动注意事项。
- [寒武纪 MLU](/zh/hardware/cambricon_mlu/) - MLU 后端环境和启动入口。
- [海光 DCU](/zh/hardware/dcu/) - 海光 DCU 后端环境和启动入口。
- [沐曦 MACA](/zh/hardware/metax_maca/) - 沐曦 MACA 后端环境和启动入口。

## 通用流程

Expand Down
Loading