Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
[English](./README.md) | [简体中文](./README.zh-CN.md)

This repository contains the Astro + Starlight documentation site for
[xLLM](https://github.com/jd-opensource/xllm), an LLM inference framework for
[xLLM](https://github.com/xLLM-AI/xllm), an LLM inference framework for
high-performance serving on domestic AI accelerators.

The site is built with Starlight and `starlight-theme-rapide`. It includes a
Expand Down Expand Up @@ -85,4 +85,4 @@ npm run preview

## Related Repository

- xLLM source code: <https://github.com/jd-opensource/xllm>
- xLLM source code: <https://github.com/xLLM-AI/xllm>
4 changes: 2 additions & 2 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

[English](./README.md) | [简体中文](./README.zh-CN.md)

本仓库是 [xLLM](https://github.com/jd-opensource/xllm) 的 Astro + Starlight
本仓库是 [xLLM](https://github.com/xLLM-AI/xllm) 的 Astro + Starlight
文档站。xLLM 是面向国产 AI 加速器的高性能大语言模型推理框架。

站点基于 Starlight 和 `starlight-theme-rapide` 构建,包含自定义顶部导航、
Expand Down Expand Up @@ -81,4 +81,4 @@ npm run preview

## 相关仓库

- xLLM 源码:<https://github.com/jd-opensource/xllm>
- xLLM 源码:<https://github.com/xLLM-AI/xllm>
10 changes: 5 additions & 5 deletions src/content/docs/en/cookbook/autoregressive_models/glm/glm_5.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "GLM-5 / GLM-5.1 / GLM-5.2"
sidebar:
order: 2
---
+ Source code: https://github.com/jd-opensource/xllm
+ Source code: https://github.com/xLLM-AI/xllm

+ Available in China: https://gitcode.com/xLLM-AI/xllm

Expand Down Expand Up @@ -57,7 +57,7 @@ sudo docker run -it --ipc=host -u 0 --privileged --name mydocker --network=host
Download the official repository and module dependencies:

```bash
git clone https://github.com/jd-opensource/xllm
git clone https://github.com/xLLM-AI/xllm.git
cd xllm
git checkout release/v0.10.0
git submodule update --init --recursive
Expand Down Expand Up @@ -339,15 +339,15 @@ msmodelslim quant \

#### PD Disaggregated Deployment

`xllm` supports PD disaggregated deployment. This must be used together with another open-source library, [xllm service](https://github.com/jd-opensource/xllm-service).
`xllm` supports PD disaggregated deployment. This must be used together with another open-source library, [xllm service](https://github.com/xLLM-AI/xllm-service).

##### xLLM Service Dependencies

First, download and install `xllm service`, similar to installing and building `xllm`:

```bash
git clone https://github.com/jd-opensource/xllm-service
cd xllm_service
git clone https://github.com/xLLM-AI/xllm-service.git
cd xllm-service
git submodule init
git submodule update
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ sidebar:
order: 2
---

- Source code: [https://github.com/jd-opensource/xllm](https://github.com/jd-opensource/xllm)
- Source code: [https://github.com/xLLM-AI/xllm](https://github.com/xLLM-AI/xllm)
- Available in China: [https://gitcode.com/xLLM-AI/xllm](https://gitcode.com/xLLM-AI/xllm)
- Kimi-K2.5 W8A8 weight download: [modelscope-Kimi-K2.5-W8A8-xLLM](https://www.modelscope.cn/models/Eco-Tech/Kimi-K2.5-W8A8-xLLM)
- Kimi-K2.6 W8A8 weight download: [modelscope-Kimi-K2.6-w8a8-xllm](https://www.modelscope.cn/models/Eco-Tech/Kimi-K2.6-w8a8-xllm)
Expand Down Expand Up @@ -55,7 +55,7 @@ sudo docker run -it --ipc=host -u 0 --privileged --name xllm_kimi_k25 --network=
Download the official repository and module dependencies:

```bash
git clone https://github.com/jd-opensource/xllm
git clone https://github.com/xLLM-AI/xllm.git
cd xllm
git checkout main
git submodule init
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ sidebar:
order: 3
---

+ Source code: https://github.com/jd-opensource/xllm
+ Source code: https://github.com/xLLM-AI/xllm

+ Available in China: https://gitcode.com/xLLM-AI/xllm

Expand Down Expand Up @@ -73,7 +73,7 @@ sudo docker run -it --ipc=host -u 0 --privileged --name xllm_minimax --network=h
Download the official repository and module dependencies:

```bash
git clone https://github.com/jd-opensource/xllm
git clone https://github.com/xLLM-AI/xllm.git
cd xllm
git checkout preview/minimax-minimal
git submodule init
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ sidebar:
order: 1
---

+ Source code: https://github.com/jd-opensource/xllm
+ Source code: https://github.com/xLLM-AI/xllm

+ Available in China: https://gitcode.com/xLLM-AI/xllm

Expand Down Expand Up @@ -57,7 +57,7 @@ docker run -it -d \
Download the official repository and module dependencies:

```bash
git clone https://github.com/jd-opensource/xllm
git clone https://github.com/xLLM-AI/xllm.git
cd xllm
pip install pre-commit
pre-commit install
Expand Down
4 changes: 2 additions & 2 deletions src/content/docs/en/cookbook/diffusion_models/flux/flux2.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ sidebar:

This section will collect Flux2 diffusion model inference recipes for xLLM.

+ Source code: https://github.com/jd-opensource/xllm
+ Source code: https://github.com/xLLM-AI/xllm

+ Available in China: https://gitcode.com/xLLM-AI/xllm

Expand Down Expand Up @@ -57,7 +57,7 @@ docker exec -it $CONTAINER bash
Download the official repository and module dependencies:

```bash
git clone https://github.com/jd-opensource/xllm
git clone https://github.com/xLLM-AI/xllm.git
cd xllm

```
Expand Down
2 changes: 1 addition & 1 deletion src/content/docs/en/features/disagg_pd.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ The overall architecture is shown below:
### Preparation
#### Install Dependencies
- **xLLM**: Refer to [Installation && Compilation](/en/getting_started/quick_start/)
- **xLLM Service**: Refer to [xLLM Service](https://github.com/jd-opensource/xllm-service)
- **xLLM Service**: Refer to [xLLM Service](https://github.com/xLLM-AI/xllm-service)

#### Obtain Environment Information
Deploying Disaggregated PD Service requires obtaining the Device IP of the machine to create communication resources. Execute the command `cat /etc/hccn.conf | grep address` on the current AI Server to get the Device IP, for example:
Expand Down
2 changes: 1 addition & 1 deletion src/content/docs/en/features/global_kvcache.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ The overall architecture is shown in the diagram below:

#### Install Dependencies
- **xLLM**: Refer to [Quick Start](/en/getting_started/quick_start/)
- **xLLM Service**: Refer to [xLLM Service](https://github.com/jd-opensource/xllm-service)
- **xLLM Service**: Refer to [xLLM Service](https://github.com/xLLM-AI/xllm-service)

### Usage Instructions

Expand Down
2 changes: 1 addition & 1 deletion src/content/docs/en/features/xllm_service_overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "xLLM Service"
sidebar:
order: 90
---
[:simple-github: xLLM Service](https://github.com/jd-opensource/xllm-service)
[:simple-github: xLLM Service](https://github.com/xLLM-AI/xllm-service)

## Project Overview

Expand Down
8 changes: 4 additions & 4 deletions src/content/docs/en/getting_started/offline_service.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ To facilitate users in quickly using xLLM for offline inference, we provide Pyth

## LLM

LLM inference example: [:simple-github: https://github.com/jd-opensource/xllm/blob/main/examples/generate.py](https://github.com/jd-opensource/xllm/blob/main/examples/generate.py)
LLM inference example: [:simple-github: https://github.com/xLLM-AI/xllm/blob/main/examples/generate.py](https://github.com/xLLM-AI/xllm/blob/main/examples/generate.py)

LLM Beam Search example: [:simple-github: https://github.com/jd-opensource/xllm/blob/main/examples/generate_beam_search.py](https://github.com/jd-opensource/xllm/blob/main/examples/generate_beam_search.py)
LLM Beam Search example: [:simple-github: https://github.com/xLLM-AI/xllm/blob/main/examples/generate_beam_search.py](https://github.com/xLLM-AI/xllm/blob/main/examples/generate_beam_search.py)

Use `BeamSearchParams` with `beam_width` greater than `1`, then call `llm.beam_search(...)`:

Expand All @@ -36,8 +36,8 @@ For LLM Beam Search, use `beam_width` as the switch. `top_logprobs` controls the

## Embedding

Generate embedding example: [:simple-github: https://github.com/jd-opensource/xllm/blob/main/examples/generate_embedding.py](https://github.com/jd-opensource/xllm/blob/main/examples/generate_embedding.py)
Generate embedding example: [:simple-github: https://github.com/xLLM-AI/xllm/blob/main/examples/generate_embedding.py](https://github.com/xLLM-AI/xllm/blob/main/examples/generate_embedding.py)

## VLM

VLM inference example: [:simple-github: https://github.com/jd-opensource/xllm/blob/main/examples/generate_vlm.py](https://github.com/jd-opensource/xllm/blob/main/examples/generate_vlm.py)
VLM inference example: [:simple-github: https://github.com/xLLM-AI/xllm/blob/main/examples/generate_vlm.py](https://github.com/xLLM-AI/xllm/blob/main/examples/generate_vlm.py)
4 changes: 2 additions & 2 deletions src/content/docs/en/getting_started/quick_start.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ docker run -it \

### NVIDIA GPU

We provide a [Dockerfile](https://github.com/jd-opensource/xllm/blob/main/docker/Dockerfile.cuda) for NVIDIA GPU usage, which can be used to build custom image. Of course, you can also use dev image we built based on the default Dockerfile:
We provide a [Dockerfile](https://github.com/xLLM-AI/xllm/blob/main/docker/Dockerfile.cuda) for NVIDIA GPU usage, which can be used to build custom image. Of course, you can also use dev image we built based on the default Dockerfile:
```bash
docker pull quay.io/jd_xllm/xllm-ai:xllm-dev-cuda-x86
```
Expand Down Expand Up @@ -148,7 +148,7 @@ If you download a release image, i.e., an image with a version number in the tag

Download xllm and dependencies:
```bash
git clone https://github.com/jd-opensource/xllm
git clone https://github.com/xLLM-AI/xllm.git
cd xllm

# Install pre-commit for the first time
Expand Down
10 changes: 5 additions & 5 deletions src/content/docs/zh/cookbook/autoregressive_models/glm/glm_5.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ sidebar:
order: 2
---

+ 源码地址:https://github.com/jd-opensource/xllm
+ 源码地址:https://github.com/xLLM-AI/xllm

+ 国内可用: https://gitcode.com/xLLM-AI/xllm

Expand Down Expand Up @@ -58,7 +58,7 @@ sudo docker run -it --ipc=host -u 0 --privileged --name mydocker --network=host
下载官方仓库与模块依赖:

```bash
git clone https://github.com/jd-opensource/xllm
git clone https://github.com/xLLM-AI/xllm.git
cd xllm
git checkout release/v0.10.0
git submodule update --init --recursive
Expand Down Expand Up @@ -330,15 +330,15 @@ msmodelslim quant \

#### PD分离部署

`xllm`支持PD分离部署,这需要与另一个开源库[xllm service](https://github.com/jd-opensource/xllm-service)配套使用。
`xllm`支持PD分离部署,这需要与另一个开源库[xllm service](https://github.com/xLLM-AI/xllm-service)配套使用。

##### xLLM Service依赖

首先,我们下载安装`xllm service`,与安装编译`xllm`类似:

```bash
git clone https://github.com/jd-opensource/xllm-service
cd xllm_service
git clone https://github.com/xLLM-AI/xllm-service.git
cd xllm-service
git submodule init
git submodule update
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ sidebar:
order: 2
---

- 源码地址:[https://github.com/jd-opensource/xllm](https://github.com/jd-opensource/xllm)
- 源码地址:[https://github.com/xLLM-AI/xllm](https://github.com/xLLM-AI/xllm)
- 国内可用: [https://gitcode.com/xLLM-AI/xllm](https://gitcode.com/xLLM-AI/xllm)
- Kimi-K2.5 W8A8权重下载: [modelscope-Kimi-K2.5-W8A8-xLLM](https://www.modelscope.cn/models/Eco-Tech/Kimi-K2.5-W8A8-xLLM)
- Kimi-K2.6 W8A8权重下载: [modelscope-Kimi-K2.6-w8a8-xllm](https://www.modelscope.cn/models/Eco-Tech/Kimi-K2.6-w8a8-xllm)
Expand Down Expand Up @@ -55,7 +55,7 @@ sudo docker run -it --ipc=host -u 0 --privileged --name xllm_kimi_k25 --network=
下载官方仓库与模块依赖:

```bash
git clone https://github.com/jd-opensource/xllm
git clone https://github.com/xLLM-AI/xllm.git
cd xllm
git checkout main
git submodule init
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ sidebar:
order: 3
---

+ 源码地址:https://github.com/jd-opensource/xllm
+ 源码地址:https://github.com/xLLM-AI/xllm

+ 国内可用: https://gitcode.com/xLLM-AI/xllm

Expand Down Expand Up @@ -73,7 +73,7 @@ sudo docker run -it --ipc=host -u 0 --privileged --name xllm_minimax --network=h
下载官方仓库与模块依赖:

```bash
git clone https://github.com/jd-opensource/xllm
git clone https://github.com/xLLM-AI/xllm.git
cd xllm
git checkout preview/minimax-minimal
git submodule init
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ sidebar:
order: 1
---

+ 源码地址:https://github.com/jd-opensource/xllm
+ 源码地址:https://github.com/xLLM-AI/xllm

+ 国内可用: https://gitcode.com/xLLM-AI/xllm

Expand Down Expand Up @@ -57,7 +57,7 @@ docker run -it -d \
下载官方仓库与模块依赖:

```bash
git clone https://github.com/jd-opensource/xllm
git clone https://github.com/xLLM-AI/xllm.git
cd xllm
pip install pre-commit
pre-commit install
Expand Down
4 changes: 2 additions & 2 deletions src/content/docs/zh/cookbook/diffusion_models/flux/flux2.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ sidebar:

本章节用于汇总 Flux2 扩散模型在 xLLM 中的推理实践。

+ 源码地址:https://github.com/jd-opensource/xllm
+ 源码地址:https://github.com/xLLM-AI/xllm

+ 国内可用: https://gitcode.com/xLLM-AI/xllm

Expand Down Expand Up @@ -58,7 +58,7 @@ docker exec -it $CONTAINER bash
下载官方仓库与模块依赖:

```bash
git clone https://github.com/jd-opensource/xllm
git clone https://github.com/xLLM-AI/xllm.git
cd xllm

```
Expand Down
2 changes: 1 addition & 1 deletion src/content/docs/zh/features/disagg_pd.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ xLLM PD分离功能主要通过以下三个模块实现:
### 使用准备
#### 安装相关依赖
- **xLLM**: 参见[安装编译](/zh/getting_started/quick_start/)
- **xLLM Service**: 参见[xLLM Service](https://github.com/jd-opensource/xllm-service)
- **xLLM Service**: 参见[xLLM Service](https://github.com/xLLM-AI/xllm-service)

### 启动PD分离服务
1. 启动etcd
Expand Down
2 changes: 1 addition & 1 deletion src/content/docs/zh/features/global_kvcache.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ xLLM 全局KV Cache功能主要通过以下三个模块实现:
### 使用准备
#### 安装相关依赖
- **xLLM**: 参见[快速开始](/zh/getting_started/quick_start/)
- **xLLM Service**: 参见[xLLM Service](https://github.com/jd-opensource/xllm-service)
- **xLLM Service**: 参见[xLLM Service](https://github.com/xLLM-AI/xllm-service)

### 使用方式
1. etcd启动配置:
Expand Down
2 changes: 1 addition & 1 deletion src/content/docs/zh/features/xllm_service_overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "xLLM Service"
sidebar:
order: 90
---
[:simple-github: xLLM Service](https://github.com/jd-opensource/xllm-service)
[:simple-github: xLLM Service](https://github.com/xLLM-AI/xllm-service)

## 简介

Expand Down
8 changes: 4 additions & 4 deletions src/content/docs/zh/getting_started/offline_service.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ sidebar:

## LLM

LLM推理示例:[:simple-github: https://github.com/jd-opensource/xllm/blob/main/examples/generate.py](https://github.com/jd-opensource/xllm/blob/main/examples/generate.py)
LLM推理示例:[:simple-github: https://github.com/xLLM-AI/xllm/blob/main/examples/generate.py](https://github.com/xLLM-AI/xllm/blob/main/examples/generate.py)

LLM Beam Search 示例:[:simple-github: https://github.com/jd-opensource/xllm/blob/main/examples/generate_beam_search.py](https://github.com/jd-opensource/xllm/blob/main/examples/generate_beam_search.py)
LLM Beam Search 示例:[:simple-github: https://github.com/xLLM-AI/xllm/blob/main/examples/generate_beam_search.py](https://github.com/xLLM-AI/xllm/blob/main/examples/generate_beam_search.py)

使用 `BeamSearchParams` 设置大于 `1` 的 `beam_width`,然后调用 `llm.beam_search(...)`:

Expand All @@ -36,9 +36,9 @@ LLM Beam Search 使用 `beam_width` 作为开启参数。`top_logprobs` 控制

## Embedding

生成Embedding示例:[:simple-github: https://github.com/jd-opensource/xllm/blob/main/examples/generate_embedding.py](https://github.com/jd-opensource/xllm/blob/main/examples/generate_embedding.py)
生成Embedding示例:[:simple-github: https://github.com/xLLM-AI/xllm/blob/main/examples/generate_embedding.py](https://github.com/xLLM-AI/xllm/blob/main/examples/generate_embedding.py)

## VLM

VLM推理示例:[:simple-github: https://github.com/jd-opensource/xllm/blob/main/examples/generate_vlm.py](https://github.com/jd-opensource/xllm/blob/main/examples/generate_vlm.py)
VLM推理示例:[:simple-github: https://github.com/xLLM-AI/xllm/blob/main/examples/generate_vlm.py](https://github.com/xLLM-AI/xllm/blob/main/examples/generate_vlm.py)

4 changes: 2 additions & 2 deletions src/content/docs/zh/getting_started/quick_start.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ docker run -it \

### NVIDIA GPU

我们提供了NVIDIA GPU使用的[Dockerfile](https://github.com/jd-opensource/xllm/blob/main/docker/Dockerfile.cuda),可以构建自定义镜像,当然也可以使用我们根据默认Dockerfile构建的开发镜像:
我们提供了NVIDIA GPU使用的[Dockerfile](https://github.com/xLLM-AI/xllm/blob/main/docker/Dockerfile.cuda),可以构建自定义镜像,当然也可以使用我们根据默认Dockerfile构建的开发镜像:
```bash
docker pull quay.io/jd_xllm/xllm-ai:xllm-dev-cuda-x86
```
Expand Down Expand Up @@ -148,7 +148,7 @@ docker run -it \

下载xllm及依赖
```bash
git clone https://github.com/jd-opensource/xllm
git clone https://github.com/xLLM-AI/xllm.git
cd xllm

# 第一次需要进行pre-commit安装
Expand Down
Loading