[SGL ATOM][Feat] Enable sgl atom pcp

[ZLkanyo009]

Motivation

should be merged after #1553
Enable sgl atom pcp in deepseek v4 and glm5.2.Also can run in model based on deepseekv2 attention backend like kimi, but now it will get bad performance.

Command

GLM5.2:

export AITER_QUICK_REDUCE_QUANTIZATION=INT4
export AITER_USE_FLYDSL_MOE_SORTING=1
export SGLANG_USE_AITER=1
export SGLANG_EXTERNAL_MODEL_PACKAGE=atom.plugin.sglang.models

export CUDA_VISIBLE_DEVICES=0,1,2,3
MODEL_PATH=/models/GLM-5.2-FP8


TP=4
PORT=9000
MODEL_LOADER_EXTRA_CONFIG='{"online_quant_config":{"global_quant_config":"ptpc_fp8","layer_quant_config":{"model.layers.*.mlp.experts":"mxfp8"},"exclude_layer":["lm_head","model.embed_tokens","*.mlp.gate"]}}'

export ATOM_SGLANG_PCP_SIZE=2
export ATOM_PCP_MOE_MERGE=1

TORCHINDUCTOR_COMPILE_THREADS=128 \
python3 -m sglang.launch_server \
    --model-path "${MODEL_PATH}" \
    --host localhost \
    --port "${PORT}" \
    --trust-remote-code \
    --tp-size "${TP}" \
    --enable-nsa-prefill-context-parallel \
    --attn-cp-size $ATOM_SGLANG_PCP_SIZE \
    --nsa-prefill-cp-mode round-robin-split \
    --max-prefill-tokens 30720 \
    --chunked-prefill-size 30720 \
    --mem-fraction-static 0.85 \
    --disable-radix-cache \
    --kv-cache-dtype fp8_e4m3 \
    --model-loader-extra-config "${MODEL_LOADER_EXTRA_CONFIG}"

Deepseek v4:

export AITER_BF16_FP8_MOE_BOUND=0
export ATOM_MOE_GU_ITLV=1
export SGLANG_DEFAULT_THINKING=1
export SGLANG_DSV4_REASONING_EFFORT=max
export SGLANG_USE_AITER=1
export SGLANG_DSV4_FP4_EXPERTS=true
# Introduce ATOM as external model package of SGLang
export SGLANG_EXTERNAL_MODEL_PACKAGE=atom.plugin.sglang.models

export CUDA_VISIBLE_DEVICES=0,1,2,3
model_path=/models/DeepSeek-V4-Flash-FP8/

TP=4
ATTN_CP_SIZE=2

TORCHINDUCTOR_COMPILE_THREADS=128 \
python3 -m sglang.launch_server \
    --model-path $model_path \
    --host localhost \
    --port 9000 \
    --trust-remote-code \
    --tensor-parallel-size "${TP}" \
    --enable-nsa-prefill-context-parallel \
    --attn-cp-size $ATTN_CP_SIZE \
    --nsa-prefill-cp-mode round-robin-split \
    --kv-cache-dtype fp8_e4m3 \
    --mem-fraction-static 0.9 \
    --swa-full-tokens-ratio 0.1 \
    --max-running-requests 512 \
    --max-prefill-tokens 30720 \
    --chunked-prefill-size 30720 \
    --page-size 256 \
    --disable-shared-experts-fusion \
    --disable-radix-cache \
    --tool-call-parser deepseekv4 \
    --reasoning-parser deepseek-v4

Acc

glm:

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     3|exact_match|↑  |0.9424|±  |0.0064|
|     |       |strict-match    |     3|exact_match|↑  |0.9409|±  |0.0065|

dpsk v4:

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     3|exact_match|↑  |0.9325|±  |0.0069|
|     |       |strict-match    |     3|exact_match|↑  |0.9318|±  |0.0069|

Performance

glm:
TP4:

============ Serving Benchmark Result ============
Successful requests:                     192       
Benchmark duration (s):                  778.15    
Total input tokens:                      11796480  
Total generated tokens:                  115200    
Request throughput (req/s):              0.25      
Output token throughput (tok/s):         148.04    
Total Token throughput (tok/s):          15307.70  
---------------Time to First Token----------------
Mean TTFT (ms):                          197197.47 
Median TTFT (ms):                        223083.68 
P99 TTFT (ms):                           254167.43 
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          40.08     
Median TPOT (ms):                        38.71     
P99 TPOT (ms):                           62.80     
---------------Inter-token Latency----------------
Mean ITL (ms):                           40.08     
Median ITL (ms):                         18.88     
P99 ITL (ms):                            20.23     
----------------End-to-end Latency----------------
Mean E2EL (ms):                          221207.79 
Median E2EL (ms):                        241903.43 
P99 E2EL (ms):                           283149.80 
==================================================

PCP2 TP2:

============ Serving Benchmark Result ============
Successful requests:                     192       
Benchmark duration (s):                  717.53    
Total input tokens:                      11796480  
Total generated tokens:                  115200    
Request throughput (req/s):              0.27      
Output token throughput (tok/s):         160.55    
Total Token throughput (tok/s):          16600.89  
---------------Time to First Token----------------
Mean TTFT (ms):                          185039.34 
Median TTFT (ms):                        215047.27 
P99 TTFT (ms):                           236552.86 
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          28.58     
Median TPOT (ms):                        28.66     
P99 TPOT (ms):                           40.02     
---------------Inter-token Latency----------------
Mean ITL (ms):                           28.58     
Median ITL (ms):                         18.21     
P99 ITL (ms):                            19.14     
----------------End-to-end Latency----------------
Mean E2EL (ms):                          202158.06 
Median E2EL (ms):                        233620.00 
P99 E2EL (ms):                           260121.72 
==================================================

dpsk v4:
TP4:

============ Serving Benchmark Result ============
Successful requests:                     12        
Benchmark duration (s):                  48.57     
Total input tokens:                      737280    
Total generated tokens:                  7200      
Request throughput (req/s):              0.25      
Output token throughput (tok/s):         148.25    
Total Token throughput (tok/s):          15328.86  
---------------Time to First Token----------------
Mean TTFT (ms):                          6387.68   
Median TTFT (ms):                        6568.72   
P99 TTFT (ms):                           9771.16   
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          16.36     
Median TPOT (ms):                        16.15     
P99 TPOT (ms):                           22.46     
---------------Inter-token Latency----------------
Mean ITL (ms):                           16.36     
Median ITL (ms):                         10.94     
P99 ITL (ms):                            11.44     
----------------End-to-end Latency----------------
Mean E2EL (ms):                          16184.50  
Median E2EL (ms):                        16172.32  
P99 E2EL (ms):                           16317.23  
==================================================

PCP2 TP2:

============ Serving Benchmark Result ============
Successful requests:                     12        
Benchmark duration (s):                  41.31     
Total input tokens:                      737280    
Total generated tokens:                  7200      
Request throughput (req/s):              0.29      
Output token throughput (tok/s):         174.27    
Total Token throughput (tok/s):          18019.81  
---------------Time to First Token----------------
Mean TTFT (ms):                          4509.41   
Median TTFT (ms):                        4723.75   
P99 TTFT (ms):                           6509.25   
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          15.46     
Median TPOT (ms):                        15.14     
P99 TPOT (ms):                           18.85     
---------------Inter-token Latency----------------
Mean ITL (ms):                           15.46     
Median ITL (ms):                         12.23     
P99 ITL (ms):                            12.76     
----------------End-to-end Latency----------------
Mean E2EL (ms):                          13767.89  
Median E2EL (ms):                        13778.53  
P99 E2EL (ms):                           13802.82  
==================================================

[github-actions]

🏷️ CI Guide

Runs automatically on every eligible PR before approval:

• ✅ Pre Checkin: Black, Ruff, catalog schema validation, non-GPU unit tests

Heavy model tests:

• ✅ Run after the PR is approved and Pre Checkin passes
• ✅ Run immediately when an approval review is submitted
• ✅ Can be requested before approval with labels

Label	Tests
ci:full	Run all heavy PR model tests: native ATOM, vLLM, and SGLang
ci:atom	Run native ATOM model accuracy tests
ci:vllm	Run ATOM vLLM OOT model accuracy tests
ci:sglang	Run ATOM SGLang model accuracy tests

Heavy jobs are skipped when the PR is not approved and no matching ci:* label is present.
Add labels via the sidebar or gh pr edit 1624 --add-label <label>