feat: support multi machine for offline inference.

[weizhehuang0827]

Description

支持 Python 离线推理的多机部署。当单台机器设备数不足以加载模型时，可将模型分布在多台机器上进行推理。

核心改动：

• 新增 Shutdown RPC，允许驱动节点在推理完成后远程关闭辅助节点进程
• 新增 LLMAssistantMaster / VLMAssistantMaster，辅助节点构造 LLM 后自动进入等待循环
• Python 侧通过 nnodes、node_rank、master_node_addr 参数区分驱动/辅助角色
• 新增 require_multi_node_master_addr 和 is_offline_worker_node 工具函数进行参数校验
• 补充中英文离线推理文档中多机使用说明

Related Issues

N/A

Change Type

• New feature
• Documentation

Pull Request Checklist

PR Title and Commit Messages

• The PR title and each commit message follow the xLLM commit format: <type>: <subject>.

Pre-commit Checks

• I have installed pre-commit by running pip install pre-commit or an equivalent command.
• I have installed the hooks with pre-commit install.
• I have run pre-commit run --all-files and fixed any reported issues.

Self Review

• I have self-reviewed the code according to .agents/skills/code-review/references/custom-code-style.md, especially code written or assisted by AI.
• I have rebased this PR onto the latest main branch.

Build and Test Coverage

• Tests have been added or updated as needed.
• CUDA: python setup.py build test has passed on a CUDA machine.
• NPU: python setup.py build test has passed on an NPU machine.
• MLU: python setup.py build test has passed on an MLU machine.

Reviewer Notes

• Shutdown RPC 目前仅供多机离线推理使用，在线服务场景不会触发
• 辅助节点只需构造 LLM 对象，无需调用 generate() 或定义 prompts；node_rank>0 时初始化完成后自动进入等待，驱动节点 llm.finish() 时通过 shutdown_remote_workers() 远程关闭
• 已在 2 机 16 卡 NPU 环境验证功能正常

[gemini-code-assist]

Code Review

This pull request introduces support for multi-machine offline inference in xLLM. It adds documentation, implements a coordinated shutdown mechanism via a new Shutdown RPC to gracefully terminate remote worker processes, and introduces LLMAssistantMaster and VLMAssistantMaster to manage assistant nodes. Feedback on the changes highlights a critical resource leak and process hang in DistManager::shutdown_remote_workers(), where only the first worker of each remote machine is shut down, leaving other worker processes orphaned on those nodes. Removing the modulo check to send the shutdown signal to all remote workers is recommended to ensure clean termination.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.