bugfix: fix MTP DP concurrent inference failures.

[DongheJin]

Description

Related Issues

Change Type

• Bug fix
• New feature
• Performance improvement
• Refactor
• Documentation
• Test
• Build or CI

Pull Request Checklist

Thank you for contributing to xLLM. Before requesting review, please make sure the following items are complete.

PR Title and Commit Messages

• The PR title and each commit message follow the xLLM commit format: <type>: <subject>.

Allowed types: feat, bugfix, docs, test, refactor, chore, style, revert, perf, model, build, release.
The subject should use clear English, start with a verb, include at least 4 words, and end with ..

Pre-commit Checks

• I have installed pre-commit by running pip install pre-commit or an equivalent command.
• I have installed the hooks with pre-commit install.
• I have run pre-commit run --all-files and fixed any reported issues.

If you are unsure how to set up pre-commit, see the pre-commit documentation.

Self Review

• I have self-reviewed the code according to .agents/skills/code-review/references/custom-code-style.md, especially code written or assisted by AI.
• I have rebased this PR onto the latest main branch.

Build and Test Coverage

• Tests have been added or updated as needed.
• CUDA: python setup.py build test has passed on a CUDA machine.
• NPU: python setup.py build test has passed on an NPU machine.
• MLU: python setup.py build test has passed on an MLU machine.

Reviewer Notes

[gemini-code-assist]

Code Review

This pull request introduces raw_dp_global_token_nums to track original DP token counts before padding across various engines and padding structures. It also updates EmbeddingCache to clone detached embeddings, preventing dependencies on producer output storage, and adds serialization mutexes and stream synchronization in MTPWorkerImpl to support safe schedule overlap. Feedback suggests optimizing request ID lookups in WorkerImpl by changing last_step_request_ids_ from a std::vector to an std::unordered_set to avoid linear search overhead.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

The use of std::find on a std::vector<std::string> results in a linear search, which can be inefficient for large numbers of requests. The time complexity is O(N*M), where N is the number of current requests and M is the number of requests in the last step.

For better performance, consider changing last_step_request_ids_ from std::vector<std::string> to std::unordered_set<std::string>. This would reduce the lookup time to O(1) on average, making the check significantly faster for large batches.

This would require changing worker_impl.h and update_last_step_output. The check in this function would then become:

if (last_step_request_ids_.count(request_id) > 0) {
  return true;
}