Skip to content

feat(audio): batch Qwen3-ASR transcriptions#5172

Open
qinxuye wants to merge 1 commit into
xorbitsai:mainfrom
qinxuye:feat/qwen3-asr-batching
Open

feat(audio): batch Qwen3-ASR transcriptions#5172
qinxuye wants to merge 1 commit into
xorbitsai:mainfrom
qinxuye:feat/qwen3-asr-batching

Conversation

@qinxuye

@qinxuye qinxuye commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Summary

  • batch concurrent Qwen3-ASR transcription requests through the model's native list inference
  • preserve per-request language and response formatting while grouping compatible inference kwargs
  • harden the shared batch worker against cancellation, batch-size overflow, and result-count mismatches

Why

Qwen3-ASR supports batched transcription, but Xinference currently invokes each concurrent audio request separately. This change lets compatible requests share one model call while preserving the existing API response behavior.

Validation

  • pytest -q xinference/model/audio/tests/test_qwen3_asr.py
  • pre-commit run --files xinference/model/audio/qwen3_asr.py xinference/model/audio/tests/test_qwen3_asr.py xinference/model/batch.py

@XprobeBot XprobeBot added this to the v2.x milestone Jul 13, 2026
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants