Skip to content

Add Qwen (Alibaba) qwen-audio-3.0-tts as a fourth TTS backend#167

Merged
ballPointPenguin merged 2 commits into
mainfrom
qwen3-tts
Jul 22, 2026
Merged

Add Qwen (Alibaba) qwen-audio-3.0-tts as a fourth TTS backend#167
ballPointPenguin merged 2 commits into
mainfrom
qwen3-tts

Conversation

@ballPointPenguin

Copy link
Copy Markdown
Owner

What & why

Adds a fourth TTS backend, qwen (Alibaba Model Studio's newly released qwen-audio-3.0-tts), alongside ElevenLabs, Mistral (Voxtral), and xAI (Grok). It follows the xAI backend pattern end to end so professors can be assigned a Qwen preset voice from the voice-selection UI.

Changes

Backend

  • New artificial_u/integrations/qwen/ package:
    • QwenTTSClient — wraps the official dashscope SDK SpeechSynthesizer (the 3.0 models are WebSocket-only). Requests MP3 output so TTSService's existing chunk-concat + ffmpeg remux pipeline works unchanged. Retry/backoff mirrors the xAI client. Resolves flash vs. plus model from the chosen voice.
    • QwenVoiceManager — static 14-voice preset catalog (Alibaba exposes no voice-list API for these models).
  • QwenTTSBackend adapter, registered in integrations/tts/factory.py.
  • ALIBABA_API_KEY, TTS_QWEN_MODEL, ALIBABA_TTS_WSS_URL in settings/defaults, wired through system.py. Explicit qwen branch in the speech processor.

API + frontend

  • GET /api/v1/voices/qwen/catalog (static list, filterable by language/gender).
  • qwen enrichment branch in VoiceService.manual_voice_assignment_generic — extracted per-provider enrichment into a _enrich_generic_voice helper to stay under the flake8 complexity limit. Preview and assign already work generically.
  • New Qwen (Alibaba) tab in ProfessorVoice.tsx with a QwenVoiceCard (name, description, gender, flash/plus tier badge), plus types.ts/config.ts/voice-service.ts and i18n keys in en/es/fr/zh.

Tests / CI / docs

  • New tests/unit/integrations/test_qwen_tts.py (13 tests), api-key assertions in test_api_keys.py + conftest.py, ALIBABA_API_KEY added to both CI workflows, speech-processor test, scripts/seed_qwen_voices.py, and docs/configuration.md.

Verification

  • All 344 unit tests pass; backend black/isort/flake8 and frontend eslint/biome/stylelint/build all clean.
  • Catalog endpoint returns correct data with language/gender filters.
  • Live smoke test confirms the integration is correct up to the auth boundary (well-formed SDK call, model, voice, MP3 request).

⚠️ Reviewer note — region availability

qwen-audio-3.0-tts is hosted only in the Singapore and Beijing regions, not us-east-1 (Virginia). Verified by probing with a real key:

Region HTTP text-gen WebSocket TTS
us-east-1 (dashscope-us) 200 (key valid) 404 — no websocket TTS route
Singapore (dashscope-intl) 401 401 InvalidApiKey — route/model exist, key rejected
Beijing (dashscope) 401 InvalidApiKey

Alibaba API keys are region-locked, so a Singapore- (or Beijing-) region ALIBABA_API_KEY is required to generate audio. The default ALIBABA_TTS_WSS_URL is set to Singapore (dashscope-intl) accordingly — a us-east-1 default provably cannot serve this model.

Deferred

  • Voice cloning ("clone-to-qwen") and Voice Design — the international qwen-voice-enrollment cloning API currently targets only the qwen3-tts-vc family, not 3.0, and Voice Design has no international API docs yet. A clone-to-qwen flow can later mirror the existing clone-to-mistral feature.

🤖 Generated with Claude Code

ballPointPenguin and others added 2 commits July 22, 2026 00:00
Adds "qwen" alongside elevenlabs, mistral, and xai, following the xAI
backend pattern end to end.

Backend:
- New artificial_u/integrations/qwen/ package: QwenTTSClient (wraps the
  dashscope SDK SpeechSynthesizer, WebSocket-only, requests MP3 so the
  existing chunk-concat + ffmpeg remux pipeline is unchanged) and a static
  QwenVoiceManager preset catalog (Alibaba exposes no voice-list API).
- QwenTTSBackend adapter, registered in the TTS factory.
- ALIBABA_API_KEY / TTS_QWEN_MODEL / ALIBABA_TTS_WSS_URL settings + defaults,
  wired through system.py; explicit qwen branch in the speech processor.

API + frontend:
- GET /api/v1/voices/qwen/catalog; qwen enrichment branch in VoiceService
  (extracted per-provider enrichment into a helper to stay under the flake8
  complexity limit). Preview/assign already work generically.
- New "Qwen (Alibaba)" tab in ProfessorVoice.tsx with a QwenVoiceCard, plus
  types/config/service and i18n keys in en/es/fr/zh.

Tests/CI/docs:
- New tests/unit/integrations/test_qwen_tts.py (13 tests), api-key
  assertions, both CI workflows, speech-processor test, seed script, and
  configuration.md.

Region note: qwen-audio-3.0-tts is hosted only in the Singapore and Beijing
regions (verified: us-east-1 404s the websocket route). Keys are
region-locked, so the default endpoint is Singapore
(dashscope-intl); a Singapore-region ALIBABA_API_KEY is required to
generate audio. Voice cloning / Voice Design are deferred pending 3.0-family
international API docs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Refreshed from the Alibaba Qwen-Audio-TTS voice list (verified live against
the API with a Singapore-region key):

- Fixed Longan Lufeng gender to Male.
- Added 11 Mandarin-only base (cloned) voices, which use the model-prefixed
  voice parameter (e.g. qwen-audio-3.0-tts-plus-longyinghaikai); useful for
  Chinese-language content. Long Ying Hai Kai set to Female per the source.
- Omitted child voices (age < 18): Longjie Lidou, Long Paopao, Long Huohuo.
- Kept the existing English/bilingual system voices (all confirmed working).

Updated unit tests for the new catalog shape (base voices, child exclusion).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ballPointPenguin
ballPointPenguin merged commit 1c4e2ea into main Jul 22, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant