Use built-in wespeaker model for batch diarization embeddings#8082
Merged
Conversation
Batch /v2/transcribe was making external HTTP calls to the diarizer service for every audio segment (~18 req/sec at peak). The streaming path already loads wespeaker-voxceleb-resnet34-LM locally but the batch path never used it. Changes: - Move embedding model singleton and WAV loader into transcribe.py (avoids circular import since stream_handler imports from transcribe) - Batch _get_embedding() now tries built-in model first, HTTP fallback - stream_handler.py imports shared helpers instead of duplicating them - Replace torchaudio.load() with wave+numpy+torch (torchaudio is a stub in the Docker image) - 9 new unit tests covering built-in priority, HTTP fallback, and gating Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add 8-bit unsigned PCM and 32-bit PCM support. Raise ValueError for unsupported widths (e.g. 24-bit) so _get_embedding_builtin returns None and falls back to HTTP instead of producing corrupted waveforms. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
1 issue found and verified against the latest diff
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="backend/parakeet/transcribe.py">
<violation number="1" location="backend/parakeet/transcribe.py:54">
P2: Built-in model load failures are not cached, causing repeated `from_pretrained` attempts per segment. This can add large latency and log noise before HTTP fallback.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Fix all with cubic | Re-trigger cubic
| if _PyannoteModel is None or _PyannoteInference is None: | ||
| logger.warning("pyannote.audio not installed, built-in embedding unavailable") | ||
| return None | ||
| model = _PyannoteModel.from_pretrained( |
There was a problem hiding this comment.
P2: Built-in model load failures are not cached, causing repeated from_pretrained attempts per segment. This can add large latency and log noise before HTTP fallback.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At backend/parakeet/transcribe.py, line 54:
<comment>Built-in model load failures are not cached, causing repeated `from_pretrained` attempts per segment. This can add large latency and log noise before HTTP fallback.</comment>
<file context>
@@ -24,6 +25,70 @@
+ if _PyannoteModel is None or _PyannoteInference is None:
+ logger.warning("pyannote.audio not installed, built-in embedding unavailable")
+ return None
+ model = _PyannoteModel.from_pretrained(
+ "pyannote/wespeaker-voxceleb-resnet34-LM", token=os.getenv("HUGGINGFACE_TOKEN")
+ )
</file context>
- test_returns_none_when_builtin_fails_and_http_fails: both paths fail - TestGetBuiltinEmbeddingModel: pyannote unavailable returns None, cached model returned without re-loading - TestEmbeddingBuiltinDuration: short audio below MIN_SEGMENT_DURATION returns None without calling model, at-duration audio proceeds Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- test_audio_at_exact_min_duration: use 0.6s (MIN_SEGMENT_DURATION) - test_audio_just_above_min_duration: use 0.7s - test_successful_load_is_cached: verify pyannote load result is stored - test_returns_cached_model_without_reload: verify cached across calls Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Collaborator
Author
CP9A — Changed-Path Coverage ChecklistPR #8082: Built-in embedding for batch diarization
L1 Evidence Summary
L1 Limitation NoteParakeet service requires L4 GPU for full ASR model loading. L1 tested: service startup (NIM mode), all embedding code paths functionally (real torch/pyannote), 19 unit tests. Full GPU integration tested at L2 (GKE dev cluster). by AI for @beastoin |
Collaborator
Author
CP9B — Level 2 Integrated Test ResultsBuilt-in model availability
Integration chain tested
Test results
Updated coverage checklist (L2 column)
L2 limitationFull GPU-accelerated ASR + diarization pipeline requires GKE L4 GPU (dev cluster). L2 tested all embedding/diarization paths with real pyannote model on CPU. ASR transcription itself is unchanged by this PR. by AI for @beastoin |
Collaborator
Author
CP8 — Test Detail Table
All 19/19 tests pass. Coverage gaps: none. by AI for @beastoin |
Collaborator
Author
Unit Test Suite — 19/19 PASSED (0.24s)Test coverage by class
by AI for @beastoin |
Collaborator
Author
DER Benchmark — Built-in wespeaker DiarizationBenchmark using LibriSpeech test-clean samples (12 speakers). Multi-speaker conversations created by concatenating samples with known speaker ground truth. DER calculated with Model: wespeaker-voxceleb-resnet34-LM (built-in, CPU) ResultsScenarios detail
Key findings
by AI for @beastoin |
Collaborator
Author
Prod /v2/transcribe DER BenchmarkBenchmark against current production parakeet (HTTP diarizer path). LibriSpeech test-clean samples concatenated into multi-speaker conversations, evaluated with ResultsAnalysisThis benchmarks the current prod HTTP diarizer — PR #8082 isn't deployed yet. Key observations:
Why DER differs from local benchmark (0% vs 9.9%)
Expected impact of this PRAfter deployment, DER should be unchanged (same wespeaker model, same cosine-distance clustering, same threshold). The improvement is:
by AI for @beastoin |
Collaborator
Author
DER Benchmark: Dev (PR #8082 built-in) vs Prod (HTTP diarizer)Dev parakeet deployed with PR #8082 image ( Head-to-head comparisonSummary
Key findings
ConclusionSafe to merge — diarization quality unchanged, eliminates external diarizer dependency for embedding extraction. by AI for @beastoin |
Collaborator
Author
|
lgtm |
This was referenced Jun 21, 2026
Collaborator
Author
Post-Deploy Monitor T+0 (09:33 UTC)Deploy: image
T+0 Log Scan (mon)Built-in embedding: NOT ACTIVE — silently falling back to HTTP diarizer. Every request logs
Monitoring continues. Next checkpoint after #8085 redeploy. by AI for @beastoin |
Collaborator
Author
Post-Deploy Monitor T+20m (09:55 UTC)
by AI for @beastoin |
Collaborator
Author
Post-Deploy Monitor T+30m (10:06 UTC)
Next checkpoint: T+1h (~10:33 UTC) by AI for @beastoin |
Collaborator
Author
Post-Deploy Monitor T+1h (10:37 UTC)
Next checkpoint: T+2h (~11:33 UTC) by AI for @beastoin |
Collaborator
Author
Post-Deploy Monitor T+2h (11:39 UTC)
Next checkpoint: T+4h (~13:33 UTC) by AI for @beastoin |
Collaborator
Author
Post-Deploy Monitor T+4h (13:33 UTC)
Next checkpoint: T+8h (~17:33 UTC) by AI for @beastoin |
Collaborator
Author
Post-Deploy Monitor T+8h (16:44 UTC)
Next checkpoint: T+12h (~21:33 UTC) by AI for @beastoin |
Collaborator
Author
Post-Deploy Monitor T+12h (20:49 UTC)
Next checkpoint: T+16h (~01:33 UTC June 22) by AI for @beastoin |
Collaborator
Author
Post-Deploy Monitor T+16h (00:53 UTC June 22)
Next checkpoint: T+20h (~05:33 UTC June 22) by AI for @beastoin |
6 tasks
beastoin
added a commit
that referenced
this pull request
Jun 22, 2026
## Summary Fixes parakeet Dockerfile so the built-in wespeaker speaker embedding model from PR #8082 actually activates in the NGC container. Without this, pyannote.audio fails to import and all embedding requests silently fall back to the external HTTP diarizer. Closes #8081 ## Problem PR #8082 code deployed cleanly but the built-in embedding was inactive. Three import chain failures in the NGC container: 1. **torch_audiomentations**: `pyannote.audio.core.task` imports it for training-time augmentation — missing from container 2. **torchaudio**: wespeaker model needs `kaldi.fbank` for mel filterbank features — the old stub didn't expose the compliance module 3. **pyannote telemetry**: imports opentelemetry OTLP exporter — not installed and unnecessary for inference ## Fix (verified on dev GKE L4 GPU) 1. **torchaudio**: Install real package \`--no-deps\`, patch \`__init__.py\` to skip C extension loader and expose \`compliance\` + \`functional\` modules. Keeps NGC torch ABI intact. 2. **torch_audiomentations**: Stub package with all symbols pyannote imports — \`Identity\`, \`BaseWaveformTransform\`, \`Mix\`, \`from_dict\`. Never called at inference time. 3. **pyannote telemetry**: Post-install stub with 5 no-op functions. 4. **pyannote.audio pinned to <4.0**: Prevents untested major version upgrades that could break stubs. ## DER Benchmark (dev v7 vs prod HTTP diarizer) | Scenario | Dev DER | Prod DER | Delta | |---|---|---|---| | 2-spk short | 10.8% | 11.1% | -0.3pp | | 2-spk long | 3.4% | 3.4% | -0.0pp | | 3-spk | 4.7% | 4.7% | +0.0pp | | 4-spk round-robin | 17.4% | 17.4% | -0.0pp | | 2-spk interleaved | 12.9% | 12.9% | -0.0pp | | **Average** | **9.8%** | **9.9%** | **-0.1pp** | ## Test evidence - 19/19 unit tests pass (test_parakeet_builtin_embedding.py) - Dev GKE L4 GPU: pyannote import OK, wespeaker model load OK, 256-dim embedding on GPU OK - DER identical to prod HTTP diarizer across all 5 scenarios ## Risk - Minimal — stubs only satisfy import-time symbols for training code paths never executed at inference - If any stub is insufficient, the existing try/except in \`get_builtin_embedding_model()\` catches the error and falls back to HTTP (no regression) - torchaudio compliance.kaldi is pure Python — no C extension ABI risk - pyannote.audio pinned to <4.0 prevents version drift that could break stubs _by AI for @beastoin_
Collaborator
Author
Post-Deploy Monitoring — CLOSED (T+18h)Deprecating this monitoring cycle per manager. PR #8085 (Dockerfile fix) has merged and will be deployed separately with its own monitoring. Summary across 18h (T+0 through T+16h):
New monitoring will start when PR #8085's Dockerfile changes are deployed to prod. by AI for @beastoin |
Git-on-my-level
pushed a commit
to Git-on-my-level/omi
that referenced
this pull request
Jun 22, 2026
pyannote.audio imports torch_audiomentations via pyannote.audio.core.task, but it was missing from the --no-deps install list. Without it, get_builtin_embedding_model() silently returns None and all embedding requests fall back to the external HTTP diarizer — defeating the built-in embedding feature from BasedHardware#8082. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Batch
/v2/transcribediarization now uses the built-in wespeaker-voxceleb-resnet34-LM speaker embedding model instead of making external HTTP calls to the diarizer service for every audio segment. Falls back to HTTP only when the built-in model is unavailable or errors. Streaming/v3/streamupdated to share the same embedding helpers, eliminating code duplication and thetorchaudiodependency.Problem
Issue #8081 — inconsistency between streaming and batch speaker embedding paths:
/v3/stream): loads wespeaker locally, computes embeddings on-GPU/v2/transcribe): sends every segment over HTTP toprod-omi-diarizer.../v2/embeddingAt peak load this produced ~18 embedding HTTP requests/sec = 1,118 httpx log lines/min (82% of all parakeet logs). The external round-trip also adds latency per segment.
Changes
transcribe.pyget_builtin_embedding_model()— thread-safe singleton that loads wespeaker-voxceleb-resnet34-LM via pyannote, with CUDA placement when availablewav_bytes_to_waveform()— parses WAV bytes to torch tensor usingwave+numpy+torch(replacestorchaudio.load()which is a stub in the Docker image). Handles 8-bit unsigned, 16-bit signed, 32-bit signed PCM; stereo downmix; raisesValueErroron unsupported sample widths_get_embedding_builtin()— runs local model inference with MIN_SEGMENT_DURATION (0.6s) gate_get_embedding()HTTP logic to_get_embedding_http()_get_embedding()— tries built-in first, falls back to HTTP if built-in unavailable or fails_diarize_segments()— proceeds with diarization when built-in model is available even withoutSPEAKER_EMBEDDING_URLstream_handler.py_get_builtin_embedding_model,_embedding_model,_embedding_lock)torchaudioimportget_builtin_embedding_modelandwav_bytes_to_waveformfromtranscribe.pyTests — 19 unit tests
TestWavBytesToWaveformTestGetEmbeddingTestGetBuiltinEmbeddingModelTestEmbeddingBuiltinDurationTestDiarizeSegmentsGatingDER Benchmark
Ran against LibriSpeech test-clean samples (12 distinct speakers). Multi-speaker conversations with known ground truth, evaluated with
pyannote.metrics.DiarizationErrorRate.Average DER: 0.0% — perfect separation, perfect re-identification, 120x realtime on CPU.
Risks & mitigations
HOSTED_SPEAKER_EMBEDDING_API_URLstill works as fallback; no Helm changes requiredCloses #8081
🤖 Generated with Claude Code