feat[notask]: add RNN-T (Transducer) support to transcription-parakeet#3228
Draft
rends-east wants to merge 2 commits into
Draft
feat[notask]: add RNN-T (Transducer) support to transcription-parakeet#3228rends-east wants to merge 2 commits into
rends-east wants to merge 2 commits into
Conversation
Adds the `rnnt` model type for plain RNN-T (Transducer) ASR -- e.g. the Georgian stt_ka_fastconformer_hybrid_large_pc hybrid transducer head -- running entirely in QVAC/GGML. - converter (vendored convert-nemo-to-gguf.py + convert-nemo.sh): emit the rnnt GGUF (parakeet.rnnt.* metadata + rnnt.predict/joint tensors); `--type rnnt`. - addon: ModelType::RNNT + map the engine's "rnnt" model_type string. - JS: 'rnnt' added to the public ModelType. Requires a parakeet-cpp engine that ships the RNN-T head (companion qvac-ext-lib-whisper.cpp PR); the registry parakeet-cpp port pin should be bumped past that merge before this lands. Engine verified standalone: the C++ greedy decoder reproduces NeMo greedy RNN-T token IDs bit-for-bit (75/75 on a FLEURS ka clip; CPU==Metal at f16; Metal==NeMo at q8_0). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Review StatusCurrent Status: ❌ PENDING Pending reviews: Needs 1 Management or Team Lead, and 1 more from Management, Team Lead, or Member. |
victorchimakanu
added
the
community-contribution
Post-review polish of the transcription-parakeet rnnt support; no behaviour changes to the addon mapping itself. - Converter (vendored copy stays hunk-identical with the engine-side script): guard --type rnnt on non-transducer checkpoints, assert joint rows == vocab+1 and joint.num_classes == decoder.vocab_size, warn when NeMo's uncapped max_symbols is baked to 10, fix the wrong "label vocab + blank" comment, document the rnnt flavour in the module docstring; vendoring header now states the rnnt divergence ahead of the pinned commit (re-pin + resync once the engine PR merges). - convert-nemo.sh: the missing-.nemo hint no longer points rnnt users at download-models.sh (which has no rnnt entry and exits 2) -- it says to place the HF checkpoint manually. - Docs: package README (binding intro, convert-nemo.sh usage line, source-repos table, auto-detect list, examples, Model Variants row), index.js / parakeet.js JSDoc enumerations, ParakeetModel.cpp and ParakeetTypes.hpp comments. - CHANGELOG: [Unreleased] entry for the rnnt model type. - rtf-benchmark: accept 'rnnt' in VALID_MODEL_TYPES. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
rends-east
force-pushed
the
pr/transcription-parakeet-rnnt
branch
from
38480dd to
b1ed9f8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
transcription-parakeetsupports CTC / TDT / EOU / Sortformer GGUFs, but not plain RNN-T (Transducer) checkpoints — e.g. the Transducer branch of NeMo hybridEncDecHybridRNNTCTCBPEModelmodels such as the Georgianstt_ka_fastconformer_hybrid_large_pc. The vendored converter mis-detects such checkpoints astdtand crashes on the missingmodel_defaults.tdt_durations, and the addon has nornntmodel type.How does it solve it?
convert-nemo-to-gguf.py+convert-nemo.sh): detect hybrid / plain RNN-T checkpoints, emitparakeet.rnnt.*metadata +rnnt.predict.*/rnnt.joint.*tensors;--type rnntoverride with convert-time shape guards (joint rows must equalvocab+1). Mirrors the engine-side converter change hunk-for-hunk; the vendoring header records the divergence, to be re-pinned once the engine PR merges.ModelType::RNNT+ mapping of the engine's"rnnt"model_type string.'rnnt'added to the publicModelType; JSDoc/README/CHANGELOG updated; RTF benchmark acceptsrnnt.Depends on the engine PR: tetherto/qvac-ext-lib-whisper.cpp#90 (plain RNN-T head in parakeet-cpp). Draft until that merges and the registry
parakeet-cppport pin is bumped past it — no registry/overlay changes are included here, so the new model type is inert against the currently pinned engine.Engine-side verification (2026-06-09,
nvidia/stt_ka_fastconformer_hybrid_large_pc): the C++ greedy decoder reproduces NeMo greedy RNN-T token IDs 75/75 bit-for-bit on a FLEURSkaclip; CPU == Metal at f16; Metal == NeMo at q8_0; FLEURSkaWER matches NeMo within 0.31%.Developed with AI assistance (disclosed in the commit trailers); reviewed and verified by the author.
Breaking changes
None — additive: new model type; existing
ctc/tdt/eou/sortformerpaths are untouched.