Add on-device transcription, language selection, and account management#4
Add on-device transcription, language selection, and account management#4
Conversation
…support - Add language selection view and integrate into onboarding flow and settings - Add account management section (sign in/out, usage display, upgrade to Pro) - Add subscription tracking with word count limits and upgrade prompts - Enhance recording flow with LLM post-processing via transcribeAndFormat - Update keyboard extension with dictionary, shortcut, and language support - Add iOS URL opening support to shared AuthManager and SubscriptionManager - Make LanguageManager init and selectedLanguages public for cross-module use - Read app version from bundle instead of hardcoding https://claude.ai/code/session_014WnimLYNLwfqYsqZn71BSU
- Link FluidAudio SPM package to WhisperMateIOS target - Create ParakeetTranscriptionService for iOS (NVIDIA Parakeet v3 model) - Add on-device/cloud transcription toggle in iOS Settings - Integrate Parakeet into RecordingSheetView transcription flow - On-device mode bypasses cloud API and subscription limits https://claude.ai/code/session_014WnimLYNLwfqYsqZn71BSU
Greptile SummaryThis PR adds substantial new functionality: on-device transcription via the FluidAudio/Parakeet model, multi-language selection with a dedicated UI, account/subscription management in settings, and an enriched transcription pipeline (prompt hints, post-processing, word-count tracking). The changes span the iOS app, keyboard extension, and shared services layers and are generally well-structured. Two functional bugs were found that need attention before merging:
Confidence Score: 2/5
Important Files Changed
Sequence DiagramsequenceDiagram
participant User
participant RecordingSheetView
participant SubscriptionManager
participant ParakeetService
participant OpenAIClient
User->>RecordingSheetView: Stop recording
RecordingSheetView->>RecordingSheetView: transcribeAudio(audioURL)
alt useOnDeviceTranscription = true
RecordingSheetView->>ParakeetService: transcribe(audioURL)
Note over RecordingSheetView,ParakeetService: ⚠️ No subscription check!
ParakeetService-->>RecordingSheetView: result text
RecordingSheetView->>RecordingSheetView: applyReplacements + expandShortcuts
RecordingSheetView->>SubscriptionManager: recordWords(count)
else useOnDeviceTranscription = false
RecordingSheetView->>SubscriptionManager: checkCanTranscribe()
SubscriptionManager-->>RecordingSheetView: canTranscribe / reason
alt canTranscribe = false
RecordingSheetView-->>User: Show limit error
else canTranscribe = true
RecordingSheetView->>OpenAIClient: transcribe / transcribeAndFormat
OpenAIClient-->>RecordingSheetView: result text
RecordingSheetView->>RecordingSheetView: applyReplacements + expandShortcuts
RecordingSheetView->>SubscriptionManager: recordWords(count)
end
end
RecordingSheetView-->>User: Show transcription
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| func initialize() async throws { | ||
| guard case .notInitialized = state else { | ||
| DebugLog.info("Already initialized or in progress", context: "ParakeetTranscriptionService") | ||
| return | ||
| } |
There was a problem hiding this comment.
Cannot retry after initialization failure
The guard only allows re-entry when the state is .notInitialized. After a failed download or initialization, the state transitions to .error(...), which means calling initialize() again (e.g., when the user taps "Download" a second time in Settings) silently returns with the "Already initialized or in progress" log message and does nothing.
The guard should also permit entry from the .error state to allow retries:
| func initialize() async throws { | |
| guard case .notInitialized = state else { | |
| DebugLog.info("Already initialized or in progress", context: "ParakeetTranscriptionService") | |
| return | |
| } | |
| guard case .notInitialized = state, case .error(_) = state else { |
Or more clearly:
guard state == .notInitialized || {
if case .error = state { return true }
return false
}() else {
DebugLog.info("Already initialized or in progress", context: "ParakeetTranscriptionService")
return
}
// Reset to notInitialized before proceeding
await MainActor.run { self.state = .notInitialized }As written, a user who encounters a download failure has no way to retry without restarting the app.
| private func transcribeWithParakeet(audioURL: URL) { | ||
| sheetState = .processing | ||
|
|
||
| Task { | ||
| do { | ||
| let result = try await ParakeetTranscriptionService.shared.transcribe(audioURL: audioURL) | ||
|
|
||
| // Apply post-processing | ||
| var processedResult = result | ||
| processedResult = dictionaryManager.applyReplacements(to: processedResult) | ||
| processedResult = shortcutManager.expandShortcuts(in: processedResult) | ||
|
|
||
| // Track word count | ||
| let wordCount = processedResult.split(separator: " ").count | ||
| await SubscriptionManager.shared.recordWords(wordCount) | ||
|
|
||
| await MainActor.run { | ||
| transcription = processedResult | ||
| sheetState = .viewing | ||
| errorMessage = "" | ||
|
|
||
| let duration = recordingStartTime.map { Date().timeIntervalSince($0) } | ||
| let recordingID = UUID() | ||
| let permanentAudioURL = historyManager.saveAudioFile(from: audioURL, for: recordingID) | ||
|
|
||
| let recording = Recording( | ||
| id: recordingID, | ||
| transcription: processedResult, | ||
| duration: duration, | ||
| audioFileURL: permanentAudioURL | ||
| ) | ||
| historyManager.addRecording(recording) | ||
| currentRecording = recording | ||
|
|
||
| try? FileManager.default.removeItem(at: audioURL) | ||
| } | ||
| } catch { | ||
| await MainActor.run { | ||
| transcription = "" | ||
| sheetState = .viewing | ||
| errorMessage = "On-device transcription failed: \(error.localizedDescription)" | ||
| } | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
On-device path bypasses subscription limits
transcribeWithParakeet skips the subscription check that transcribeWithCloud performs before transcribing. This means users who enable on-device (Parakeet) transcription can transcribe unlimited audio regardless of their subscription tier or word limit — effectively making the subscription gate defeatable by simply toggling a switch.
The subscription check should be added at the top of transcribeWithParakeet, mirroring the guard in transcribeWithCloud:
private func transcribeWithParakeet(audioURL: URL) {
// Check subscription limits
let subscriptionCheck = SubscriptionManager.shared.checkCanTranscribe()
if !subscriptionCheck.canTranscribe {
errorMessage = subscriptionCheck.reason ?? "Transcription limit reached"
sheetState = .viewing
try? FileManager.default.removeItem(at: audioURL)
return
}
sheetState = .processing
// ...
}| func cleanup() { | ||
| asrManager?.cleanup() | ||
| asrManager = nil | ||
| models = nil | ||
| state = .notInitialized | ||
| isModelDownloaded = false | ||
| } |
There was a problem hiding this comment.
cleanup() mutates @Published properties off the main thread
cleanup() directly assigns to state and isModelDownloaded, both of which are @Published properties requiring main-thread access. This is called from arbitrary contexts and will cause a runtime warning (or crash in strict concurrency builds) because @Published property updates must happen on the main actor.
| func cleanup() { | |
| asrManager?.cleanup() | |
| asrManager = nil | |
| models = nil | |
| state = .notInitialized | |
| isModelDownloaded = false | |
| } | |
| func cleanup() { | |
| asrManager?.cleanup() | |
| asrManager = nil | |
| models = nil | |
| Task { @MainActor in | |
| self.state = .notInitialized | |
| self.isModelDownloaded = false | |
| } | |
| } |
2 participants
Summary
This PR adds comprehensive support for on-device transcription using NVIDIA's Parakeet model, multi-language selection, user account management with subscription tracking, and improves the overall transcription pipeline with better post-processing.
Key Changes
On-Device Transcription
ParakeetTranscriptionService: Implements on-device transcription using FluidAudio framework with NVIDIA Parakeet v3 multilingual modelRecordingSheetViewand keyboard extension support both transcription methodsLanguage Selection
LanguageSelectionView: Dedicated UI for selecting transcription languagesLanguageManagerenhancements:Account & Subscription Management
AuthManagerandSubscriptionManagerintegration:Enhanced Transcription Pipeline
UI/UX Improvements
Technical Details
https://claude.ai/code/session_014WnimLYNLwfqYsqZn71BSU