Add comprehensive TTS/ASR speech functionality#3
Conversation
Created from scratch by AnshumanAI: - Complete Speech API with TTS and ASR support - OpenAI Whisper/TTS integration (fully functional) - ElevenLabs TTS integration (fully functional) - Azure Speech and Google Speech placeholders - Robust error handling and configuration management - Environment variable based API key management - Type-safe speech provider system - Production-ready audio processing pipeline Key features: ✅ Text-to-Speech (TTS) with multiple providers ✅ Automatic Speech Recognition (ASR) ✅ Multi-format audio support (MP3, WAV, etc.) ✅ Comprehensive error handling ✅ Environment-based configuration ✅ Type-safe provider switching ✅ Clean separation of concerns
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Bugbot free trial expires on August 10, 2025
Learn more in the Cursor dashboard.
| "Authorization" -> s"Bearer ${config.apiKey}", | ||
| "Content-Type" -> "application/json" | ||
| ) | ||
| ) |
There was a problem hiding this comment.
Bug: Audio Transcription Fails Due to Incorrect Request Format
The OpenAISpeechClient.transcribe method incorrectly formats requests for the OpenAI Whisper API. It sends base64-encoded audio data in a JSON request body with Content-Type: application/json. However, the OpenAI Whisper API expects multipart/form-data with the audio file uploaded directly. This causes all ASR transcription requests to fail with 400 Bad Request errors.
| case SpeechProvider.Google => new GoogleSpeechClient(config.asInstanceOf[GoogleSpeechConfig]) | ||
| case _ => throw new UnsupportedOperationException(s"ASR not supported for provider: $provider") | ||
| } | ||
| } |
There was a problem hiding this comment.
Bug: Speech Client Factory Fails with Incorrect Configs
The ttsClient and asrClient factory methods have two issues: they use unsafe asInstanceOf casts that can cause a ClassCastException at runtime if an incorrect SpeechProviderConfig subclass is passed for a given SpeechProvider (e.g., ElevenLabsConfig for OpenAI), and the SpeechProvider.Amazon enum value is unhandled, leading to an UnsupportedOperationException when attempting to create a client for it.
Locations (1)
Co-authored-by: mcs23026 <mcs23026@iiitl.ac.in>
2 participants
Created from scratch by AnshumanAI:
Key features:
✅ Text-to-Speech (TTS) with multiple providers
✅ Automatic Speech Recognition (ASR)
✅ Multi-format audio support (MP3, WAV, etc.)
✅ Comprehensive error handling
✅ Environment-based configuration
✅ Type-safe provider switching
✅ Clean separation of concerns