lele is a standalone, dependency-free inference engine for audio intelligence, built from scratch in pure Rust.
It rejects the "general-purpose runtime" approach (wrapping C++ libs like ORT or using heavy Torch ports) in favor of hand-crafted, domain-specific kernels.
lele is designed to run deep learning models (specifically speech-related ones like SenseVoice, Silero VAD, and TTS) with minimal overhead. It avoids heavy runtimes like ONNX Runtime or burn by compiling ONNX graphs directly into optimized Rust source code.
Why Not ORT/Burn?
- Generic Overhead: General runtimes carry massive baggage (graph optimization, dynamic shapes, thousands of unused ops) that slows down specific, small-batch audio models.
- FFI Penalties: Binding layers introduce latency and inhibit compiler inlining.
- Black Box Memory: We need absolute control over every byte of allocation for embedded/real-time constraints.
- Zero Runtime Dependencies: Generated models are pure Rust.
- AOT Compilation: Converts ONNX models to specialized Rust code for maximum performance.
- SIMD Optimized: Hand-written kernels using Apple Silicon (NEON) and x86_64 (AVX/SSE) intrinsics.
- Memory Efficient: Static buffer allocation and zero-copy weight loading.
- Speech Optimized: Built-in feature extraction for audio (FFT, Mel-spectrogram, LFR, CMVN).
- SenseVoiceSmall: High-accuracy multi-lingual ASR.
- Silero VAD: Reliable Voice Activity Detection.
- Supertonic: Fast and high-quality Text-to-Speech.
- Rust (Latest stable)
cargo
To compile an ONNX model into Rust code:
cargo run --release --bin lele_gen -- <model.onnx> <output_path.rs># SenseVoice ASR
./run_sensevoice.sh
# Supertonic TTS
./run_supertonic.sh- Performance optimizations (SIMD, multi-threading, etc.), better than ONNX Runtime.
- Support for more audio models (e.g., Whisper, CosyVoice, etc.)
- GPU acceleration backend (wgpu); Quantization (INT8/FP16)
- Advanced attention mechanisms (FlashAttention, PagedAttention)
- Voice API server (RESTful service), including ASR/TTS/Denoise endpoints.
MIT