State::full blocks for the duration of the decode (seconds to minutes on large-v3-turbo). Server-side callers using tokio / async-std currently have to spawn their own threads and bridge channels.
Goal: a first-party async wrapper that runs the FFI on a blocking task pool and yields completion.
Sketch
impl State {
/// Tokio-flavored `full()`. Runs the FFI on `spawn_blocking`
/// and returns a Future that resolves when the decode
/// completes (or aborts via the abort callback).
#[cfg(feature = "tokio")]
pub async fn full_async(
&mut self,
params: &Params,
samples: &[f32],
) -> WhisperResult<()> {
// ... spawn_blocking, share &mut self ...
}
}
Design questions
- Runtime coupling. Should this be
tokio-specific (tokio::task::spawn_blocking), runtime-agnostic via async_global_executor or blocking, or both behind feature flags?
&mut self lifetime. spawn_blocking requires 'static. Either:
- Take ownership of
State, return (State, Result<()>).
- Use a
Mutex<State> shared via Arc and lock from the blocking task.
- Use an unsafe
Send-bound helper that the wrapper enforces correctness on.
- Cancellation. The Rust future's
Drop should ideally signal Params's abort callback so the FFI returns promptly. This needs the abort callback to be installed by full_async itself, not by the caller — composition with caller-supplied abort callbacks needs design.
- Feature gate. Probably
tokio (or async) feature, with dep:tokio = { ... optional = true }. Don't block stable consumers.
Out of scope
Streaming / partial-result emission — that's a separate issue (the new-segment callback path; see TODO.md "Mid-decode callbacks").
From whispercpp/TODO.md § 3 "Larger work".
State::fullblocks for the duration of the decode (seconds to minutes onlarge-v3-turbo). Server-side callers usingtokio/async-stdcurrently have to spawn their own threads and bridge channels.Goal: a first-party async wrapper that runs the FFI on a blocking task pool and yields completion.
Sketch
Design questions
tokio-specific (tokio::task::spawn_blocking), runtime-agnostic viaasync_global_executororblocking, or both behind feature flags?&mut selflifetime.spawn_blockingrequires'static. Either:State, return(State, Result<()>).Mutex<State>shared viaArcand lock from the blocking task.Send-bound helper that the wrapper enforces correctness on.Dropshould ideally signalParams's abort callback so the FFI returns promptly. This needs the abort callback to be installed byfull_asyncitself, not by the caller — composition with caller-supplied abort callbacks needs design.tokio(orasync) feature, withdep:tokio = { ... optional = true }. Don't block stable consumers.Out of scope
Streaming / partial-result emission — that's a separate issue (the new-segment callback path; see TODO.md "Mid-decode callbacks").
From
whispercpp/TODO.md§ 3 "Larger work".