Skip to content

Async-friendly State::full (tokio / blocking task pool) #4

@uqio

Description

@uqio

State::full blocks for the duration of the decode (seconds to minutes on large-v3-turbo). Server-side callers using tokio / async-std currently have to spawn their own threads and bridge channels.

Goal: a first-party async wrapper that runs the FFI on a blocking task pool and yields completion.

Sketch

impl State {
    /// Tokio-flavored `full()`. Runs the FFI on `spawn_blocking`
    /// and returns a Future that resolves when the decode
    /// completes (or aborts via the abort callback).
    #[cfg(feature = "tokio")]
    pub async fn full_async(
        &mut self,
        params: &Params,
        samples: &[f32],
    ) -> WhisperResult<()> {
        // ... spawn_blocking, share &mut self ...
    }
}

Design questions

  1. Runtime coupling. Should this be tokio-specific (tokio::task::spawn_blocking), runtime-agnostic via async_global_executor or blocking, or both behind feature flags?
  2. &mut self lifetime. spawn_blocking requires 'static. Either:
    • Take ownership of State, return (State, Result<()>).
    • Use a Mutex<State> shared via Arc and lock from the blocking task.
    • Use an unsafe Send-bound helper that the wrapper enforces correctness on.
  3. Cancellation. The Rust future's Drop should ideally signal Params's abort callback so the FFI returns promptly. This needs the abort callback to be installed by full_async itself, not by the caller — composition with caller-supplied abort callbacks needs design.
  4. Feature gate. Probably tokio (or async) feature, with dep:tokio = { ... optional = true }. Don't block stable consumers.

Out of scope

Streaming / partial-result emission — that's a separate issue (the new-segment callback path; see TODO.md "Mid-decode callbacks").

From whispercpp/TODO.md § 3 "Larger work".

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions