Audio intelligence — song to vocal-separated, word-timed transcript (+ lyrics forced alignment)

[taskmasterpeace]

✅ Decided — locked, do not re-litigate

• Transcription pipeline: Demucs (separate the vocal — don't transcribe the beat) → WhisperX (victor-upmeet/whisperx, true word-level timing). Lyrics + audio = forced alignment (the words come from the real lyrics, the timing from aligning them to the audio). The only new infrastructure is one small custom Replicate Cog for provided-lyrics alignment (wrapping whisperx.align() / ctc-forced-aligner) — ship audio-only on WhisperX first, lyrics-aligned as a fast-follow.
• Reuse, don't reinvent: Director's Palette already ships lyric-analysis.service (Whisper word timing → verse/chorus structure → themes) as a stateless, portable service — mirror it into the desktop, don't rebuild.

---

Part of the design spec (Pillar 4). Give the app a song; it returns a beat-accurate transcript. Research-backed (Replicate-first).

Feature goals

• Ingest audio → word-level timed transcript for timeline placement.
• Vocal separation pre-pass (Demucs cjwbw/demucs, htdemucs_ft) so the beat isn't transcribed.
• Two modes: (a) audio only → Demucs → WhisperX (victor-upmeet/whisperx) word timing; (b) audio + provided lyrics → Demucs → forced alignment (words = real lyrics, timing = aligned). The app knows which mode it's in and labels the source (lyrics-provided = high confidence).
• Song structure (verse/chorus/sections) — reuse DP's lyric-analysis.service.
• Editable transcript (fix wrong words) + persisted with the project/asset (today it's lost on refresh).

Quality goals

• Reliable word-level boundaries on rap/spoken vocals; lyrics-provided path materially more accurate than ASR alone; low-confidence words flagged for quick nudge (overlapping adlibs are known-hard).
• "Don't transcribe the music": instrumental sections don't emit garbage words. ~cents/song; 5–8 min songs complete without the user chunking.

Acceptance

Drop a song (no lyrics) → usable timed transcript that persists. Drop it with lyrics → words are the exact lyrics, timed, marked "high confidence," sections labeled.

Reuse / files

DP: lyric-analysis.service. Desktop: backend/handlers/transcription_handler.py (already calls incredibly-fast-whisper — add Demucs + WhisperX paths), backend/_routes/transcription.py, frontend/components/TranscriptPanel.tsx, transcript persistence in frontend/types/project.ts.