EPIC: Agent-native timeline + character intelligence

[taskmasterpeace]

Design spec: Agent-Native Timeline + Character Intelligence

North star

Let the AI perceive and act on a project the way the user can — read the whole timeline + transcript, know exactly where every clip/generation sits (to the second), know who each character is, and place/replace/adjust clips on command. Drop in a song → beat-accurate transcript → characters surfaced automatically → tied to real references → generate the right shot into the right slot, with every generation remembering what it is, where it lives, and who is in it.

Pillars

• Log in to the desktop the same way as Director's Palette (email/password + Google), fix dp_ key auth, enable agent login #9 — Login parity (email/password + Google, fix dp_ keys, enable agent login). Prerequisite.
• Project Context API — AI reads the timeline + acts on it (with undo) #10 — Project Context API — AI reads the timeline + acts on it (with undo). The AI's eyes + hands.
• Generation metadata — every clip remembers what it is, where it sits, who's in it #11 — Generation metadata — every clip remembers what/where/who. Solves "filename isn't the prompt."
• Audio intelligence — song to vocal-separated, word-timed transcript (+ lyrics forced alignment) #12 — Audio intelligence — song → vocal-separated, word-timed transcript (+ lyrics forced alignment).
• Character identity + entity extraction + Review Cast panel #13 — Character identity + entity extraction + Review Cast — name ↔ tag ↔ aliases, confirm-and-link UI.

Dependency order

#9 (login) ──► #10 (context API: read+act)
#11 (metadata) underpins #10 and #13
#12 (audio → transcript) ──► #13 (extract + Review Cast)

Build order is not scheduled here — it's the user's call (the spec is documentation only, no build).

Key research-backed decisions

• Transcription: Demucs vocal-sep → WhisperX (Replicate); lyrics+audio = forced alignment (one small custom Cog is the only new infra).
• Review UX: the "Review Cast" side panel (machine proposes, human confirms once → all mentions re-link; aliases = find-and-replace).
• Reuse Director's Palette's stateless brains (extractEntities, lyric-analysis, name-replacement, StoryboardCharacter) — port, don't reinvent.

Note on overlap with earlier handoff issues

Some earlier issues are subsumed/refined by these: #5 (metadata) → folded into #11; #8 (audio refs) relates to #12; #7 (context-aware) and #3 (universal placeholders) plug into #10. Consolidate when scheduling.