Skip to content

Build the SA3 Song Builder: structured/pattern song generation + wired controls #89

Description

@brxs

Decided in ADR-0033 (song = arrangement of reusable parts) and ADR-0034 (backend continuation-conditioning render), merged in #88.

Problem

SA3 song generation is one-shot: one prompt → one WAV up to 6:20, authored in the Generate tab of frontend/src/media/MediaExplorer.tsx, sent as {prompt, seconds, kind}. It can't produce real structure (a chorus that returns identically), it renders the full length every time, and it isn't rerollable per section. Issue #54 (PR #87) already threaded SA3's full model surface — init_audio/audio-to-audio, inpaint, negative_prompt, cfg/apg, seed — through /api/generate, but API-only: none of it is reachable from the UI. We have powerful primitives no user can touch, and no way to compose a song from parts.

Proposed approach

Per ADR-0033/0034: a standalone Song Builder window (ADR-0032 pattern) with One-shot and Structured modes. Structured = a song base prompt, parts as prompt extensions (+ per-part SA3 controls + length), an arrangement letter string with fresh/repeat/variation slots, and shared coherence (seed/BPM/key). A backend orchestration renders distinct parts with neighbour-tail continuation conditioning, reuses clips for repeats, renders variations from a parent (audio-to-audio or inpaint), and stitches beat-aligned parts with crossfades (Python-side) into one GeneratedTrack WAV — cancellable, per-part progress. The #54 controls get wired into the UI. Detailed plan is for /berlitz-engineering:fix-issue.

Acceptance criteria

  • A standalone Song Builder window toggles open/hide per ADR-0032; its rendered output is a normal GeneratedTrack that loads to a deck via the existing save/preview/load path.
  • One-shot mode reproduces today's generation exactly (same request/argv for a plain prompt+length).
  • Structured mode: set a base prompt, define parts (prompt extension + optional SA3 controls + length), and an arrangement (ABABCD, AB A' B). Repeat reuses the same clip; fresh generates; variation (A') renders from its parent.
  • Distinct parts render with continuation conditioning on the neighbour's tail; parts stitch on bar/beat boundaries with crossfades into one WAV.
  • A song render is cancellable and reports per-part progress.
  • The serialized song config is saved under generated_songs/configs/ and referenced by relative path from the registry row (resolved via scoped_path; the rendered .wav stays the registry identity).
  • The Song Builder window lists saved configs and reloads one back into the editor for re-rolling/adapting.
  • The Thread Stable Audio 3's full generation surface (audio-to-audio, inpainting, CFG, seed) through the pipeline #54 controls (seed, cfg/apg, negative prompt, init audio) are reachable and functional from the UI.
  • just check passes; a human checklist covers the listen tests unit tests can't hear (identical repeats, coherent seams, variation similarity).

Scope & non-goals

In scope: the Song Builder window; one-shot + structured modes; the part/arrangement/variation model; the config artifact + configs/ library; the backend continuation-conditioning render + Python-side stitch; wiring the #54 controls.

Non-goals: changing the Magenta /api/render engine (SA3 only); auto-detecting structure from an existing track; a full drag-drop arrangement DAW timeline beyond the letter-string + per-part settings; bar-accurate lengths depend on issue #77 (target BPM) — if #77 isn't ready, parts fall back to seconds; deck-source→init-audio conversion beyond what #54 accepts natively.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions