Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -248,11 +248,14 @@ Optimized inference runtimes are available under [optimized/](optimized) — pic
| [optimized/tensorRT](optimized/tensorRT) | Linux + NVIDIA GPU | CUDA/TensorRT | `curl -LsSf https://raw.githubusercontent.com/Stability-AI/stable-audio-3/main/optimized/tensorRT/bootstrap.sh \| bash` |


Beyond inference, **[optimized/mlx](optimized/mlx)** and **[optimized/tflite](optimized/tflite)** each ship a **web UI** (`./sa3-gradio`), and **optimized/mlx** also does **LoRA training** on Apple Silicon (pure-MLX, no PyTorch) — see its [LoRA training](optimized/mlx/README.md#lora-training) section, or [underfit](https://github.com/dada-bots/underfit) for a full training dashboard built on it.

## Docs

| Guide | Description |
|-------|-------------|
| [Inference Methods](docs/workflows/inference.md) | Overview of inference modes (text-to-audio, inpainting, etc.) |
| [MLX LoRA training](optimized/mlx/README.md#lora-training) | Finetune on Apple Silicon (pure-MLX); powers underfit's Mac backend |
| [LoRA Training](docs/workflows/lora.md) | Fine-tune with LoRA: setup, training loop, and checkpointing |
| [Autoencoder Workflows](docs/workflows/autoencoder.md) | Encode and decode audio with the VAE directly |
| [Prompting Guide](docs/guides/prompting.md) | Prompt and control signal reference |
Expand Down
46 changes: 46 additions & 0 deletions optimized/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Optimized runtimes

Platform-specific runtimes for **Stable Audio 3** — each a self-contained,
minimal-dependency way to run SA3 on a given accelerator. Pick by hardware:

| Runtime | Platforms | Accelerator | Inference | Web UI | Training |
|---|---|---|---|---|---|
| **[mlx](mlx)** | Apple Silicon Macs | Metal GPU (MLX) | ✅ | ✅ `./sa3-gradio` | ✅ LoRA (pure-MLX) |
| **[tflite](tflite)** | macOS / Linux / Windows, x86 & ARM | CPU (LiteRT / XNNPACK) | ✅ | ✅ `./sa3-gradio` | LoRA *inference* only (weight-patching) |
| **[tensorRT](tensorRT)** | Linux + NVIDIA GPU | CUDA / TensorRT | ✅ | — | — |

All three read the same weights from
[`stabilityai/stable-audio-3-optimized`](https://huggingface.co/stabilityai/stable-audio-3-optimized)
(auto-downloaded on first use) and cover text-to-audio, audio-to-audio,
inpainting, and CFG / negative-prompt guidance.

## One-liners

```bash
# Apple Silicon (Metal GPU)
curl -LsSf https://raw.githubusercontent.com/Stability-AI/stable-audio-3/main/optimized/mlx/bootstrap.sh | bash

# Any CPU (macOS / Linux / Windows — needs Git Bash or WSL on Windows for curl|bash)
curl -LsSf https://raw.githubusercontent.com/Stability-AI/stable-audio-3/main/optimized/tflite/bootstrap.sh | bash

# Linux + NVIDIA
curl -LsSf https://raw.githubusercontent.com/Stability-AI/stable-audio-3/main/optimized/tensorRT/bootstrap.sh | bash
```

## Highlights

- **Web UI** — the **mlx** and **tflite** runtimes each ship a gradio app
(`./sa3-gradio`) with every generation mode wired (text-to-audio, CFG /
negative prompt, audio-to-audio, inpainting), tinted mel-spectrogram
previews, model / precision hot-swap, and LoRA support.
- **LoRA training** — the **mlx** runtime has a complete pure-MLX LoRA trainer
(pre-encode → train → generate) that matches
[underfit](https://github.com/dada-bots/underfit)'s conventions and
checkpoint format, so it doubles as underfit's Apple-Silicon backend. See
[mlx → LoRA training](mlx/README.md#lora-training). For a full training
**dashboard** on a Mac, use underfit.
- **LoRA inference** — every runtime loads `.safetensors` adapters (mlx/tflite
merge or patch them into the graph; per-adapter strength + sampling-step
gating on mlx).

See each runtime's `README.md` for install, usage, flags, and benchmarks.
8 changes: 8 additions & 0 deletions optimized/mlx/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,14 @@ Apple-Silicon backend). Three steps: **pre-encode → train → generate**. See
`TRAINING_CONVENTIONS.md` for the complete convention inventory and the
torch(MPS)-vs-MLX forward+backward parity results.

> **Want a UI instead of the CLI? Use [underfit](https://github.com/dada-bots/underfit).**
> It's a full LoRA-training dashboard that drives *this* MLX trainer as its
> Apple-Silicon backend — dataset scanning/encoding, live loss + loss-by-timestep
> curves, demo MP3s with spectrograms, per-checkpoint gradio launch, and
> one-click model downloads. On a Mac it's the recommended way to train; follow
> underfit's *Apple-Silicon quickstart*. The commands below are the lower-level
> path — good for scripting, CI, or headless runs.

**1. Pre-encode** your audio to SAME latents (once, offline — torch-free):

```bash
Expand Down
Loading