From 8789d942a7e3be3331a95e76f0d17d82c26a990a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E2=80=9CCortexelus=E2=80=9D?= <“emperorcj@gmail.com”> Date: Wed, 15 Jul 2026 20:23:06 -0400 Subject: [PATCH] docs: MLX training/gradio + tflite gradio across the optimized READMEs - optimized/README.md (new): index of the mlx / tflite / tensorRT runtimes with an inference / web-UI / training capability matrix + install one-liners. - optimized/mlx/README.md: call out underfit as the recommended (dashboard) way to train on a Mac; the CLI stays the lower-level path. - README.md: note the mlx/tflite web UIs + MLX LoRA training after the hardware table, plus a Docs row for MLX training. --- README.md | 3 +++ optimized/README.md | 46 +++++++++++++++++++++++++++++++++++++++++ optimized/mlx/README.md | 8 +++++++ 3 files changed, 57 insertions(+) create mode 100644 optimized/README.md diff --git a/README.md b/README.md index 668abde..5e4bd02 100644 --- a/README.md +++ b/README.md @@ -248,11 +248,14 @@ Optimized inference runtimes are available under [optimized/](optimized) — pic | [optimized/tensorRT](optimized/tensorRT) | Linux + NVIDIA GPU | CUDA/TensorRT | `curl -LsSf https://raw.githubusercontent.com/Stability-AI/stable-audio-3/main/optimized/tensorRT/bootstrap.sh \| bash` | +Beyond inference, **[optimized/mlx](optimized/mlx)** and **[optimized/tflite](optimized/tflite)** each ship a **web UI** (`./sa3-gradio`), and **optimized/mlx** also does **LoRA training** on Apple Silicon (pure-MLX, no PyTorch) — see its [LoRA training](optimized/mlx/README.md#lora-training) section, or [underfit](https://github.com/dada-bots/underfit) for a full training dashboard built on it. + ## Docs | Guide | Description | |-------|-------------| | [Inference Methods](docs/workflows/inference.md) | Overview of inference modes (text-to-audio, inpainting, etc.) | +| [MLX LoRA training](optimized/mlx/README.md#lora-training) | Finetune on Apple Silicon (pure-MLX); powers underfit's Mac backend | | [LoRA Training](docs/workflows/lora.md) | Fine-tune with LoRA: setup, training loop, and checkpointing | | [Autoencoder Workflows](docs/workflows/autoencoder.md) | Encode and decode audio with the VAE directly | | [Prompting Guide](docs/guides/prompting.md) | Prompt and control signal reference | diff --git a/optimized/README.md b/optimized/README.md new file mode 100644 index 0000000..78aa2dd --- /dev/null +++ b/optimized/README.md @@ -0,0 +1,46 @@ +# Optimized runtimes + +Platform-specific runtimes for **Stable Audio 3** — each a self-contained, +minimal-dependency way to run SA3 on a given accelerator. Pick by hardware: + +| Runtime | Platforms | Accelerator | Inference | Web UI | Training | +|---|---|---|---|---|---| +| **[mlx](mlx)** | Apple Silicon Macs | Metal GPU (MLX) | ✅ | ✅ `./sa3-gradio` | ✅ LoRA (pure-MLX) | +| **[tflite](tflite)** | macOS / Linux / Windows, x86 & ARM | CPU (LiteRT / XNNPACK) | ✅ | ✅ `./sa3-gradio` | LoRA *inference* only (weight-patching) | +| **[tensorRT](tensorRT)** | Linux + NVIDIA GPU | CUDA / TensorRT | ✅ | — | — | + +All three read the same weights from +[`stabilityai/stable-audio-3-optimized`](https://huggingface.co/stabilityai/stable-audio-3-optimized) +(auto-downloaded on first use) and cover text-to-audio, audio-to-audio, +inpainting, and CFG / negative-prompt guidance. + +## One-liners + +```bash +# Apple Silicon (Metal GPU) +curl -LsSf https://raw.githubusercontent.com/Stability-AI/stable-audio-3/main/optimized/mlx/bootstrap.sh | bash + +# Any CPU (macOS / Linux / Windows — needs Git Bash or WSL on Windows for curl|bash) +curl -LsSf https://raw.githubusercontent.com/Stability-AI/stable-audio-3/main/optimized/tflite/bootstrap.sh | bash + +# Linux + NVIDIA +curl -LsSf https://raw.githubusercontent.com/Stability-AI/stable-audio-3/main/optimized/tensorRT/bootstrap.sh | bash +``` + +## Highlights + +- **Web UI** — the **mlx** and **tflite** runtimes each ship a gradio app + (`./sa3-gradio`) with every generation mode wired (text-to-audio, CFG / + negative prompt, audio-to-audio, inpainting), tinted mel-spectrogram + previews, model / precision hot-swap, and LoRA support. +- **LoRA training** — the **mlx** runtime has a complete pure-MLX LoRA trainer + (pre-encode → train → generate) that matches + [underfit](https://github.com/dada-bots/underfit)'s conventions and + checkpoint format, so it doubles as underfit's Apple-Silicon backend. See + [mlx → LoRA training](mlx/README.md#lora-training). For a full training + **dashboard** on a Mac, use underfit. +- **LoRA inference** — every runtime loads `.safetensors` adapters (mlx/tflite + merge or patch them into the graph; per-adapter strength + sampling-step + gating on mlx). + +See each runtime's `README.md` for install, usage, flags, and benchmarks. diff --git a/optimized/mlx/README.md b/optimized/mlx/README.md index f9c4a7a..4b00ff8 100644 --- a/optimized/mlx/README.md +++ b/optimized/mlx/README.md @@ -200,6 +200,14 @@ Apple-Silicon backend). Three steps: **pre-encode → train → generate**. See `TRAINING_CONVENTIONS.md` for the complete convention inventory and the torch(MPS)-vs-MLX forward+backward parity results. +> **Want a UI instead of the CLI? Use [underfit](https://github.com/dada-bots/underfit).** +> It's a full LoRA-training dashboard that drives *this* MLX trainer as its +> Apple-Silicon backend — dataset scanning/encoding, live loss + loss-by-timestep +> curves, demo MP3s with spectrograms, per-checkpoint gradio launch, and +> one-click model downloads. On a Mac it's the recommended way to train; follow +> underfit's *Apple-Silicon quickstart*. The commands below are the lower-level +> path — good for scripting, CI, or headless runs. + **1. Pre-encode** your audio to SAME latents (once, offline — torch-free): ```bash