Skip to content

README.md

Latest commit

 

History

History
116 lines (84 loc) · 4.84 KB

README.md

File metadata and controls

116 lines (84 loc) · 4.84 KB
Raw

Coqui TTS Voice Cloner

This project provides a voice cloning Python application (XTTS-v2) that compiles to an executable. It is designed to be generic, allowing for different "agents" (voices) to be used for text-to-speech generation.

It automatically uses the GPU when a healthy CUDA device with enough free VRAM is present (quick VRAM + benchmark probe at startup) and falls back to CPU otherwise. Per-agent conditioning latents are cached to disk (latent_cache/*.pt) so startup is instant and no voice re-conditioning happens per request.

Configuration

config.json drives the build: the executable name and the version stamped into the exe (voice_server.spec generates file_version.txt from it).

{
  "executable_name": "valorantNarrator-agentVoices.exe",
  "version": "1.5.0.0",
  "company_name": "ValorantNarrator",
  "product_name": "ValorantNarrator",
  "file_description": "ValorantNarrator Agent Voices",
  "copyright": "Copyright (C) 2025"
}

Usage

  1. Agents: Place your reference audio files (mp3) in the agents folder. The filename (without extension) will be the agent's name.
  2. Running: Run the generated executable or the voice_server.py script, server runs at localhost:5005
  3. API: The server exposes POST /speak and GET /health.
    • Body (only agent and text are required): 
      {
  "agent": "agent_name",
  "text": "Text to speak",
  "language": "en",
  "speed": 1.0,
  "temperature": 0.65,
  "top_p": 0.80,
  "top_k": 50,
  "repetition_penalty": 10.0,
  "length_penalty": 1.0
}
    • Returns audio/mpeg (mp3). GET /health reports device, cached agents, and settings.

Latents, agents, and adding a voice

  • Reference audio (agents/<name>.mp3) is cleaned with ffmpeg (highpass/lowpass, silence trim, loudness) into short clips, then conditioning latents are computed and cached to latent_cache/<name>.<key>.pt.
  • The cache is bundled into the exe (voice_server.spec), so a shipped build starts instantly with all voices ready.
  • Querying an agent with no mp3 and no cached .pt returns HTTP 404. If the mp3 is present but the .pt isn't, the latent is computed on first use and cached.
  • Add/update an agent: drop agents/<name>.mp3, run python precompute_latents.py (see below) to generate the .pt, bump config.json version, rebuild. If you ship the agents/ folder alongside the exe, a new agent can also be added by shipping just its mp3 (the server computes the latent on first request).

Precompute latents before building

python precompute_latents.py            # compute any missing latents
python precompute_latents.py --force    # recompute all
python precompute_latents.py --list     # show agents + cache status

Environment variables (optional)

Variable Default Effect
XTTS_WARMUP 1 Run a warmup inference at startup so the first /speak is fast. 0 for faster startup, slower first call.
XTTS_PRELOAD_CACHED_VOICES 1 Load all cached .pt latents into memory at startup.
XTTS_PRELOAD_VOICES 0 Compute latents for every agent at startup (only needed with no cache).
XTTS_PREPARE_REFERENCES 1 Clean/segment reference audio before conditioning.
XTTS_TEMPERATURE, XTTS_TOP_P, XTTS_TOP_K 0.65 / 0.80 / 50 Generation sampling defaults.
XTTS_MAX_TEXT_CHARS 900 Reject overly long /speak text.

Performance Notes

  • GPU/CPU auto: uses CUDA when a healthy GPU with enough free VRAM is available, else CPU. Short callouts run ~400–600 ms warm on a laptop RTX 3050; CPU is several seconds per call.
  • Size: the CPU build (requirements.txt installs CPU torch) is small (~357 MB). A GPU build needs a CUDA torch env (pip install torch==2.5.1 ... --index-url https://download.pytorch.org/whl/cu121); it is much larger (~2.7 GB, exceeds GitHub's 2 GB release-asset limit — distribute via Hugging Face).

Setup and Building

Quick Start (Recommended)

Simply run the automated build script, which will create the virtual environment, install dependencies, and build the executable:

run_cmd.bat

Manual Setup (Optional)

If you prefer to set up manually:

  1. Create Virtual Environment: Create a Python 3.11 virtual environment named env311:

    python -m venv env311

  2. Activate Environment:

    env311\Scripts\activate.bat

  3. Install Dependencies:

    pip install -r requirements.txt

  4. Build the Executable:

    python -O -m PyInstaller voice_server.spec

This command uses PyInstaller with the voice_server.spec file, which reads configuration from config.json.

License

This project is licensed under the MIT License - see the LICENSE file for details.