Anushka — AI Voice Agent

A production-grade, real-time AI voice agent that talks to visitors in Hindi and English, qualifies leads, and emails a conversation summary — all without human intervention.

The site also includes a 3D zombie shooter mini-game built with Three.js — wave-based survival with animated GLTF models, progressive difficulty, and score tracking. Because why not.

Live demo: https://voicebot-api-4s40.onrender.com

---

Architecture

Browser (Three.js + WebRTC)
        |
        | LiveKit SDK (WebRTC)
        v
  LiveKit Cloud ──────────────────────────────────────────┐
        |                                                  |
        | Room events / audio track                        |
        v                                                  |
  Python Voice Agent (livekit-agents)                     |
    ├── STT  → Sarvam AI (en-IN)                          |
    ├── LLM  → Groq (llama-3.3-70b-versatile)            |
    └── TTS  → ElevenLabs (multilingual v2)              |
                                                          |
  .NET 9 Backend API ◄──────────────────────────────────-─┘
    ├── /token        — issues LiveKit room tokens
    ├── /dispatch     — starts agent in a room
    ├── /transcript   — receives + stores conversation
    └── /contact      — sends summary email via Gmail SMTP
                           (summary generated by Groq LLaMA)

---

Tech Stack

Layer	Technology
Frontend	Vanilla JS, Three.js (3D avatar + zombie game), LiveKit Browser SDK
Backend API	.NET 9 (ASP.NET Core), deployed on Render
Voice Agent	Python 3.11, livekit-agents framework
Realtime transport	LiveKit Cloud (WebRTC)
STT	Sarvam AI (en-IN)
LLM	Groq (llama-3.3-70b-versatile)
TTS	ElevenLabs (multilingual v2)
Email summary	Groq (llama-3.3-70b) via MailKit / Gmail SMTP
Containerisation	Docker (multi-stage builds)
Hosting	Render (free tier — web service + background worker)

---

How It Works

1. Visitor opens the site and clicks Talk to Anushka.
2. The browser calls .NET /token — receives a signed LiveKit JWT.
3. The browser joins a LiveKit room via WebRTC.
4. .NET /dispatch tells the Python agent to enter the same room.
5. The Python agent listens with Sarvam STT, reasons with Groq LLaMA, and speaks with ElevenLabs TTS — all in real time.
6. Anushka has a natural bilingual conversation and collects the visitor's name and email if they show interest.
7. On disconnect, the agent posts the full transcript to .NET /transcript.
8. The .NET API calls Groq to generate a concise summary, then emails it via Gmail SMTP.

---

Run Locally

Prerequisites

• Docker & Docker Compose or .NET 9 SDK + Python 3.11 + uv
• Accounts for: LiveKit Cloud, Sarvam AI, Groq, ElevenLabs
• A Gmail account with an App Password enabled

1. Clone and configure

git clone https://github.com/your-username/your-repo.git
cd your-repo
cp .env.example .env
# Edit .env and fill in all API keys

2a. Run with Docker Compose

docker compose up --build

API available at http://localhost:8080.

2b. Run services manually

Backend (.NET):

cd SimpleApi
dotnet run
# Listens on http://localhost:5000

Python voice agent:

cd voice-agent-py
uv sync
uv run python agent.py dev

Environment variables

See .env.example for the full documented list. Key variables:

Variable	Description
LIVEKIT_URL	Your LiveKit Cloud WSS endpoint
LIVEKIT_API_KEY	LiveKit project API key
LIVEKIT_API_SECRET	LiveKit project API secret
SARVAM_API_KEY	Sarvam AI key (STT)
ELEVENLABS_API_KEY	ElevenLabs key (TTS)
GROQ_API_KEY	Groq key (LLM + email summary generation)
Email__SenderEmail	Gmail sender address
Email__SenderPassword	Gmail App Password (not your account password)

---

Deploy to Render

The render.yaml blueprint defines both services (web API + background worker). Set all sync: false environment variables in the Render dashboard before first deploy.

---

Author

Faiz Shaikh — Software Engineer (AI/ML) faiz.corsair@gmail.com

---

License

MIT