A unified API gateway that provides a single API key interface for multiple LLM providers (OpenAI, Anthropic, Google, Mistral, Cohere) with built-in usage tracking, billing, and a modern web dashboard.
UniLLM solves the complexity of managing multiple LLM API keys by providing:
- 🔑 Single API Key: Use one key to access all supported providers
- 🤖 Automatic Provider Selection: Smart routing based on model names
- 📊 Usage Tracking: Built-in analytics and cost monitoring
- 🎨 Modern Dashboard: React-based web interface for management
- 💳 Credit System: Pre-paid credits with automatic deduction
- 🔄 Drop-in Replacements: Compatible with OpenAI and Anthropic libraries
- ⚡ Retry Logic: Automatic handling of server overloads
Get started in minutes with our managed UniLLM service. No setup required - just sign up and start using the unified API.
pip install unifyllm-sdk- Sign up at unillm.com (or your deployed frontend URL)
- Get your API key from the dashboard
- Set your API key:
export UNILLM_API_KEY="your-api-key-here"
from unillm import UniLLM
# Just provide your API key - everything else is pre-configured!
client = UniLLM(api_key="your-api-key")
# Same code works with any provider!
response = client.chat(
model="gpt-4", # OpenAI
messages=[{"role": "user", "content": "Hello! What's 2+2?"}]
)
print(response.content) # "2+2 equals 4."
# Switch to Anthropic with the same code
response = client.chat(
model="claude-3-sonnet", # Anthropic
messages=[{"role": "user", "content": "Hello! What's 2+2?"}]
)
print(response.content) # "2+2 equals 4."Access your modern React dashboard for:
- User Management: Register, login, manage API keys
- Usage Analytics: Real-time usage statistics and cost tracking
- Billing: Credit purchase and management with Stripe
- Chat Interface: Test models directly in the browser
- API Keys: Secure key management with copy/hide functionality
- Pay-as-you-go: Purchase credits and use them across all models
- No monthly fees: Only pay for what you use
- Transparent pricing: See exact costs per model
- Bulk discounts: Save more with larger credit packages
Starting at $5 for thousands of GPT-3.5 calls
from unillm import openai
# Configure
openai.api_key = "your-unillm-api-key"
# No need to set api_base - it's pre-configured for SaaS!
# Use exactly like OpenAI
response = openai.ChatCompletion.create(
model="gpt-4",
messages=[{"role": "user", "content": "Hello!"}],
max_tokens=50
)
print(response.choices[0].message.content)from unillm import anthropic
# Configure
anthropic.api_key = "your-unillm-api-key"
# No need to set api_base - it's pre-configured for SaaS!
# Use exactly like Anthropic
response = anthropic.messages.create(
model="claude-3-sonnet",
messages=[{"role": "user", "content": "Hello!"}],
max_tokens=50
)
print(response.content[0].text)UniLLM automatically routes to the correct provider based on the model name:
gpt-4- Most capable modelgpt-4-turbo- Latest GPT-4 with improved performancegpt-4o- Latest OpenAI modelgpt-4o-mini- Faster, more efficient modelgpt-3.5-turbo- Fast and cost-effective
claude-3-opus- Most capable Claude modelclaude-3-sonnet- Balanced performance and speedclaude-3-haiku- Fastest and most cost-effectiveclaude-3-5-sonnet-20241022- Latest Sonnet modelclaude-3-5-haiku-20241022- Latest Haiku model
gemini-pro- Google's most capable modelgemini-pro-vision- Vision-enabled modelgemini-1.5-pro- Latest Gemini Progemini-1.5-flash- Fast Gemini model
mistral-large- Most capable Mistral modelmistral-medium- Balanced performancemistral-small- Fast and efficient
command- Cohere's flagship modelcommand-light- Faster, lighter model
export UNILLM_API_KEY="your-api-key"
# No need to set UNILLM_BASE_URL for SaaS - it's pre-configured!# For SaaS users - just provide your API key
client = UniLLM(api_key="your-api-key")
# The library automatically connects to our managed service# Check if the API gateway is healthy
is_healthy = client.health_check()
print(f"API Gateway healthy: {is_healthy}")# Switch between providers seamlessly
providers = [
("gpt-4", "OpenAI"),
("claude-3-sonnet", "Anthropic"),
("gemini-pro", "Google"),
("mistral-large", "Mistral")
]
for model, provider in providers:
response = client.chat(
model=model,
messages=[{"role": "user", "content": "Say hello from your provider!"}],
max_tokens=30
)
print(f"{provider}: {response.content}")response = client.chat(
model="gpt-4",
messages=[{"role": "user", "content": "Generate a creative story."}],
temperature=0.8, # Creativity (0.0-2.0)
max_tokens=200, # Maximum response length
top_p=0.9, # Nucleus sampling
frequency_penalty=0.1, # Reduce repetition
presence_penalty=0.1 # Encourage new topics
)Deploy UniLLM on your own infrastructure for full privacy, compliance, and customization. Perfect for enterprises and teams with special requirements.
- 🔒 Privacy: Your data never leaves your infrastructure
- 💰 Cost Control: No markup on API calls, pay only what providers charge
- ⚡ Performance: Lower latency, no shared infrastructure
- 🔧 Customization: Modify the code to fit your specific needs
- 📊 Full Analytics: Complete control over usage tracking and billing
# Clone and setup
git clone https://github.com/yourusername/unillm
cd unillm
./quick_start.sh
# Or manually:
cd api_gateway
pip install -r requirements.txt
cp env_example.txt .env
# Edit .env with your API keys
python main_phase2.py# Create .env file with your API keys
cp env_example.txt .env
# Edit .env
# Deploy with Docker Compose
docker-compose up -dComplete guide for deploying on your own server with nginx, SSL, and PostgreSQL.
# Verify your self-hosted instance
python test_self_hosted.py
# Or test a specific URL
python test_self_hosted.py https://your-domain.comWhen self-hosting, you need to specify your own server URL:
export UNILLM_API_KEY="your-api-key"
export UNILLM_BASE_URL="https://your-self-hosted-domain.com" # Your server URLclient = UniLLM(
api_key="your-api-key",
base_url="https://your-self-hosted-domain.com" # Your server URL
)from unillm import openai
openai.api_key = "your-api-key"
openai.api_base = "https://your-self-hosted-domain.com" # Your server URL📖 Complete self-hosting guide: SELF_HOSTING_GUIDE.md
from unillm import UniLLM
client = UniLLM(api_key="your-api-key")
# Same code, different models - UniLLM handles the rest!
messages = [{"role": "user", "content": "Write a short poem about coding"}]
# Using OpenAI's GPT-4
response_openai = client.chat(
model="gpt-4",
messages=messages,
max_tokens=100
)
print("OpenAI GPT-4:")
print(response_openai.content)
print()
# Same code, now using Anthropic's Claude
response_anthropic = client.chat(
model="claude-3-sonnet",
messages=messages,
max_tokens=100
)
print("Anthropic Claude:")
print(response_anthropic.content)
print()
# Same code, now using Google's Gemini
response_gemini = client.chat(
model="gemini-pro",
messages=messages,
max_tokens=100
)
print("Google Gemini:")
print(response_gemini.content)from unillm import UniLLM
client = UniLLM(api_key="your-api-key")
def chat_with_provider(provider, message):
models = {
"OpenAI": "gpt-4",
"Anthropic": "claude-3-sonnet",
"Google": "gemini-pro",
"Mistral": "mistral-large"
}
response = client.chat(
model=models[provider],
messages=[{"role": "user", "content": message}],
max_tokens=100
)
return response.content
# Test all providers
question = "What is the capital of France?"
for provider in ["OpenAI", "Anthropic", "Google", "Mistral"]:
answer = chat_with_provider(provider, question)
print(f"{provider}: {answer}")class ConversationBot:
def __init__(self, api_key):
self.client = UniLLM(api_key=api_key)
self.conversation = []
def add_message(self, role, content):
self.conversation.append({"role": role, "content": content})
def get_response(self, user_message, model="gpt-4"):
self.add_message("user", user_message)
response = self.client.chat(
model=model,
messages=self.conversation,
max_tokens=150
)
self.add_message("assistant", response.content)
return response.content
# Usage
bot = ConversationBot("your-api-key")
bot.add_message("system", "You are a helpful assistant.")
print(bot.get_response("My name is John."))
print(bot.get_response("What's my name?")) # Remembers the conversationfrom unillm import UniLLM, UniLLMError
client = UniLLM(api_key="your-api-key")
try:
response = client.chat(
model="invalid-model",
messages=[{"role": "user", "content": "Hello"}]
)
except UniLLMError as e:
print(f"Error: {e}")
# Try with a valid model
response = client.chat(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "Hello"}]
)
print(f"Success: {response.content}")Test your setup with our comprehensive test suite:
# Set your API key
export UNILLM_API_KEY="your-api-key"
# Run comprehensive tests
python test_comprehensive.pyThis will test:
- ✅ Basic chat functionality
- ✅ Multi-provider support
- ✅ Drop-in replacements
- ✅ Error handling
- ✅ Retry logic
- ✅ Environment variables
- ✅ Advanced features
The client automatically tracks usage through your API gateway:
- Token usage (prompt, completion, total)
- Cost tracking per model
- Request history
- Analytics dashboard
-
"ModuleNotFoundError: No module named 'fastapi'"
- Install backend dependencies:
pip install -r requirements.txt
- Install backend dependencies:
-
"Invalid API key"
- Check your API key is correct
- Ensure you have credits in your account
-
"Model not available"
- Check the supported models list
- Ensure the model name is correct
-
"Server overloaded"
- The retry logic will handle this automatically
- Wait a few moments and try again
- Issues: GitHub Issues
- Documentation: GitHub Wiki
- Discussions: GitHub Discussions
- Fork the repository
- Create a feature branch:
git checkout -b feature/amazing-feature - Make your changes
- Add tests:
python test_comprehensive.py - Commit your changes:
git commit -m 'Add amazing feature' - Push to the branch:
git push origin feature/amazing-feature - Submit a pull request
MIT License - see LICENSE file for details.
Ready to get started? Choose your option above and try the quick start examples!