This guide walks you through deploying the LLM-API-Key-Proxy as a hosted service on Render.com. The project provides a universal, OpenAI-compatible API endpoint for all your LLM providers (like Gemini or OpenAI), powered by an intelligent key management library. It's perfect for integrating with platforms like JanitorAI, where you can use it as a custom proxy for highly available and resilient chats.
The process is beginner-friendly and takes about 15-30 minutes. We'll use Render's free tier (with limitations like sleep after 15 minutes of inactivity) and upload your .env file as a secret for easy key management—no manual entry of variables required.
- A free Render.com account (sign up at render.com).
- A GitHub account (for forking the repo).
- Basic terminal access (e.g., Command Prompt, Terminal, or Git Bash).
- API keys from LLM providers (e.g., Gemini, OpenAI—get them from their dashboards). For details on supported providers and how to format their keys (e.g., API key naming conventions), refer to the LiteLLM Providers Documentation.
Note: You don't need Python installed for initial testing—use the pre-compiled Windows EXE from the repo's releases for a quick local trial.
Before deploying, try the proxy locally to ensure your keys work. This uses a pre-built executable that's easy to set up.
- Go to the repo's GitHub Releases page.
- Download the latest release ZIP file (e.g., for Windows).
- Unzip the file.
- Double-click
setup_env.bat. A window will open—follow the prompts to add your PROXY_API_KEY (a strong secret you create) and provider keys. Use the LiteLLM Providers Documentation for guidance on key formats (e.g.,GEMINI_API_KEY_1="your-key"). - Double-click
proxy_app.exeto start the proxy. It runs athttp://127.0.0.1:8000—visit in a browser to confirm "API Key Proxy is running". - Test with curl (replace with your PROXY_API_KEY):
curl -X POST http://127.0.0.1:8000/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer your-proxy-key" -d '{"model": "gemini/gemini-2.5-flash", "messages": [{"role": "user", "content": "What is the capital of France?"}]}'
- Expected: A JSON response with the answer (e.g., "Paris").
If it works, you're ready to deploy. If not, double-check your keys against LiteLLM docs.
- Go to the original repo: https://github.com/Mirrowel/LLM-API-Key-Proxy.
- Click Fork in the top-right to create your own copy (this lets you make changes if needed).
- Clone your forked repo locally:
git clone https://github.com/YOUR-USERNAME/LLM-API-Key-Proxy.git
cd LLM-API-Key-Proxy
The proxy uses a .env file to store your API keys securely. We'll create this based on the repo's documentation.
- In your cloned repo, copy the example:
copy .env.example .env(Windows) orcp .env.example .env(macOS/Linux). - Open
.envin a text editor (e.g., Notepad or VS Code). - Add your keys following the format from the repo's README and LiteLLM Providers Documentation:
- PROXY_API_KEY: Create a strong, unique secret (e.g., "my-super-secret-proxy-key"). This authenticates requests to your proxy.
- Provider Keys: Add keys for your chosen providers. You can add multiple per provider (e.g., _1, _2) for rotation.
Example .env (customize with your real keys):
# Your proxy's authentication key (invent a strong one)
PROXY_API_KEY="my-super-secret-proxy-key"
# Provider API keys (get from provider dashboards; see LiteLLM docs for formats)
GEMINI_API_KEY_1="your-gemini-key-here"
GEMINI_API_KEY_2="another-gemini-key"
OPENROUTER_API_KEY_1="your-openrouter-key"
- Supported providers: Check LiteLLM docs for a full list and specifics (e.g., GEMINI, OPENROUTER, NVIDIA_NIM).
- Tip: Start with 1-2 providers to test. Don't share this file publicly!
If you are using providers that require complex OAuth files (like Gemini CLI, Qwen Code, or iFlow), you don't need to upload the JSON files manually. The proxy includes a tool to "export" these credentials into environment variables.
- Run the credential tool locally:
python -m rotator_library.credential_tool - Select the "Export ... to .env" option for your provider.
- The tool will generate a file (e.g.,
gemini_cli_user_at_gmail.env) containing variables likeGEMINI_CLI_ACCESS_TOKEN,GEMINI_CLI_REFRESH_TOKEN, etc. - Copy the contents of this file and paste them directly into your
.envfile or Render's "Environment Variables" section. - The proxy will automatically detect and use these variables—no file upload required!
The Antigravity provider requires OAuth2 authentication similar to Gemini CLI. It provides access to:
- Gemini 2.5 models (Pro/Flash)
- Gemini 3 models (Pro/Image-preview) - requires paid-tier Google Cloud project
- Claude Sonnet 4.5 via Google's Antigravity proxy
Setting up Antigravity locally:
- Run the credential tool:
python -m rotator_library.credential_tool - Select "Add OAuth Credential" and choose "Antigravity"
- Complete the OAuth flow in your browser
- The credential is saved to
oauth_creds/antigravity_oauth_1.json
Exporting for stateless deployment:
- Run:
python -m rotator_library.credential_tool - Select "Export Antigravity to .env"
- Copy the generated environment variables to your deployment platform:
ANTIGRAVITY_ACCESS_TOKEN="..." ANTIGRAVITY_REFRESH_TOKEN="..." ANTIGRAVITY_EXPIRY_DATE="..." ANTIGRAVITY_EMAIL="your-email@gmail.com"
Important Notes:
- Antigravity uses Google OAuth with additional scopes for cloud platform access
- Gemini 3 models require a paid-tier Google Cloud project (free tier will fail)
- The provider automatically handles thought signature caching for multi-turn conversations
- Tool hallucination prevention is enabled by default for Gemini 3 models
- Save the file. (We'll upload it to Render in Step 5.)
- Log in to render.com and go to your Dashboard.
- Click New > Web Service.
- Choose Build and deploy from a Git repository > Next.
- Connect your GitHub account and select your forked repo.
- In the setup form:
- Name: Something like "llm-api-key-proxy".
- Region: Choose one close to you (e.g., Oregon for US West).
- Branch: "main" (or your default).
- Runtime: Python 3.
- Build Command:
pip install -r requirements.txt. - Start Command:
uvicorn src.proxy_app.main:app --host 0.0.0.0 --port $PORT. - Instance Type: Free (for testing; upgrade later for always-on service).
- Click Create Web Service. Render will build and deploy—watch the progress in the Events tab.
Render mounts secret files securely at runtime, making your .env available to the app without exposing it.
- In your new service's Dashboard, go to Environment > Secret Files.
- Click Add Secret File.
- File Path: Don't change. Keep it as root directory of the repo.
- Contents: Upload the
.envfile you created previously. - Save. This injects the file for the app to load via
dotenv(already in the code). - Trigger a redeploy: Go to Deploy > Manual Deploy > Deploy HEAD (or push a small change to your repo).
Your keys are now loaded automatically!
- Note your service URL: It's in the Dashboard (e.g., https://llm-api-key-proxy.onrender.com).
- Test with curl (replace with your PROXY_API_KEY):
curl -X POST https://your-service.onrender.com/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer your-proxy-key" -d '{"model": "gemini/gemini-2.5-flash", "messages": [{"role": "user", "content": "What is the capital of France?"}]}'
- Expected: A JSON response with the answer (e.g., "Paris").
- Check logs in Render's Dashboard for startup messages (e.g., "RotatingClient initialized").
- Log in to janitorai.com and go to API settings (usually in a chat or account menu).
- Select "Proxy" mode.
- API URL:
https://your-service.onrender.com/v1. - API Key: Your PROXY_API_KEY (from .env).
- Model: Format as "provider/model" (e.g., "gemini/gemini-2.5-flash"; check LiteLLM docs for options).
- Save and test a chat—messages should route through your proxy.
- Build Fails: Check Render logs for missing dependencies—ensure
requirements.txtis up to date. - 401 Unauthorized: Verify your PROXY_API_KEY matches exactly (case-sensitive) and includes "Bearer " in requests. Or you have no keys for the provider/model added that you are trying to use.
- 405 on OPTIONS: If CORS issues arise, add the middleware from Step 3 and redeploy.
- Service Sleeps: Free tier sleeps after inactivity—first requests may delay.
- Provider Key Issues: Double-check formats in LiteLLM Providers Documentation.
- More Help: Check Render docs or the repo's README. If stuck, share error logs.
That is it.
Docker provides a consistent, portable deployment option for any platform. The proxy image is automatically built and published to GitHub Container Registry (GHCR) on every push to main or dev branches.
This is the fastest way to deploy the proxy using Docker.
- Create your configuration files:
# Clone the repo (or just download docker-compose.yml and .env.example)
git clone https://github.com/Mirrowel/LLM-API-Key-Proxy.git
cd LLM-API-Key-Proxy
# Create your .env file
cp .env.example .env
nano .env # Add your PROXY_API_KEY and provider keys
# Create key_usage.json file (required before first run)
touch key_usage.jsonImportant: You must create
key_usage.jsonbefore running Docker Compose. If this file doesn't exist on the host, Docker will create it as a directory instead of a file, causing the container to fail.
- Start the proxy:
docker compose up -d- Verify it's running:
# Check container status
docker compose ps
# View logs
docker compose logs -f
# Test the endpoint
curl http://localhost:8000/If you prefer not to use Docker Compose:
# Create necessary directories and files
mkdir -p oauth_creds logs
touch key_usage.json
# Run the container
docker run -d \
--name llm-api-proxy \
--restart unless-stopped \
-p 8000:8000 \
-v $(pwd)/.env:/app/.env:ro \
-v $(pwd)/oauth_creds:/app/oauth_creds \
-v $(pwd)/logs:/app/logs \
-v $(pwd)/key_usage.json:/app/key_usage.json \
-e SKIP_OAUTH_INIT_CHECK=true \
-e PYTHONUNBUFFERED=1 \
ghcr.io/mirrowel/llm-api-key-proxy:latest| Tag | Description | Use Case |
|---|---|---|
latest |
Latest stable build from main branch |
Production |
dev-latest |
Latest build from dev branch |
Testing new features |
YYYYMMDD-HHMMSS-<sha> |
Specific version with timestamp and commit hash | Pinned deployments |
Example using a specific version:
docker pull ghcr.io/mirrowel/llm-api-key-proxy:20250106-143022-abc1234| Host Path | Container Path | Purpose | Mode |
|---|---|---|---|
./.env |
/app/.env |
Configuration and API keys | Read-only (:ro) |
./oauth_creds/ |
/app/oauth_creds/ |
OAuth credential JSON files | Read-write |
./logs/ |
/app/logs/ |
Request logs and detailed logging | Read-write |
./key_usage.json |
/app/key_usage.json |
Usage statistics persistence | Read-write |
OAuth providers (Antigravity, Gemini CLI, Qwen Code, iFlow) require interactive browser authentication. Since Docker containers run headless, you must authenticate outside the container first.
- Set up the project locally:
git clone https://github.com/Mirrowel/LLM-API-Key-Proxy.git
cd LLM-API-Key-Proxy
pip install -r requirements.txt- Run the credential tool and complete OAuth flows:
python -m rotator_library.credential_tool
# Select "Add OAuth Credential" → Choose provider
# Complete authentication in browser- Deploy with Docker, mounting the oauth_creds directory:
docker compose up -d
# The oauth_creds/ directory is automatically mountedFor truly stateless deployments (no mounted credential files):
-
Complete OAuth locally as above
-
Export credentials to environment variables:
python -m rotator_library.credential_tool
# Select "Export [Provider] to .env"- Add the exported variables to your
.envfile:
# Example for Antigravity
ANTIGRAVITY_ACCESS_TOKEN="ya29.a0AfB_byD..."
ANTIGRAVITY_REFRESH_TOKEN="1//0gL6dK9..."
ANTIGRAVITY_EXPIRY_DATE="1735901234567"
ANTIGRAVITY_EMAIL="user@gmail.com"
ANTIGRAVITY_CLIENT_ID="1071006060591-..."
ANTIGRAVITY_CLIENT_SECRET="GOCSPX-..."- Deploy with Docker:
docker compose up -d
# Credentials are loaded from .env, no oauth_creds mount neededFor development or customization, use the development compose file:
# Build and run from local source
docker compose -f docker-compose.dev.yml up -d --build
# Rebuild after code changes
docker compose -f docker-compose.dev.yml up -d --build --force-recreate# Stop the proxy
docker compose down
# Restart the proxy
docker compose restart
# View real-time logs
docker compose logs -f
# Check container resource usage
docker stats llm-api-proxy
# Update to latest image
docker compose pull
docker compose up -dThe image is built for both linux/amd64 and linux/arm64 architectures, so it works on:
- Linux servers (x86_64, ARM64)
- macOS (Intel and Apple Silicon)
- Windows with WSL2/Docker Desktop
- Raspberry Pi 4+ (ARM64)
| Issue | Solution |
|---|---|
| Container exits immediately | Check logs: docker compose logs — likely missing .env or invalid config |
| Permission denied on volumes | Ensure directories exist and have correct permissions: mkdir -p oauth_creds logs && chmod 755 oauth_creds logs |
| OAuth credentials not loading | Verify oauth_creds/ is mounted and contains valid JSON files, or check environment variables are set |
| Port already in use | Change the port mapping: -p 9000:8000 or edit docker-compose.yml |
| Image not updating | Force pull: docker compose pull && docker compose up -d |
If you're deploying the proxy to a custom VPS (DigitalOcean, AWS EC2, Linode, etc.) instead of Render.com, you'll encounter special considerations when setting up OAuth providers (Antigravity, Gemini CLI, iFlow). This section covers the professional deployment workflow.
OAuth providers like Antigravity, Gemini CLI, and iFlow require an interactive authentication flow that:
- Opens a browser for you to log in
- Redirects back to a local callback server running on specific ports
- Receives an authorization code to exchange for tokens
The callback servers bind to localhost on these ports:
| Provider | Port | Notes |
|---|---|---|
| Antigravity | 51121 | Google OAuth with extended scopes |
| Gemini CLI | 8085 | Google OAuth for Gemini API |
| iFlow | 11451 | Authorization Code flow with API key fetch |
| Qwen Code | N/A | Uses Device Code flow - works on remote VPS ✅ |
The Issue: When running on a remote VPS, your local browser cannot reach http://localhost:51121 (or other callback ports) on the remote server, causing authentication to fail with a "connection refused" error.
There are three professional approaches to handle OAuth authentication for VPS deployment, listed from most recommended to least:
This is the cleanest and most secure approach. Complete OAuth flows on your local machine, export to environment variables, then deploy.
# On your local development machine
git clone https://github.com/YOUR-USERNAME/LLM-API-Key-Proxy.git
cd LLM-API-Key-Proxy
# Install dependencies
pip install -r requirements.txt# Start the credential tool
python -m rotator_library.credential_toolSelect "Add OAuth Credential" and choose your provider:
- Antigravity
- Gemini CLI
- iFlow
- Qwen Code (works directly on VPS, but can authenticate locally too)
The tool will:
- Open your browser automatically
- Start a local callback server
- Complete the OAuth flow
- Save credentials to
oauth_creds/<provider>_oauth_N.json
Still in the credential tool, select the export option for each provider:
- "Export Antigravity to .env"
- "Export Gemini CLI to .env"
- "Export iFlow to .env"
- "Export Qwen Code to .env"
The tool generates a .env file snippet like:
# Antigravity OAuth Credentials
ANTIGRAVITY_ACCESS_TOKEN="ya29.a0AfB_byD..."
ANTIGRAVITY_REFRESH_TOKEN="1//0gL6dK9..."
ANTIGRAVITY_EXPIRY_DATE="1735901234567"
ANTIGRAVITY_EMAIL="user@gmail.com"
ANTIGRAVITY_CLIENT_ID="1071006060591-..."
ANTIGRAVITY_CLIENT_SECRET="GOCSPX-..."
ANTIGRAVITY_TOKEN_URI="https://oauth2.googleapis.com/token"
ANTIGRAVITY_UNIVERSE_DOMAIN="googleapis.com"Copy these variables to a file (e.g., oauth_credentials.env).
Method A: Using Environment Variables (Recommended)
# On your VPS
cd /path/to/LLM-API-Key-Proxy
# Create or edit .env file
nano .env
# Paste the exported environment variables
# Also add your PROXY_API_KEY and other provider keys
# Start the proxy
uvicorn src.proxy_app.main:app --host 0.0.0.0 --port 8000Method B: Upload Credential Files
# On your local machine - copy credential files to VPS
scp -r oauth_creds/ user@your-vps-ip:/path/to/LLM-API-Key-Proxy/
# On VPS - verify files exist
ls -la oauth_creds/
# Start the proxy
uvicorn src.proxy_app.main:app --host 0.0.0.0 --port 8000Note: Environment variables are preferred for production deployments (more secure, easier to manage, works with container orchestration).
If you need to authenticate directly on the VPS (e.g., you don't have a local development environment), use SSH port forwarding to create secure tunnels.
SSH tunnels forward ports from your local machine to the remote VPS, allowing your local browser to reach the callback servers.
Step 1: Create SSH Tunnels
From your local machine, open a terminal and run:
# Forward all OAuth callback ports at once
ssh -L 51121:localhost:51121 -L 8085:localhost:8085 -L 11451:localhost:11451 user@your-vps-ip
# Alternative: Forward ports individually as needed
ssh -L 51121:localhost:51121 user@your-vps-ip # For Antigravity
ssh -L 8085:localhost:8085 user@your-vps-ip # For Gemini CLI
ssh -L 11451:localhost:11451 user@your-vps-ip # For iFlowKeep this SSH session open during the entire authentication process.
Step 2: Run Credential Tool on VPS
In the same SSH terminal (or open a new SSH connection):
cd /path/to/LLM-API-Key-Proxy
# Ensure Python dependencies are installed
pip install -r requirements.txt
# Run the credential tool
python -m rotator_library.credential_toolStep 3: Complete OAuth Flow
- Select "Add OAuth Credential" → Choose your provider
- The tool displays an authorization URL
- Click the URL in your local browser (works because of the SSH tunnel!)
- Complete the authentication flow
- The browser redirects to
localhost:<port>- this now routes through the tunnel to your VPS - Credentials are saved to
oauth_creds/on the VPS
Step 4: Export to Environment Variables
Still in the credential tool:
- Select the export option for each provider
- Copy the generated environment variables
- Add them to
/path/to/LLM-API-Key-Proxy/.envon your VPS
Step 5: Close Tunnels and Deploy
# Exit the SSH session with tunnels (Ctrl+D or type 'exit')
# Tunnels are no longer needed
# Start the proxy on VPS (in a screen/tmux session or as a service)
uvicorn src.proxy_app.main:app --host 0.0.0.0 --port 8000If you've already authenticated locally and have credential files, you can copy them directly.
# From your local machine
scp -r oauth_creds/ user@your-vps-ip:/path/to/LLM-API-Key-Proxy/
# Verify on VPS
ssh user@your-vps-ip
ls -la /path/to/LLM-API-Key-Proxy/oauth_creds/Expected files:
antigravity_oauth_1.jsongemini_cli_oauth_1.jsoniflow_oauth_1.jsonqwen_code_oauth_1.json
On your VPS, edit .env:
# Option A: Use credential files directly (not recommended for production)
# No special configuration needed - the proxy auto-detects oauth_creds/ folder
# Option B: Export to environment variables (recommended)
# Run credential tool and export each provider to .env| Aspect | Environment Variables | Credential Files |
|---|---|---|
| Security | ✅ More secure (no files on disk) | |
| Container-Friendly | ✅ Perfect for Docker/K8s | ❌ Requires volume mounts |
| Ease of Rotation | ✅ Update .env and restart | |
| Backup/Version Control | ✅ Easy to manage with secrets managers | ❌ Binary files, harder to manage |
| Auto-Refresh | ✅ Uses refresh tokens | ✅ Uses refresh tokens |
| Recommended For | Production deployments | Local development / testing |
Best Practice: Always export to environment variables for VPS/cloud deployments.
- Never commit
.envoroauth_creds/to version control - Use environment variables instead of credential files in production
- Secure your VPS firewall - do not open OAuth callback ports (51121, 8085, 11451) to public internet
- Use SSH port forwarding only during initial authentication
- Rotate credentials regularly using the credential tool's export feature
- Set file permissions on
.env:chmod 600 .env
OAuth callback ports should never be publicly exposed:
# ❌ DO NOT DO THIS - keeps ports closed
# sudo ufw allow 51121/tcp
# sudo ufw allow 8085/tcp
# sudo ufw allow 11451/tcp
# ✅ Only open your proxy API port
sudo ufw allow 8000/tcp
# Check firewall status
sudo ufw statusThe SSH tunnel method works without opening these ports because traffic routes through the SSH connection (port 22).
Create a systemd service file on your VPS:
# Create service file
sudo nano /etc/systemd/system/llm-proxy.service[Unit]
Description=LLM API Key Proxy
After=network.target
[Service]
Type=simple
User=your-username
WorkingDirectory=/path/to/LLM-API-Key-Proxy
Environment="PATH=/path/to/python/bin"
ExecStart=/path/to/python/bin/uvicorn src.proxy_app.main:app --host 0.0.0.0 --port 8000
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target# Enable and start the service
sudo systemctl daemon-reload
sudo systemctl enable llm-proxy
sudo systemctl start llm-proxy
# Check status
sudo systemctl status llm-proxy
# View logs
sudo journalctl -u llm-proxy -fCause: Trying to authenticate directly on VPS without SSH tunnel.
Solution: Use Option 1 (authenticate locally) or Option 2 (SSH port forwarding).
# Check if environment variables are set
printenv | grep -E '(ANTIGRAVITY|GEMINI_CLI|IFLOW|QWEN_CODE)'
# Verify .env file exists and is readable
ls -la .env
cat .env | grep -E '(ANTIGRAVITY|GEMINI_CLI|IFLOW|QWEN_CODE)'
# Check credential files if using file-based approach
ls -la oauth_creds/The proxy automatically refreshes tokens using refresh tokens. If refresh fails:
- Re-authenticate: Run credential tool again and export new credentials
- Check token expiry: Some providers require periodic re-authentication
- Verify credentials: Ensure
REFRESH_TOKENis present in environment variables
# Set correct permissions
chmod 600 .env
chown your-username:your-username .env- Authenticate locally on your development machine (easiest, most secure)
- Export to environment variables using the credential tool's built-in export feature
- Deploy to VPS by adding environment variables to
.env - Never open OAuth callback ports to the public internet
- Use SSH port forwarding only if you must authenticate directly on VPS
- Run as a systemd service for production reliability
- Monitor logs for authentication errors and token refresh issues
This approach ensures secure, production-ready deployment while maintaining the convenience of OAuth authentication.