This proxy is intended for local developer use with ZCode and NVIDIA NIM OpenAI-compatible chat completions. Keep the default loopback binding unless you have separately configured TLS termination, firewall rules, and client access controls.
- Do not commit API keys.
- Do not commit
.env; it is intentionally ignored by Git. - In default
envmode, setNVIDIA_API_KEYin the local environment. - In
clientmode, put each NVIDIA API key in the corresponding ZCode provider API key field. - In
poolmode, keep real NVIDIA keys only in.envand put onlyNIM_PROXY_CLIENT_KEYin ZCode. - Generate a random local proxy key, keep it different from every NVIDIA key, and rotate it if exposed.
- Restrict
.envfilesystem permissions to the Windows account that runs the proxy. - The
.envloader accepts only the documented credential names and never executes file content. - Upstream URLs containing credentials, query parameters, or fragments are rejected before the proxy binds.
- The proxy never logs API key values, local proxy credentials, or full message content in any mode.
/healthexposes aggregate queue/key-state counts only. It does not expose raw keys or fingerprints.
env mode cannot bind to a non-loopback address. client and pool modes still require explicit --allow-remote; that flag does not add TLS or firewall protection.
408and429cool down only the affected key.401and403quarantine only the affected key until restart.- A quarantined key should be removed or replaced in
.envbefore restarting. - Streaming requests are never replayed after response streaming begins.
When reporting issues, include:
- proxy version
- Python version
- sanitized stripped key names
- HTTP status code
- queue mode and aggregate
/healthcounts
Do not include .env, API keys, local proxy credentials, key fingerprints, or full prompt/message content.