Multi-Provider LLM CLI Tool

A resilient, type-safe command-line tool built to query and stream across OpenAI GPT and Anthropic Claude models. Designed with strict SOLID and DRY design patterns, featuring exact cost metrics and real-time streaming.

---

Architecture

This project is structured as a modular package to isolate provider logic and enable clean dependency inversion:

classDiagram
  direction TB

  class BaseLLMClient {
    <<abstract>>
    +model : str
    +stream(prompt, temp, max_tokens)* Generator
    +complete(prompt, temp, max_tokens)* tuple
  }

  class OpenAIClient {
    +client : OpenAI
    +stream(...)
    +complete(...)
  }

  class AnthropicClient {
    +client : Anthropic
    +stream(...)
    +complete(...)
  }

  class LLMClient {
    -_delegate : BaseLLMClient
    +stream(...)
    +complete(...)
  }

  BaseLLMClient <|-- OpenAIClient
  BaseLLMClient <|-- AnthropicClient
  LLMClient --> BaseLLMClient

Loading

Pattern Highlights

• SRP / OCP: Provider SDKs are isolated in openai_client.py and anthropic_client.py. Adding a new provider requires zero modifications to existing clients.
• DRY: Routing is resolved once at instantiation inside LLMClient (factory.py), removing duplicate if/else loops in operational methods.
• Resiliency: Built-in exponential-backoff retries via tenacity protecting against rate limits, connection drops, and HTTP 5xx errors.

---

Setup & Installation

1. Initialize Sandbox

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

2. Configure Credentials

Create a .env file in the root directory:

OPENAI_API_KEY="your-openai-key"
ANTHROPIC_API_KEY="your-anthropic-key"

---

Usage

# Query default model (gpt-4o-mini)
python cli.py "Explain quantum computing in one sentence."

# Query Anthropic Claude 3.5 Sonnet
python cli.py "Explain quantum computing in one sentence." --model "claude-3-5-sonnet"

# Disable streaming
python cli.py "What is the capital of France?" --no-stream

# Output metrics and response as JSON
python cli.py "List SOLID principles." --json-output

---

Testing

Verify correct execution of all mocked clients and gateway routing:

python -m unittest test_cli.py -v