From a49bada532d64e0b93bd078c3aa4e367a97be6b0 Mon Sep 17 00:00:00 2001 From: Isaac Miller Date: Thu, 13 Nov 2025 21:10:38 -0500 Subject: [PATCH 1/2] try shortening docs --- README.md | 186 +--------- docs/OPENAPI.md | 184 +++------- docs/commands/index.md | 23 +- docs/commands/new.md | 165 +-------- docs/commands/serve.md | 132 +------ docs/deployment.md | 626 +++++++--------------------------- docs/getting-started.md | 347 ++----------------- docs/index.md | 25 +- docs/use-cases/ai-features.md | 117 +------ mkdocs.yml | 18 +- 10 files changed, 239 insertions(+), 1584 deletions(-) diff --git a/README.md b/README.md index 1a74b55..8dce059 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,8 @@ # dspy-cli -Deploy DSPy programs as HTTP APIs with standard containerization, routing, and OpenAPI specifications. Setup time: 3-5 minutes. +CLI for deploying DSPy programs as HTTP APIs. Auto-generates endpoints, OpenAPI specs, and Docker configs. -**For:** Developers embedding AI features in existing applications (Chrome extensions, Notion automations, web apps) who need HTTP endpoints without manual infrastructure work. - -**Problem:** Prototype DSPy modules work locally but require Docker configs, API boilerplate, and route management to deploy. This creates friction between development and production. - -**Solution:** Convention-based project structure with auto-discovery. Define signatures, implement modules, deploy. Infrastructure handled automatically. +Reduces deployment setup from hours to minutes for developers embedding LLM features in applications. ## Quick Start @@ -40,177 +36,29 @@ Response: ## Features -- **Auto-discovery** - Modules in `src/*/modules/` automatically exposed as HTTP endpoints -- **Standard containers** - Docker image generation with FastAPI server included -- **OpenAPI specs** - Auto-generated from DSPy signatures for integration with tools and clients -- **Hot reload** - Edit modules without restarting the development server -- **Model configuration** - Switch LLMs via config file without code changes -- **MCP tool integration** - Served programs work as MCP tools for AI assistants -- **Convention-based structure** - Organized directories for modules, signatures, optimizers, metrics - -## Project Structure - -``` -my-project/ -├── pyproject.toml -├── dspy.config.yaml # Model registry and configuration -├── .env # API keys and secrets -├── README.md -├── src/ -│ └── dspy_project/ # Importable package -│ ├── __init__.py -│ ├── modules/ # DSPy program implementations -│ ├── signatures/ # Reusable signatures -│ ├── optimizers/ # Optimizer configurations -│ ├── metrics/ # Evaluation metrics -│ └── utils/ # Shared helpers -├── data/ -├── logs/ -└── tests/ -``` +- Auto-discovery of modules as HTTP endpoints +- Docker configs and OpenAPI specs generated +- Hot reload development server +- Model switching via config file +- MCP tool support ## Commands -### Create Project - -```bash -dspy-cli new PROJECT_NAME [OPTIONS] -``` - -Options: -- `-s, --signature TEXT` - Inline signature (e.g., `"question -> answer"`) -- `-p, --program-name TEXT` - Initial program name - -### Generate Components - -```bash -dspy-cli generate scaffold PROGRAM_NAME [OPTIONS] -``` - -Options: -- `-m, --module TEXT` - Module type (`Predict`, `ChainOfThought`, `ReAct`) -- `-s, --signature TEXT` - Inline signature (e.g. "question -> answer") - -### Serve Locally - -```bash -dspy-cli serve [--port PORT] [--host HOST] -``` - -Auto-generated endpoints: -- `GET /programs` - List all programs with schemas -- `POST /{program}` - Execute program with JSON payload - -### Deploy - -```bash -flyctl launch -flyctl secrets set OPENAI_API_KEY=your-key-here -flyctl deploy -``` - -See [Deployment Guide](docs/deployment.md) for detailed instructions and other platforms. - -## Configuration - -### Model Settings - -`dspy.config.yaml`: - -```yaml -models: - default: openai:gpt-4o-mini - registry: - openai:gpt-4o-mini: - model: openai/gpt-4o-mini - env: OPENAI_API_KEY - max_tokens: 16000 - temperature: 1.0 - model_type: chat - -# Per-program overrides -program_models: - MySpecialProgram: anthropic:sonnet-4-5 -``` - -### Environment Variables - -`.env`: - -``` -OPENAI_API_KEY=sk-... -ANTHROPIC_API_KEY=sk-ant-... -``` - -See [Configuration Guide](docs/configuration.md) for complete settings reference. - -## Integration Examples - -### Chrome Extension - -```javascript -async function summarizePage(content) { - const response = await fetch('https://your-app.fly.dev/summarizer', { - method: 'POST', - body: JSON.stringify({ content }), - headers: { 'Content-Type': 'application/json' } - }); - return response.json(); -} -``` - -### Python Client - -```python -import requests - -def tag_content(page_content): - response = requests.post( - 'https://your-app.fly.dev/BlogTaggerPredict', - json={'post': page_content} - ) - return response.json()['tags'] -``` - -### JavaScript/TypeScript - -```javascript -fetch('https://your-app.fly.dev/analyzer', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ - text: document.body.innerText, - context: ['documentation', 'technical'] - }) -}) -.then(res => res.json()) -.then(data => console.log(data.summary, data.sentiment)); -``` - -## Development Installation - -For contributing or testing dspy-cli itself: - ```bash -cd /path/to/dspy-cli -uv sync --extra dev -dspy-cli --help +dspy-cli new [-s "input -> output"] # Create project +dspy-cli serve [--ui] # Start HTTP server +dspy-cli g scaffold [-m CoT] # Add module to project ``` -Run tests: - -```bash -cd dspy-cli -uv run pytest -``` +See [Command Reference](docs/commands/) for complete documentation. -## Next Steps +## Documentation -- [Architecture Overview](docs/architecture.md) - Project structure and design decisions -- [CLI Reference](docs/cli-reference.md) - Complete command documentation -- [Configuration Guide](docs/configuration.md) - Model settings and environment variables -- [Deployment Guide](docs/deployment.md) - Production deployment workflows -- [MCP Integration](docs/mcp-integration.md) - Using with AI assistants +- [Getting Started](docs/getting-started.md) - Quickstart guide +- [Commands](docs/commands/) - CLI reference +- [Deployment](docs/deployment.md) - Production deployment +- [Configuration](docs/configuration.md) - Model and environment settings +- [Examples](examples/) - Sample projects ## License diff --git a/docs/OPENAPI.md b/docs/OPENAPI.md index 0bb5a40..81808a2 100644 --- a/docs/OPENAPI.md +++ b/docs/OPENAPI.md @@ -1,53 +1,40 @@ # OpenAPI Specification Generation -The `dspy-cli serve` command automatically generates OpenAPI 3.1.0 specifications for your DSPy programs. +`dspy-cli serve` automatically generates OpenAPI 3.1.0 specifications for DSPy programs. ## Features - **Automatic generation** on server start (enabled by default) -- **Multiple formats** supported: JSON and YAML +- **Multiple formats**: JSON and YAML - **Enhanced metadata** from `dspy.config.yaml` (app_id, description) -- **DSPy-specific extensions** with program and model information -- **Always available** via `/openapi.json` endpoint, regardless of file generation +- **DSPy extensions** with program and model information +- **Always available** at `/openapi.json` endpoint ## Usage -### Default Behavior (JSON) - ```bash +# JSON format (default) dspy-cli serve -# Creates openapi.json in project root -``` - -### YAML Format -```bash +# YAML format dspy-cli serve --openapi-format yaml -# Creates openapi.yaml in project root -``` - -### Disable File Generation -```bash +# Disable file generation (still available at /openapi.json) dspy-cli serve --no-save-openapi -# Spec still available at http://localhost:8000/openapi.json ``` ## Generated Specification -### Standard OpenAPI Fields - -The generated spec includes: +Includes: -- **Endpoints**: All discovered DSPy programs as POST endpoints -- **Request schemas**: Dynamically generated from DSPy module forward() signatures -- **Response schemas**: Output types from your modules -- **Validation**: Input/output validation via Pydantic models -- **Additional endpoints**: `/programs` for listing all programs +- **Endpoints**: All DSPy programs as POST endpoints +- **Request/response schemas**: From module `forward()` signatures +- **Validation**: Input/output validation via Pydantic +- **Additional endpoints**: `/programs` for listing programs -### DSPy-Specific Extensions +### DSPy Extensions -The spec includes custom OpenAPI extensions (x-* fields): +Custom OpenAPI extensions (x-* fields): ```json { @@ -66,8 +53,7 @@ The spec includes custom OpenAPI extensions (x-* fields): } ], "x-dspy-program-models": { - "CategorizerPredict": "openai:gpt-4", - "SummarizerCoT": "anthropic:claude-3-sonnet" + "CategorizerPredict": "openai:gpt-4" } } } @@ -75,83 +61,62 @@ The spec includes custom OpenAPI extensions (x-* fields): ## Configuration -### Metadata Source - -The OpenAPI spec pulls metadata from `dspy.config.yaml`: +Metadata from `dspy.config.yaml`: ```yaml app_id: my-blog-tools description: A set of functions for a content management system. ``` -These values populate the OpenAPI `title` and `description` fields. - -### Fallback Defaults - -If not specified in config: +Fallback defaults if not specified: - **title**: "DSPy API" - **description**: "Automatically generated API for DSPy programs" - **version**: "0.1.0" ## Accessing the Specification -### As a File - +**As a file:** ```bash -# After running dspy-cli serve cat openapi.json ``` -### Via HTTP Endpoint - +**Via HTTP:** ```bash -# While server is running curl http://localhost:8000/openapi.json ``` -### In Code - +**In code:** ```python -from fastapi import FastAPI from dspy_cli.utils.openapi import generate_openapi_spec - -# After creating your FastAPI app spec = generate_openapi_spec(app) ``` -## Integration with Tools +## Integration -### Swagger UI +### Interactive Documentation -FastAPI automatically provides interactive API documentation at: +FastAPI provides: - **Swagger UI**: `http://localhost:8000/docs` - **ReDoc**: `http://localhost:8000/redoc` -### API Clients - -Use the generated spec to create type-safe API clients: +### Generate API Clients ```bash -# Generate TypeScript client +# TypeScript npx openapi-typescript openapi.json -o types.ts -# Generate Python client +# Python openapi-generator-cli generate -i openapi.json -g python ``` ### Validation -Validate your spec: - ```bash -# Install validator npm install -g @ibm/openapi-validator - -# Validate spec lint-openapi openapi.json ``` -## Command-Line Options +## Command Options ```bash dspy-cli serve [OPTIONS] @@ -164,93 +129,26 @@ Options: (default: json) ``` -## Examples - -### Example: JSON Spec - -```bash -cd my-project -dspy-cli serve -``` - -Creates `openapi.json`: -```json -{ - "openapi": "3.1.0", - "info": { - "title": "my-project", - "description": "Automatically generated API for DSPy programs", - "version": "0.1.0" - }, - "paths": { - "/MyProgramPredict": { - "post": { - "summary": "Run Program", - "requestBody": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/MyProgramPredictRequest" - } - } - } - } - } - } - } -} -``` - -### Example: YAML Spec - -```bash -dspy-cli serve --openapi-format yaml -``` - -Creates `openapi.yaml`: -```yaml -openapi: 3.1.0 -info: - title: my-project - description: Automatically generated API for DSPy programs - version: 0.1.0 -paths: - /MyProgramPredict: - post: - summary: Run Program - requestBody: - content: - application/json: - schema: - $ref: '#/components/schemas/MyProgramPredictRequest' -``` - ## Troubleshooting -### Spec Not Generated - -If `openapi.json` isn't created: -1. Check that `--no-save-openapi` wasn't used -2. Verify write permissions in project directory -3. Check server logs for errors +**Spec not generated:** +1. Verify `--no-save-openapi` wasn't used +2. Check write permissions +3. Check server logs -### Missing Program Schemas - -If programs don't appear in the spec: +**Missing program schemas:** 1. Ensure modules subclass `dspy.Module` -2. Verify `forward()` method has type annotations -3. Check that modules are in `src//modules/` - -### Incorrect Metadata +2. Verify `forward()` has type annotations +3. Check modules are in `src//modules/` -If title/description are wrong: +**Incorrect metadata:** 1. Check `app_id` in `dspy.config.yaml` -2. Verify `description` field in config +2. Verify `description` field 3. Ensure config file is in project root -## Implementation Details +## Implementation -- **Utility module**: `src/dspy_cli/utils/openapi.py` -- **Integration point**: `src/dspy_cli/server/app.py` (metadata enhancement) -- **File generation**: `src/dspy_cli/server/runner.py` (on server start) -- **Uses**: FastAPI's built-in `app.openapi()` method +- **Utility**: `src/dspy_cli/utils/openapi.py` +- **Integration**: `src/dspy_cli/server/app.py` +- **Generation**: `src/dspy_cli/server/runner.py` +- **Uses**: FastAPI's `app.openapi()` method diff --git a/docs/commands/index.md b/docs/commands/index.md index 13a4aa2..7d66c90 100644 --- a/docs/commands/index.md +++ b/docs/commands/index.md @@ -1,29 +1,20 @@ # Commands -DSPy CLI helps you build LLM systems that iterate fast and scale without breaking. Start simple, add complexity when you need it, and let DSPy handle optimization. - -## Workflow - -``` -dspy-cli new → Start with structure that scales -dspy-cli serve → Iterate without breaking things -dspy-cli generate → Add complexity safely -[Future: dspy-cli optimize → run DSPy optimization] -``` +CLI commands for scaffolding, serving, and generating DSPy components. ## Available Commands -### [new](new.md) - Start with structure that scales +### [new](new.md) -Create projects with built-in best practices. Skip the boilerplate, avoid common pitfalls, and get a foundation ready for DSPy's optimization tools. +Create new DSPy project with standard directory structure. -### [serve](serve.md) - Iterate without breaking things +### [serve](serve.md) -Test your prompts and logic with a live API server. Auto-discovery turns your modules into REST endpoints. Keep your development loop tight. +Start HTTP server that exposes modules as REST endpoints. -### [generate](generate.md) - Add complexity safely +### [generate](generate.md) -Scaffold new components with the right patterns. Generate signatures, modules, and programs that integrate cleanly with your existing code and DSPy's optimizer. +Generate new modules, signatures, and programs in existing projects. ## Aliases diff --git a/docs/commands/new.md b/docs/commands/new.md index 5adad2e..08d0929 100644 --- a/docs/commands/new.md +++ b/docs/commands/new.md @@ -23,9 +23,7 @@ dspy-cli new notion-tools -p emoji_picker -s "context: str -> emoji: str" ## Description -Creates a new project with standard directory layout, dependency configuration, and initial DSPy module. Generates `modules/`, `signatures/`, `optimizers/`, `metrics/`, and `utils/` directories along with configuration files (`pyproject.toml`, `dspy.config.yaml`, `Dockerfile`, `.env`). - -The initial module is created based on the provided signature (or a default `question -> answer` signature if none is specified). +Creates project with standard directory layout, dependency configuration, and initial DSPy module. Default signature is `question -> answer` if none specified. ## Options @@ -58,125 +56,12 @@ my-feature/ └── .env # Environment variables ``` -### Directory Purposes - -- **modules/** - DSPy program implementations (`dspy.Module` subclasses) -- **signatures/** - Type-safe interfaces defining inputs and outputs -- **optimizers/** - Workflows for improving program performance -- **metrics/** - Functions for measuring program accuracy -- **utils/** - Shared code across modules -- **data/** - Example datasets for training and testing -- **logs/** - Production request/response logs -- **tests/** - Unit and integration tests - ## Signature Syntax -Signatures define the program interface using `input -> output` format with optional type annotations. - -### Basic Format - -```bash -# Single input, single output --s "question -> answer" - -# Multiple inputs --s "context, question -> answer" - -# Multiple outputs --s "text -> summary, key_points" -``` - -### Type Annotations - -```bash -# String types --s "text: str -> summary: str" - -# Lists --s "post: str -> tags: list[str]" - -# Multiple typed outputs --s "email: str -> summary: str, action_items: list[str], priority: str" -``` - -### Examples - -```bash -# Email summarizer --s "email: str -> summary: str, key_points: list[str]" - -# Content classifier --s "email: str, user_context: str -> category: str, confidence: float" - -# Code analyzer --s "code: str, language: str -> explanation: str, complexity: str" - -# Content moderator --s "content: str -> is_safe: bool, issues: list[str]" -``` - -## Behavior - -### Module Generation - -The command generates an initial module based on the signature: - -**Command:** -```bash -dspy-cli new emoji-picker -s "context: str -> emoji: str, reasoning: str" -``` - -**Generated signature** (`src/emoji_picker/signatures/emoji_picker_signature.py`): - -```python -import dspy - -class EmojiPickerSignature(dspy.Signature): - context: str = dspy.InputField(desc="") - emoji: str = dspy.OutputField(desc="") -``` - -**Generated module** (`src/emoji_picker/modules/emoji_picker_predict.py`): - -```python -import dspy -from emoji_picker.signatures.emoji_picker_signature import EmojiPickerSignature - -class EmojiPickerPredict(dspy.Module): - def __init__(self): - super().__init__() - self.predictor = dspy.Predict(EmojiPickerSignature) - - def forward(self, context: str) -> dspy.Prediction: - return self.predictor(context=context) -``` - -### Program Naming - -- Without `-p`: Program name derived from project name (`blog-tools` → `blog_tools`) -- With `-p`: Uses specified program name (`-p tagger` → `TaggerPredict`) - -### Dependencies - -`pyproject.toml` includes: -- `dspy-ai` - Core DSPy framework -- `python-dotenv` - Environment variable management -- Development dependencies for testing +Format: `input -> output` with optional type annotations. See [dspy-cli generate](generate.md) for complete syntax reference. ## Examples -### Basic Project - -```bash -dspy-cli new my-feature -cd my-feature -uv sync -source .venv/bin/activate -dspy-cli serve -``` - -Creates project with default `question -> answer` signature. Serves at `http://localhost:8000/MyFeaturePredict`. - ### Email Summarizer ```bash @@ -206,50 +91,12 @@ Project named `blog-tools`, initial program named `TaggerPredict`. Additional pr ## Next Steps -After creating a project: - -1. **Configure API keys** - Edit `.env`: - ```bash - OPENAI_API_KEY=sk-... - ``` - -2. **Install dependencies:** - ```bash - cd your-project - uv sync - source .venv/bin/activate - ``` - -3. **Customize signature descriptions** - Edit generated signature file: - ```python - class EmojiPickerSignature(dspy.Signature): - """Select appropriate emoji based on context.""" - - context: str = dspy.InputField(desc="Text surrounding emoji location") - emoji: str = dspy.OutputField(desc="Single emoji character") - reasoning: str = dspy.OutputField(desc="Explanation for emoji choice") - ``` - -4. **Start development server:** - ```bash - dspy-cli serve --ui - ``` - -5. **Test the endpoint:** - ```bash - curl -X POST http://localhost:8000/EmojiPickerPredict \ - -H "Content-Type: application/json" \ - -d '{"context": "Great work everyone!"}' - ``` - -## Limitations - -- Signature parsing supports basic Python types (`str`, `int`, `float`, `bool`, `list`) -- Complex types (nested objects, custom classes) require manual signature editing -- Generated modules use `dspy.Predict` by default (change to `ChainOfThought`, `ReAct`, etc. manually) +1. Configure `.env` with API key +2. Install dependencies: `uv sync` +3. Start server: `dspy-cli serve --ui` ## See Also +- [dspy-cli generate](generate.md) - Generate new modules in existing projects - [dspy-cli serve](serve.md) - Start development server - [Deployment Guide](../deployment.md) - Deploy to production -- [Configuration Guide](../configuration.md) - Model configuration diff --git a/docs/commands/serve.md b/docs/commands/serve.md index 37b3b0f..17582b7 100644 --- a/docs/commands/serve.md +++ b/docs/commands/serve.md @@ -26,9 +26,7 @@ dspy-cli serve --mcp ## Description -Starts an HTTP server for local testing or production deployment. Discovers DSPy modules in `src//modules/` and exposes each as a POST endpoint. Generates request/response schemas from module type hints and provides OpenAPI documentation. - -The server watches source files and configuration, restarting automatically on changes (unless `--no-reload` is specified). +Starts an HTTP server for local testing or production deployment. Discovers DSPy modules in `src//modules/` and exposes each as a POST endpoint. ## Options @@ -47,91 +45,11 @@ The server watches source files and configuration, restarting automatically on c ## Auto-Discovery -The server scans `src//modules/` for classes that subclass `dspy.Module`, analyzes their `forward()` method signatures, and generates POST endpoints automatically. - -**Example module:** - -```python -# src/blog_tools/modules/summarizer_predict.py -import dspy -from blog_tools.signatures.summarizer import SummarizerSignature -from typing import Literal, Optional - -class SummarizerPredict(dspy.Module): - def __init__(self): - super().__init__() - self.predictor = dspy.Predict(SummarizerSignature) - - def forward( - self, - blog_post: str, - summary_length: Literal['short', 'medium', 'long'], - tone: Optional[str] - ) -> dspy.Prediction: - return self.predictor( - blog_post=blog_post, - summary_length=summary_length, - tone=tone - ) -``` - -**Generates:** - -``` -POST /SummarizerPredict -``` - -### Endpoint Naming - -Endpoints match module class names: -- `SummarizerPredict` → `/SummarizerPredict` -- `HeadlineGeneratorCoT` → `/HeadlineGeneratorCoT` -- `TweetExtractorPredict` → `/TweetExtractorPredict` - -### Schema Generation - -Type hints from `forward()` parameters become JSON request schemas. Default values become optional fields. - -**Python signature:** - -```python -def forward( - self, - text: str, - options: list[str], - confidence_threshold: float = 0.8 -) -> dspy.Prediction: - ... -``` - -**Generated request schema:** - -```json -{ - "type": "object", - "properties": { - "text": {"type": "string"}, - "options": { - "type": "array", - "items": {"type": "string"} - }, - "confidence_threshold": { - "type": "number", - "default": 0.8 - } - }, - "required": ["text", "options"] -} -``` +Discovers modules in `src//modules/` and generates endpoints from class names. Schemas generated from `forward()` type hints. See [OpenAPI Reference](../OPENAPI.md). ## Hot Reload -The server watches for changes to: -- All `.py` files in `src/` directory -- `dspy.config.yaml` configuration file -- `.env` environment variables - -Saving any watched file triggers an automatic restart (~2 seconds). Disable with `--no-reload` for production deployments. +Watches `.py` files in `src/`, `dspy.config.yaml`, and `.env` for changes. Restarts automatically (~2s). Disable with `--no-reload` for production. ## Endpoints @@ -151,17 +69,7 @@ With `--mcp` enabled: ## Logging -Requests are logged to console by default. Use `--logs-dir` to persist logs per module: - -```bash -dspy-cli serve --logs-dir ./logs -``` - -Creates: -- `logs/SummarizerPredict.log` -- `logs/HeadlineGeneratorPredict.log` - -Each log entry includes timestamp, request inputs, response outputs, execution time, and model used. +Logs to console by default. Use `--logs-dir` to write per-module JSON logs with request/response data, execution time, and model info. ## Examples @@ -191,17 +99,6 @@ Response: } ``` -### Development Mode - -```bash -dspy-cli serve --ui --logs-dir ./logs --port 3000 -``` - -Enables: -- Interactive web UI at `http://localhost:3000` -- Hot reload on file changes -- Persistent logs in `./logs/` - ### Production Mode ```bash @@ -210,19 +107,7 @@ dspy-cli serve --no-reload --host 0.0.0.0 --port 8000 Disables hot reload and binds to all network interfaces for production deployment. -### Multiple Parameters - -```bash -curl -X POST http://localhost:8000/TaggerPredict \ - -H "Content-Type: application/json" \ - -d '{ - "blog_post": "Content here", - "max_tags": 5, - "style": "casual" - }' -``` - -### Error Responses +## Error Responses **Validation error (422):** @@ -268,10 +153,7 @@ curl -X POST http://localhost:8000/SummarizerPredict \ ## Environment Variables -Configure via environment variables or `.env` file: - -- `OPENAI_API_KEY` - OpenAI API key -- Model-specific configuration in `dspy.config.yaml` +See [Configuration](../configuration.md) for environment variable and model configuration details. ## Limitations @@ -282,5 +164,5 @@ Configure via environment variables or `.env` file: - [dspy-cli new](new.md) - Create new projects - [Deployment Guide](../deployment.md) - Deploy to production -- [OpenAPI Generation](../OPENAPI.md) - API specification details +- [OpenAPI Reference](../OPENAPI.md) - API specification details - [Configuration Guide](../configuration.md) - Model configuration diff --git a/docs/deployment.md b/docs/deployment.md index c776d2a..e433416 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -1,681 +1,289 @@ # Deployment -This guide covers deployment of DSPy applications to production environments using containerization and cloud platforms. +Deploy DSPy applications to production using containerization and cloud platforms. ## Overview -DSPy applications generated with `dspy-cli new` include production-ready deployment artifacts: +Applications include production-ready artifacts: - **Dockerfile** - Container configuration for standardized deployment -- **REST API endpoints** - Auto-generated from DSPy modules +- **REST API endpoints** - Auto-generated from DSPy modules - **OpenAPI specification** - Type-safe integration for clients - **Environment variable handling** - Secure secrets management -Local development servers created with `dspy-cli serve` use identical endpoints and behavior as production deployments. - ## Prerequisites -- Docker (version 20.10 or later) -- Active account on target deployment platform -- API keys for configured LLM providers (OpenAI, Anthropic, etc.) -- Git repository (optional, required for some platforms) +- Docker 20.10+ +- Account on target deployment platform +- API keys for LLM providers (OpenAI, Anthropic, etc.) ## Deployment to Fly.io -Time to complete: ~3-5 minutes - -Fly.io provides global container deployment with minimal configuration. - -### 1. Install Fly CLI +Time: ~3-5 minutes ```bash -# macOS +# Install and authenticate brew install flyctl - -# Linux -curl -L https://fly.io/install.sh | sh - -# Windows -pwsh -Command "iwr https://fly.io/install.ps1 -useb | iex" -``` - -### 2. Authenticate - -```bash flyctl auth login -``` - -Opens browser for authentication. Free tier available without credit card for starter applications. - -### 3. Initialize Application -Navigate to your project directory and run: - -```bash +# Launch and configure flyctl launch -``` - -**Expected output:** - -``` -Scanning source code -Detected a Dockerfile app -? Choose an app name (leave blank to generate one): blog-tools-prod -? Choose a region for deployment: Seattle, Washington (US) (sea) - -Creating app in /Users/you/blog-tools -Organization: Personal -Name: blog-tools-prod -Region: Seattle, Washington (US) -App Machines: shared-cpu-1x, 1GB RAM - -? Do you want to tweak these settings before proceeding? No -``` - -### 4. Deploy - -```bash -flyctl deploy -``` - -**Expected output:** - -``` ---> Building image -[+] Building 28.7s ---> Pushing image to fly ---> Creating release ---> Monitoring deployment - -Visit your app at: https://blog-tools-prod.fly.dev -``` - -### 5. Configure Secrets - -Set environment variables for API keys: - -```bash flyctl secrets set OPENAI_API_KEY=sk-proj-... -``` - -**Expected output:** - -``` -Secrets are staged for the first deployment -Release v2 created -``` - -For multiple providers: - -```bash -flyctl secrets set ANTHROPIC_API_KEY=sk-ant-... -``` - -Verify configured secrets: - -```bash -flyctl secrets list -``` - -**Expected output:** - -``` -NAME DIGEST CREATED AT -OPENAI_API_KEY a1b2c3d4e5f6... 1m ago -ANTHROPIC_API_KEY f6e5d4c3b2a1... 30s ago -``` - -### 6. Verify Deployment - -Test the deployed API: - -```bash -curl -X POST "https://blog-tools-prod.fly.dev/SummarizerPredict" \ - -H "Content-Type: application/json" \ - -d '{ - "blog_post": "Sample article text for testing deployment.", - "summary_length": "short", - "tone": "casual" - }' -``` - -**Expected output:** - -```json -{ - "summary": "Testing deployment with a sample article." -} -``` - -Verify OpenAPI specification is accessible: - -```bash -curl https://blog-tools-prod.fly.dev/openapi.json -``` - -### 7. Update Deployment - -Deploy changes: - -```bash flyctl deploy ``` -Zero-downtime rolling deployment. Previous version remains active until new version is healthy. - -### 8. Scale Deployment (Optional) - -Add instances in multiple regions: +App available at `https://your-app.fly.dev` +Verify: ```bash -flyctl scale count 2 --region sea,iad +curl -X POST "https://your-app.fly.dev/SummarizerPredict" \ + -d '{"blog_post": "Test"}' -H "Content-Type: application/json" ``` -Configure auto-scaling: +## Docker Deployment -```bash -flyctl autoscale set min=1 max=10 -``` - -## Docker Deployment (Generic) - -The generated Dockerfile is compatible with any container platform: Render, Railway, Google Cloud Run, AWS App Runner, DigitalOcean App Platform, Azure Container Instances. - -### 1. Build Container Locally - -```bash -docker build -t blog-tools:latest . -``` +Compatible with Render, Railway, Google Cloud Run, AWS App Runner, DigitalOcean, Azure Container Instances. -**Expected output:** - -``` -[+] Building 45.3s - => [1/5] FROM docker.io/library/python:3.11-slim - => [2/5] COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv - => [3/5] COPY . . - => [4/5] RUN uv sync --no-dev - => [5/5] RUN uv tool install dspy-cli - => exporting to image -Successfully tagged blog-tools:latest -``` - -### 2. Test Container Locally +### Build and Test ```bash -docker run -p 8000:8000 \ - -e OPENAI_API_KEY=sk-proj-... \ - blog-tools:latest +docker build -t my-app:latest . +docker run -p 8000:8000 -e OPENAI_API_KEY=sk-... my-app:latest ``` -**Expected output:** - -``` -INFO: Started server process [1] -INFO: Waiting for application startup. -INFO: Application startup complete. -INFO: Uvicorn running on http://0.0.0.0:8000 -``` - -Test endpoint in another terminal: - -```bash -curl -X POST http://localhost:8000/SummarizerPredict \ - -H "Content-Type: application/json" \ - -d '{"blog_post": "Test content", "summary_length": "short", "tone": null}' -``` - -### 3. Push to Container Registry +### Push to Registry **Docker Hub:** - ```bash -docker tag blog-tools:latest your-username/blog-tools:latest -docker push your-username/blog-tools:latest +docker tag my-app:latest username/my-app:latest +docker push username/my-app:latest ``` **AWS ECR:** - ```bash aws ecr get-login-password --region us-west-2 | \ - docker login --username AWS --password-stdin .dkr.ecr.us-west-2.amazonaws.com - -docker tag blog-tools:latest .dkr.ecr.us-west-2.amazonaws.com/blog-tools:latest -docker push .dkr.ecr.us-west-2.amazonaws.com/blog-tools:latest + docker login --username AWS --password-stdin .dkr.ecr.us-west-2.amazonaws.com +docker tag my-app:latest .dkr.ecr.us-west-2.amazonaws.com/my-app:latest +docker push .dkr.ecr.us-west-2.amazonaws.com/my-app:latest ``` **Google Artifact Registry:** - ```bash gcloud auth configure-docker us-central1-docker.pkg.dev - -docker tag blog-tools:latest us-central1-docker.pkg.dev//dspy-apps/blog-tools:latest -docker push us-central1-docker.pkg.dev//dspy-apps/blog-tools:latest +docker tag my-app:latest us-central1-docker.pkg.dev//dspy/my-app:latest +docker push us-central1-docker.pkg.dev//dspy/my-app:latest ``` -### 4. Deploy to Platform - -**Render:** +### Deploy to Platform -1. Connect GitHub repository in Render dashboard -2. Create new Web Service -3. Render auto-detects Dockerfile -4. Configure environment variables in Environment tab -5. Deploy +**Render:** Connect GitHub repository, create Web Service, Render auto-detects Dockerfile, configure environment variables, deploy. **Railway:** - ```bash railway up railway variables set OPENAI_API_KEY=sk-proj-... ``` -Auto-deploys on Git push when repository is connected. - **Google Cloud Run:** - ```bash -gcloud run deploy blog-tools \ +gcloud run deploy my-app \ --source . \ --region us-central1 \ --allow-unauthenticated \ --set-env-vars OPENAI_API_KEY=sk-proj-... ``` -**Expected output:** +**AWS App Runner:** Push image to ECR, create App Runner service, configure ECR image source, set environment variables. -``` -Building using Dockerfile and deploying container -✓ Deploying... Done. -Service [blog-tools] revision [blog-tools-00001] has been deployed -Service URL: https://blog-tools-xyz.a.run.app -``` - -**AWS App Runner:** - -1. Push image to ECR (see registry instructions above) -2. Create App Runner service in AWS Console -3. Configure ECR image as source -4. Set environment variables in Configuration -5. Deploy service +**DigitalOcean:** Connect GitHub repository, App Platform detects Dockerfile, configure environment variables. -**DigitalOcean App Platform:** +## Environment Variables -1. Connect GitHub repository -2. App Platform detects Dockerfile -3. Configure environment variables in Settings -4. Click Deploy +See [Configuration Reference](configuration.md) for detailed environment configuration. -## Environment Configuration +### Development -### Development Environment - -Use `.env` file for local development: +Use `.env` file (auto-loaded by `dspy-cli serve`, excluded via `.gitignore`): ```bash -# .env - Do not commit to version control +# .env - Do not commit OPENAI_API_KEY=sk-proj-... ANTHROPIC_API_KEY=sk-ant-... ``` -The `dspy-cli serve` command automatically loads variables from `.env`. File is excluded via `.gitignore`. - ### Production Secrets -Configure secrets using platform-specific secret managers: - -| Platform | Command | Injection Method | -|----------|---------|------------------| -| **Fly.io** | `flyctl secrets set KEY=value` | Environment variables | -| **Render** | Dashboard → Environment tab | Environment variables | -| **Railway** | `railway variables set KEY=value` | Environment variables | -| **Google Cloud Run** | `--set-env-vars KEY=value` or Secret Manager | Environment variables | -| **AWS** | Systems Manager Parameter Store / Secrets Manager | ECS task definition | -| **DigitalOcean** | App Platform dashboard → Settings | Environment variables | +Configure via platform secret managers: -### Model Configuration by Environment - -Configure different models for development and production using `dspy.config.yaml`: - -```yaml -models: - default: openai:gpt-4o-mini - - registry: - # Production model - openai:gpt-4o-mini: - model: openai/gpt-4o-mini - env: OPENAI_API_KEY - max_tokens: 16000 - temperature: 1.0 - model_type: chat - - # Development model - openai:gpt-4o-nano: - model: openai/gpt-4o-nano - env: OPENAI_API_KEY - max_tokens: 8000 - temperature: 1.0 - model_type: chat - - # Local model - local:qwen: - model: openai/qwen/qwen3-4b - api_base: http://127.0.0.1:1234/v1 - api_key: placeholder - max_tokens: 4096 - temperature: 1.0 - model_type: chat - -# Override per module -program_models: - SummarizerPredict: openai:gpt-4o-nano - TaggerCoT: openai:gpt-4o-mini -``` - -Override `default` or `program_models` before deploying, or maintain separate configuration files: - -```bash -# Use production config -cp dspy.config.prod.yaml dspy.config.yaml -``` +| Platform | Command | +|----------|---------| +| **Fly.io** | `flyctl secrets set KEY=value` | +| **Render** | Dashboard → Environment tab | +| **Railway** | `railway variables set KEY=value` | +| **Google Cloud Run** | `--set-env-vars KEY=value` or Secret Manager | +| **AWS** | Systems Manager Parameter Store / Secrets Manager | +| **DigitalOcean** | App Platform → Settings | -### Secrets Management Best Practices +### Secrets Best Practices **Required:** - Store secrets in platform secret managers - Use `.env` for local development only -- Add `.env` to `.gitignore` -- Rotate API keys regularly (monthly or quarterly) -- Use separate keys for development, staging, and production +- Rotate API keys regularly +- Use separate keys per environment **Prohibited:** -- Committing `.env` files to Git -- Hardcoding keys in source code or configuration files -- Sharing production keys in communication channels -- Using production keys in local development -- Exposing keys in client-side code - -## Health Checks and Monitoring +- Committing `.env` to Git +- Hardcoding keys in source code +- Sharing production keys +- Using production keys in development -### Health Check Endpoint +## Health Checks -Verify service availability using the OpenAPI endpoint: +Verify service availability: ```bash -curl https://blog-tools-prod.fly.dev/openapi.json +curl https://my-app.fly.dev/openapi.json ``` -Returns OpenAPI specification if service is healthy. +### Platform Configuration -### Log Access - -**Fly.io:** +**Fly.io (fly.toml):** +```toml +[http_service] + internal_port = 8000 + force_https = true + + [[http_service.checks]] + grace_period = "10s" + interval = "30s" + timeout = "5s" + method = "GET" + path = "/openapi.json" +``` +**Google Cloud Run:** ```bash -# Stream live logs -flyctl logs - -# View recent logs -flyctl logs --recent - -# Access log files in container -flyctl ssh console -cd logs/ -tail -f SummarizerPredict.log +gcloud run deploy my-app \ + --timeout=300 \ + --max-instances=10 \ + --cpu=1 \ + --memory=512Mi ``` -**Render:** +## Logs -View logs in dashboard under Logs tab. Export logs via Export Logs button. +**Fly.io:** +```bash +flyctl logs # Stream logs +flyctl logs --recent # Recent logs +``` **Railway:** - ```bash -# View live logs -railway logs - -# Follow logs in real-time -railway logs --follow +railway logs # View logs +railway logs --follow # Stream logs ``` **Google Cloud Run:** - ```bash -# View logs in Cloud Console gcloud logging read "resource.type=cloud_run_revision" - -# Stream logs gcloud logging tail "resource.type=cloud_run_revision" ``` -**AWS App Runner:** - -View logs in CloudWatch Logs console. Use CloudWatch Insights for structured queries. - -### Log Format +**Render:** Dashboard → Logs tab -Structured JSON logs capture request details: +**AWS App Runner:** CloudWatch Logs console +Structured JSON log format: ```json { "timestamp": "2024-01-15T10:30:45.123Z", "program": "SummarizerPredict", "model": "openai/gpt-4o-mini", "duration_ms": 847.23, - "inputs": { - "blog_post": "Article text...", - "summary_length": "short", - "tone": null - }, - "outputs": { - "summary": "Generated summary." - }, + "inputs": {...}, + "outputs": {...}, "success": true } ``` -### Platform-Specific Health Checks - -Configure health checks in platform configuration files: - -**Fly.io (fly.toml):** - -```toml -[http_service] - internal_port = 8000 - force_https = true - auto_start_machines = true - auto_stop_machines = true - min_machines_running = 0 - - [[http_service.checks]] - grace_period = "10s" - interval = "30s" - timeout = "5s" - method = "GET" - path = "/openapi.json" -``` - -**Google Cloud Run:** - -```bash -gcloud run deploy blog-tools \ - --timeout=300 \ - --max-instances=10 \ - --cpu=1 \ - --memory=512Mi -``` - ## Troubleshooting ### Build Errors **Error: `uv: command not found`** -Update Dockerfile to include `uv` binary: - +Update Dockerfile: ```dockerfile COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv ``` -Or regenerate project files: - -```bash -dspy-cli new --force -``` +Or regenerate: `dspy-cli new --force` **Error: `No module named 'dspy'`** -Verify `pyproject.toml` includes DSPy dependency: - -```toml -[project] -dependencies = [ - "dspy>=2.5.0", -] -``` - -Rebuild container: - -```bash -docker build -t blog-tools . -``` +Verify `pyproject.toml` includes `dspy>=2.5.0`, rebuild container. **Error: `failed to solve with frontend dockerfile.v0`** -Update Docker to version 20.10 or later. +Update Docker to 20.10+. ### Runtime Errors **Error: `OPENAI_API_KEY not set`** -Configure environment variable on deployment platform: - +Configure environment variable: ```bash -# Fly.io -flyctl secrets set OPENAI_API_KEY=sk-proj-... - -# Railway -railway variables set OPENAI_API_KEY=sk-proj-... - -# Google Cloud Run -gcloud run services update blog-tools --set-env-vars OPENAI_API_KEY=sk-proj-... -``` - -Verify secret is loaded: - -```bash -flyctl ssh console -echo $OPENAI_API_KEY +flyctl secrets set OPENAI_API_KEY=sk-proj-... # Fly.io +railway variables set OPENAI_API_KEY=sk-proj-... # Railway +gcloud run services update my-app --set-env-vars OPENAI_API_KEY=sk-... # Cloud Run ``` **Error: `Module not found: SummarizerPredict`** -Verify module exists with correct class name: - -```bash -ls src/blog_tools/modules/ -cat src/blog_tools/modules/summarizer_predict.py | grep "class" -``` - -Expected: `class SummarizerPredict(dspy.Module):` +Verify module exists with correct class name in `src//modules/`. **Error: `500 Internal Server Error`** -Check container logs: - -```bash -flyctl logs --recent # Fly.io -railway logs # Railway -``` - -Common causes: +Check logs for: - Missing environment variables -- Model API rate limits exceeded +- Model API rate limits - Invalid signature definition -- Import errors in module code - -### Performance Issues - -**Slow cold starts (10-20 seconds)** +- Import errors -Platforms may shut down idle containers. Solutions: +### Performance -- **Fly.io:** Upgrade to dedicated VM tier: `flyctl scale vm dedicated-cpu-1x` -- **Render:** Enable "Always On" in paid tiers -- **Railway:** Paid plans keep services active -- **Pre-warm with cron:** Configure external service (e.g., cron-job.org) to ping `/openapi.json` every 5 minutes - -**High latency** - -Deploy in regions closer to users: +**Slow cold starts:** Upgrade to dedicated tier, enable "Always On", or pre-warm with cron (ping `/openapi.json` every 5 minutes). +**High latency:** Deploy in multiple regions: ```bash -# Fly.io: Add regions +# Fly.io flyctl regions add sea iad lhr flyctl scale count 3 - -# Google Cloud Run: Deploy to multiple regions -gcloud run deploy blog-tools --region us-central1 -gcloud run deploy blog-tools --region europe-west1 -gcloud run deploy blog-tools --region asia-northeast1 -``` - -**Timeout errors (504 Gateway Timeout)** - -Increase platform timeout limits: - -```bash -# Google Cloud Run (maximum 300 seconds) -gcloud run deploy blog-tools --timeout=300 - -# Fly.io: Edit fly.toml http_service.checks.timeout ``` -### Debugging Workflow +**Timeout errors:** Increase platform timeout limits (`--timeout=300` for Cloud Run). -1. **Test locally with Docker:** +### Debugging -```bash -docker build -t blog-tools . -docker run -p 8000:8000 -e OPENAI_API_KEY=sk-proj-... blog-tools -curl -X POST http://localhost:8000/SummarizerPredict \ - -H "Content-Type: application/json" \ - -d '{"blog_post": "test", "summary_length": "short", "tone": null}' -``` - -2. **Verify OpenAPI specification:** - -```bash -curl https://blog-tools-prod.fly.dev/openapi.json -``` - -3. **Check application logs:** - -```bash -flyctl logs --recent -``` - -4. **Verify environment variables:** - -```bash -flyctl ssh console -env | grep API_KEY -``` +1. Test locally: `docker run -p 8000:8000 -e OPENAI_API_KEY=sk-... my-app` +2. Verify OpenAPI: `curl https://my-app.fly.dev/openapi.json` +3. Check logs: `flyctl logs --recent` +4. Verify env vars: `flyctl ssh console` → `env | grep API_KEY` ## Related Documentation - [Configuration](configuration.md) - Model configuration and advanced settings -- [MCP Integration](OPENAPI.md) - Model Context Protocol tool server +- [OpenAPI Generation](OPENAPI.md) - OpenAPI spec and MCP integration - [Getting Started](getting-started.md#continuous-optimization) - Production data optimization - [Use Cases: AI Features](use-cases/ai-features.md) - Integration patterns ## Platform Documentation -- [Fly.io Documentation](https://fly.io/docs/) -- [Render Documentation](https://render.com/docs) -- [Railway Documentation](https://docs.railway.app/) -- [Google Cloud Run Documentation](https://cloud.google.com/run/docs) -- [AWS App Runner Documentation](https://docs.aws.amazon.com/apprunner/) -- [DigitalOcean App Platform Documentation](https://docs.digitalocean.com/products/app-platform/) +- [Fly.io](https://fly.io/docs/) +- [Render](https://render.com/docs) +- [Railway](https://docs.railway.app/) +- [Google Cloud Run](https://cloud.google.com/run/docs) +- [AWS App Runner](https://docs.aws.amazon.com/apprunner/) +- [DigitalOcean](https://docs.digitalocean.com/products/app-platform/) diff --git a/docs/getting-started.md b/docs/getting-started.md index d5b0d06..e153070 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -56,11 +56,6 @@ Creating new DSPy project: email-subject ```bash dspy-cli new email-subject -s "body, sender, context -> subject, tone, priority" cd email-subject -``` - -Configure environment (if not done during creation): - -```bash echo "OPENAI_API_KEY=sk-..." > .env uv sync ``` @@ -68,344 +63,40 @@ uv sync ## 3. Run Locally ```bash -dspy-cli serve --ui --reload +dspy-cli serve --ui ``` -**Expected output:** -``` -✓ Configuration loaded - -Discovered Programs: - • EmailSubjectPredict - POST /EmailSubjectPredict - -Server running at http://0.0.0.0:8000 -Interactive UI at http://localhost:8000 -``` +Server starts at `http://localhost:8000` with interactive UI. -## 4. Verify Endpoints - -Test with curl: +## 4. Test Endpoint ```bash curl -X POST http://localhost:8000/EmailSubjectPredict \ -H "Content-Type: application/json" \ - -d '{ - "body": "Team, Q4 results are in. Revenue up 23%, NPS at 72.", - "sender": "Sarah (CEO)", - "context": "Company all-hands" - }' -``` - -**Expected response:** -```json -{ - "subject": "Q4 Results: +23% Revenue, 72 NPS", - "tone": "professional", - "priority": "high" -} + -d '{"body": "Team meeting notes...", "sender": "CEO", "context": "Internal"}' ``` -View interactive UI at http://localhost:8000 to test without curl. - -## 5. Deploy to Production +Or use interactive UI at `http://localhost:8000`. -### Option A: Fly.io (Recommended) +## 5. Deploy -Install Fly CLI: +See [Deployment Guide](deployment.md) for Fly.io, Docker, AWS, GCP, and Kubernetes instructions. -```bash -curl -L https://fly.io/install.sh | sh -flyctl auth login -``` +## 6. Integrate -Deploy: +Integration examples for JavaScript, Python, and other languages: [Examples](../examples/). -```bash -flyctl launch --name email-subject -flyctl secrets set OPENAI_API_KEY=sk-... -flyctl deploy -``` - -**Expected output:** -``` -==> Monitoring deployment -✓ Instance created -✓ Health checks passing - -Visit your deployment at https://email-subject.fly.dev -``` - -### Option B: Docker - -```bash -docker build -t email-subject . -docker run -p 8000:8000 -e OPENAI_API_KEY=sk-... email-subject -``` - -Deploy to AWS ECS, Google Cloud Run, Azure Container Instances, or Kubernetes. - -## 6. Verify Production Deployment - -Test deployed endpoint: - -```bash -curl -X POST https://email-subject.fly.dev/EmailSubjectPredict \ - -H "Content-Type: application/json" \ - -d '{ - "body": "Team, Q4 results are in. Revenue up 23%, NPS at 72.", - "sender": "Sarah (CEO)", - "context": "Company all-hands" - }' -``` +## Development -Retrieve OpenAPI schema: - -```bash -curl https://email-subject.fly.dev/openapi.json -``` - -## 7. Integrate with Applications - -### JavaScript/TypeScript - -```typescript -interface EmailSubjectRequest { - body: string; - sender: string; - context: string; -} - -interface EmailSubjectResponse { - subject: string; - tone: string; - priority: string; -} - -async function generateEmailSubject( - req: EmailSubjectRequest -): Promise { - const response = await fetch( - 'https://email-subject.fly.dev/EmailSubjectPredict', - { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify(req) - } - ); - - return await response.json(); -} - -// Usage -const result = await generateEmailSubject({ - body: "Team meeting notes...", - sender: "manager@company.com", - context: "Internal communication" -}); - -console.log(result.subject); // "Weekly Team Sync: Key Decisions" -``` - -### Python - -```python -import requests - -def generate_email_subject(body: str, sender: str, context: str) -> dict: - response = requests.post( - 'https://email-subject.fly.dev/EmailSubjectPredict', - json={ - 'body': body, - 'sender': sender, - 'context': context - } - ) - return response.json() - -# Usage -result = generate_email_subject( - body="Team meeting notes...", - sender="manager@company.com", - context="Internal communication" -) - -print(result['subject']) # "Weekly Team Sync: Key Decisions" -``` - -### cURL - -```bash -curl -X POST https://email-subject.fly.dev/EmailSubjectPredict \ - -H "Content-Type: application/json" \ - -d '{ - "body": "Team meeting notes...", - "sender": "manager@company.com", - "context": "Internal communication" - }' -``` - -## Module Types - -Select appropriate module type for your use case: - -| Type | Flag | Use Case | -|------|------|----------| -| **Predict** | (default) | Classification, extraction, simple generation | -| **ChainOfThought** | `-m CoT` | Tasks requiring reasoning steps | -| **ReAct** | `-m ReAct` | Tool-using agents with external actions | -| **ProgramOfThought** | `-m PoT` | Code generation, calculations, structured logic | - -Use `dspy-cli generate scaffold` to create programs with specific module types: - -```bash -# Create project first -dspy-cli new document-analyzer - -# Add ChainOfThought module -cd document-analyzer -dspy-cli g scaffold analyzer -m CoT -s "document, criteria -> verdict: bool, reasoning, confidence: float" -``` - -## Project Structure - -``` -email-subject/ -├── src/email_subject/ -│ ├── modules/ # DSPy programs (auto-discovered as endpoints) -│ ├── signatures/ # Input/output schemas -│ ├── optimizers/ # Optimization configurations -│ ├── metrics/ # Evaluation metrics -│ └── utils/ # Helper functions -├── data/ # Training/test datasets -├── logs/ # Request logs -├── tests/ # Test scaffolds -├── dspy.config.yaml # Model configuration -├── .env # API keys (gitignored) -├── Dockerfile # Container configuration -└── pyproject.toml # Dependencies -``` - -## Development Workflow - -### Hot Reload - -```bash -dspy-cli serve --ui --reload -``` - -Changes to `src/email_subject/modules/*.py` reload automatically (< 1s). - -### Switch Models - -Edit `dspy.config.yaml` to change providers without code changes: - -```yaml -models: - default: openai:gpt-4o-mini - registry: - openai:gpt-4o-mini: - model: openai/gpt-4o-mini - env: OPENAI_API_KEY - max_tokens: 16000 - temperature: 1.0 - model_type: chat -``` - -```yaml -# Anthropic -lm: - provider: anthropic - model: claude-3-sonnet-20240229 - max_tokens: 1000 - temperature: 0.7 -``` - -Restart server to apply configuration changes. - -### Custom Port - -```bash -dspy-cli serve --port 3000 -``` +- **Hot reload**: Changes to `.py` files restart server automatically +- **Switch models**: Edit `dspy.config.yaml` - see [Configuration](configuration.md) +- **Custom port**: `dspy-cli serve --port 3000` ## Next Steps -- **[Deployment Guide](deployment.md)** - Production configuration, scaling, monitoring -- **[Configuration Reference](configuration.md)** - Model providers, advanced settings -- **[Command Reference](commands/index.md)** - Complete CLI documentation -- **[Examples](../examples/)** - Working applications (blog-tools, code-review-agent) - -## Troubleshooting - -### Module Not Found - -Activate project virtual environment: - -```bash -cd email-subject -uv sync -source .venv/bin/activate # Unix/macOS -# or -.venv\Scripts\activate # Windows -``` - -Add external dependencies to `pyproject.toml`: - -```bash -uv add requests pandas -uv sync -``` - -### API Key Errors - -Verify `.env` file exists: - -```bash -cat .env # Should show OPENAI_API_KEY=sk-... -``` - -Export to current shell: - -```bash -export OPENAI_API_KEY=sk-... # Unix/macOS -# or -set OPENAI_API_KEY=sk-... # Windows -``` - -### Port In Use - -Use different port: - -```bash -dspy-cli serve --port 8080 -``` - -Identify process using port 8000: - -```bash -lsof -ti:8000 # Unix/macOS -netstat -ano | findstr :8000 # Windows -``` - -### Deployment Failures - -Check Fly.io logs: - -```bash -flyctl logs -``` - -Verify secrets configured: - -```bash -flyctl secrets list # Should show OPENAI_API_KEY -``` - -Validate Docker build: - -```bash -docker build -t email-subject . --progress=plain -``` - -For additional issues, see [Deployment Troubleshooting](deployment.md#troubleshooting). +- **[Deployment Guide](deployment.md)** - Production deployment, scaling, monitoring, troubleshooting +- **[Configuration Reference](configuration.md)** - Model providers, advanced settings, optimization +- **[Command Reference](commands/index.md)** - Complete CLI documentation (`new`, `generate`, `serve`, `optimize`) +- **[Examples](../examples/)** - Working applications (blog-tools, code-review-agent, multi-module patterns) +- **[Project Structure](commands/new.md)** - Understand generated files and conventions +- **[Module Types](commands/generate.md)** - Predict, ChainOfThought, ReAct, ProgramOfThought diff --git a/docs/index.md b/docs/index.md index 53a1652..57852b6 100644 --- a/docs/index.md +++ b/docs/index.md @@ -25,10 +25,10 @@ This overhead blocks small-to-medium AI features from shipping. A DSPy module th ## What dspy-cli Provides -**Project Scaffolding** -- Standardized directory structure with modules, signatures, and configurations -- Auto-generated DSPy signatures from type specifications -- Docker configurations for local development and production deployment +- **Project scaffolding** - Standardized structure with auto-generated DSPy signatures and Docker configs +- **HTTP interface** - FastAPI endpoints with automatic module discovery and OpenAPI docs +- **Development workflow** - Hot-reload server with built-in testing UI +- **Deployment** - Production-ready containers for Fly.io, Render, AWS, and any Docker platform **HTTP Interface** - FastAPI-based REST endpoints with automatic module discovery @@ -108,25 +108,14 @@ dspy-cli serve Access the API at `http://localhost:8000/{ModuleName}` and testing UI at `http://localhost:8000/`. -See the [Getting Started Guide](getting-started.md) for detailed walkthrough. - -## Use Cases - -**Browser Extensions** -HTTP endpoints for summarization, extraction, or classification that extensions call on user-triggered events. - -**Content Management Integration** -Embedded generation or tagging services for platforms like Notion, Confluence, or custom CMSs. - -**Application Microservices** -Standalone intelligence services that web or mobile applications consume via REST APIs. - ## Next Steps - [Getting Started Guide](getting-started.md) - Complete setup and first deployment - [Commands Reference](commands/) - CLI command documentation +- [Deployment Guide](deployment.md) - Production deployment patterns - [Configuration](configuration.md) - Model settings and environment variables -- [Examples](https://github.com/cmpnd-ai/dspy-cli-tool/tree/main/examples) - Sample projects and patterns +- [Use Cases](use-cases/) - Browser extensions, integrations, microservices +- [Examples](../examples/) - Sample projects and patterns ```bash dspy-cli --help # View all commands diff --git a/docs/use-cases/ai-features.md b/docs/use-cases/ai-features.md index 703ff27..8cf894e 100644 --- a/docs/use-cases/ai-features.md +++ b/docs/use-cases/ai-features.md @@ -56,122 +56,23 @@ AI features differ from chat interfaces in that they provide specific functional ## Integration Patterns -### Browser Extension Integration - -Browser extensions call AI feature endpoints from content scripts or background workers: - -```javascript -// Content script calling document analysis endpoint -async function analyzeDocument(content) { - const response = await fetch('https://api.example.com/DocumentAnalyzerPredict', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ document: content }) - }); - return response.json(); -} -``` +AI features are invoked via HTTP POST requests with JSON payloads. Common integration points: -### CMS Plugin Integration - -Content management systems integrate AI features to enhance authoring workflows: - -```javascript -// Notion-style plugin calling emoji suggestion endpoint -async function suggestEmoji(postContent) { - const response = await fetch('https://api.example.com/EmojiPickerPredict', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ post_content: postContent }) - }); - const { emoji } = await response.json(); - return emoji; -} -``` +- **Browser extensions**: Content scripts call endpoints for page analysis +- **CMS plugins**: Authoring tools invoke generation endpoints +- **Email clients**: Background workers call categorization endpoints +- **Background automation**: Async workers process documents via API -### Email Client Integration - -Email applications use AI features for categorization and smart replies: - -```python -# Email categorizer endpoint -class EmailCategorizer(dspy.Module): - def __init__(self): - super().__init__() - self.classify = dspy.ChainOfThought( - "email_content, sender -> category, priority" - ) - - def forward(self, email_content, sender): - return self.classify(email_content=email_content, sender=sender) -``` - -### Background Automation - -Background services invoke AI features for asynchronous processing: - -```python -# Invoice processor for automated document handling -class InvoiceProcessor(dspy.Module): - def __init__(self): - super().__init__() - self.extract = dspy.ChainOfThought( - "invoice_data -> vendor, amount, date, line_items" - ) - - def forward(self, invoice_data): - return self.extract(invoice_data=invoice_data) -``` +See [Examples](../../examples/) for complete integration code. ## Implementation with dspy-cli -dspy-cli provides tooling to create and deploy AI features. A typical workflow: +Create and deploy AI features: ```bash -# Create a new AI feature module -dspy-cli new document-summarizer --program-name Summarizer \ - --signature "document -> summary, key_points" - -# Test locally +dspy-cli new document-summarizer -s "document -> summary, key_points" dspy-cli serve - -# Deploy to production flyctl launch ``` -The resulting endpoint accepts structured input and returns structured output: - -```bash -curl http://localhost:8000/SummarizerPredict \ - -H "Content-Type: application/json" \ - -d '{"document": "Long document text..."}' - -# Returns -{ - "summary": "Brief summary of document", - "key_points": ["Point 1", "Point 2", "Point 3"] -} -``` - -## Deployment Requirements - -AI features require: - -- **HTTP server**: To expose endpoints for application integration -- **Model inference**: Runtime for executing language model operations -- **Authentication**: API keys or tokens for access control -- **Monitoring**: Logging and metrics for production operations -- **Versioning**: Ability to update features without breaking integrations - -## Examples - -- [Email Categorizer](../../examples/email-categorizer/) - Classifies emails by category and priority -- [Document Summarizer](../../examples/doc-summarizer/) - Generates structured summaries from text -- [Smart Form Filler](../../examples/form-filler/) - Completes forms using contextual understanding - -## Learn More - -- [Getting Started Guide](../getting-started.md) - Build your first AI feature -- [Deployment Guide](../deployment.md) - Deploy to production environments -- [Commands: serve](../commands/serve.md) - Local development and testing -- [Examples](../../examples/) - Complete integration examples +See [Getting Started](../getting-started.md) for complete workflow. diff --git a/mkdocs.yml b/mkdocs.yml index 546c0b5..6c244cf 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -27,8 +27,6 @@ theme: features: - navigation.instant # Instant loading - navigation.sections # Section navigation - - navigation.tabs # Top-level tabs - - navigation.tabs.sticky # Keep tabs visible - navigation.indexes # Section index pages - navigation.top # Back to top button - navigation.tracking # URL tracking @@ -96,13 +94,15 @@ markdown_extensions: nav: - Home: index.md - Getting Started: getting-started.md - - Use Cases: - - AI Features: use-cases/ai-features.md - - Deployment: deployment.md - Commands: - - serve: commands/serve.md + - Overview: commands/index.md - new: commands/new.md + - serve: commands/serve.md + - generate: commands/generate.md + - Deployment: deployment.md - Configuration: configuration.md - - Examples: - - Blog Tools: https://github.com/cmpnd-ai/dspy-cli-tool/tree/main/examples/blog-tools - - Code Review Agent: https://github.com/cmpnd-ai/dspy-cli-tool/tree/main/examples/code-review-agent + - Use Cases: + - AI Features: use-cases/ai-features.md + - Reference: + - OpenAPI: OPENAPI.md + - Contributing: contributing.md From 226df62785578b645ca82527509dc98cb943b588 Mon Sep 17 00:00:00 2001 From: Isaac Miller Date: Thu, 13 Nov 2025 21:18:05 -0500 Subject: [PATCH 2/2] remove ui from serve --- docs/commands/serve.md | 7 ------- 1 file changed, 7 deletions(-) diff --git a/docs/commands/serve.md b/docs/commands/serve.md index 17582b7..1299a04 100644 --- a/docs/commands/serve.md +++ b/docs/commands/serve.md @@ -17,9 +17,6 @@ dspy-cli serve # Production configuration dspy-cli serve --no-reload --host 0.0.0.0 --port 8000 -# Development with UI -dspy-cli serve --ui --port 3000 - # Enable MCP tools dspy-cli serve --mcp ``` @@ -35,7 +32,6 @@ Starts an HTTP server for local testing or production deployment. Discovers DSPy | `--port` | `8000` | Port to run server on (1-65535) | | `--host` | `0.0.0.0` | Host to bind to | | `--logs-dir` | `./logs` | Directory for log files | -| `--ui` / `-u` | disabled | Enable web UI for testing | | `--reload` / `--no-reload` | enabled | Auto-reload on file changes | | `--save-openapi` / `--no-save-openapi` | enabled | Save OpenAPI spec to file | | `--openapi-format` | `json` | OpenAPI format (`json` or `yaml`) | @@ -61,9 +57,6 @@ Every deployment provides: | `/programs` | GET | List all discovered modules and their schemas | | `/openapi.json` | GET | OpenAPI specification | -With `--ui` enabled: -- `/` serves interactive testing interface - With `--mcp` enabled: - `/mcp` provides Model Context Protocol server