Skip to content

OPENAPI_SETUP.md

Latest commit

 

History

History
559 lines (460 loc) · 12.7 KB

OPENAPI_SETUP.md

File metadata and controls

559 lines (460 loc) · 12.7 KB

DiffuGen OpenAPI Integration Guide

This guide provides detailed instructions for setting up, configuring, and using DiffuGen's OpenAPI server capabilities, enabling integration with OpenWebUI and other OpenAPI-compatible tools.

Prerequisites

  • Python 3.11 or newer
  • NVIDIA GPU with CUDA support (recommended)
  • DiffuGen base installation completed

Installation

  1. Install DiffuGen and its dependencies:
git clone https://github.com/CLOUDWERX-DEV/diffugen.git
cd DiffuGen
chmod +x setup_diffugen.sh
./setup_diffugen.sh
  1. Start the OpenAPI server:
python diffugen_openapi.py --port 5199

The server will be available at http://0.0.0.0:5199 with interactive documentation at http://0.0.0.0:5199/docs

Configuration

Configuration File Location

The OpenAPI server looks for configuration in the following locations:

  1. openapi_config.json in the DiffuGen root directory
  2. Custom config file path specified with --config command line option
  3. Environment variables for specific settings

Configuration Structure

The openapi_config.json file has the following structure:

{
  "server": {
    "host": "0.0.0.0",
    "port": 5199,
    "debug": false
  },
  "paths": {
    "sd_cpp_path": "stable-diffusion.cpp",
    "models_dir": "stable-diffusion.cpp/models",
    "output_dir": "outputs"
  },
  "hardware": {
    "vram_usage": "adaptive",
    "gpu_layers": -1,
    "cuda_device": "0"
  },
  "env": {
    "CUDA_VISIBLE_DEVICES": "0"
  },
  "cors": {
    "allow_origins": ["*"],
    "allow_methods": ["GET", "POST", "OPTIONS"],
    "allow_headers": ["*"]
  },
  "rate_limiting": {
    "rate": "60/minute",
    "enabled": true
  },
  "models": {
    "flux": ["flux-schnell", "flux-dev"],
    "stable_diffusion": ["sdxl", "sd3", "sd15"]
  },
  "default_model": "flux-schnell",
  "default_params": {
    "width": 512,
    "height": 512,
    "steps": {
      "flux-schnell": 8,
      "flux-dev": 20,
      "sdxl": 20,
      "sd3": 20,
      "sd15": 20
    },
    "cfg_scale": {
      "flux-schnell": 1.0,
      "flux-dev": 1.0,
      "sdxl": 7.0,
      "sd3": 7.0,
      "sd15": 7.0
    },
    "sampling_method": {
      "flux-schnell": "euler",
      "flux-dev": "euler",
      "sdxl": "euler",
      "sd3": "euler",
      "sd15": "euler"
    }
  },
  "images": {
    "serve_path": "/images",
    "cache_control": "max-age=3600"
  },
  "security": {
    "api_key_required": false,
    "api_keys": []
  }
}

Configuration Sections

Server Configuration

"server": {
  "host": "0.0.0.0",
  "port": 5199,
  "debug": false
}
  • host: The IP address the server will bind to (default: "0.0.0.0" to listen on all interfaces)
  • port: The port the server will listen on (default: 5199)
  • debug: Whether to run in debug mode (default: false)

Path Configuration

"paths": {
  "sd_cpp_path": "stable-diffusion.cpp",
  "models_dir": "stable-diffusion.cpp/models",
  "output_dir": "outputs"
}
  • sd_cpp_path: Path to the stable-diffusion.cpp directory
  • models_dir: Path to the directory containing model files (if not set, defaults to {sd_cpp_path}/models)
  • output_dir: Directory where generated images will be saved

Hardware Configuration

"hardware": {
  "vram_usage": "adaptive",
  "gpu_layers": -1,
  "cuda_device": "0"
}
  • vram_usage: VRAM usage strategy (options: "adaptive", "minimal", "balanced", "maximum")
  • gpu_layers: Number of layers to offload to GPU (-1 for auto-detect)
  • cuda_device: CUDA device to use for generation

Environment Variables

"env": {
  "CUDA_VISIBLE_DEVICES": "0"
}
  • CUDA_VISIBLE_DEVICES: Controls which GPUs are visible to the application
    • "0": Use only the first GPU
    • "1": Use only the second GPU
    • "0,1": Use both first and second GPUs
    • "-1": Disable CUDA and use CPU only

CORS Configuration

"cors": {
  "allow_origins": ["*"],
  "allow_methods": ["GET", "POST", "OPTIONS"],
  "allow_headers": ["*"]
}
  • allow_origins: List of allowed origins (default: ["*"] to allow all origins)
  • allow_methods: List of allowed HTTP methods (default: ["GET", "POST", "OPTIONS"])
  • allow_headers: List of allowed HTTP headers (default: ["*"] to allow all headers)

Rate Limiting

"rate_limiting": {
  "rate": "60/minute",
  "enabled": true
}
  • rate: Maximum request rate in format number/timeunit (default: "60/minute")
  • enabled: Whether rate limiting is enabled (default: true)

Models Configuration

"models": {
  "flux": ["flux-schnell", "flux-dev"],
  "stable_diffusion": ["sdxl", "sd3", "sd15"]
}
  • flux: List of available Flux models
  • stable_diffusion: List of available Stable Diffusion models

Default Model

"default_model": "flux-schnell"
  • Specifies which model to use when no model is explicitly requested

Default Parameters

"default_params": {
  "width": 512,
  "height": 512,
  "steps": {
    "flux-schnell": 8,
    "flux-dev": 20,
    "sdxl": 20,
    "sd3": 20,
    "sd15": 20
  },
  "cfg_scale": {
    "flux-schnell": 1.0,
    "flux-dev": 1.0,
    "sdxl": 7.0,
    "sd3": 7.0,
    "sd15": 7.0
  },
  "sampling_method": {
    "flux-schnell": "euler",
    "flux-dev": "euler",
    "sdxl": "euler",
    "sd3": "euler",
    "sd15": "euler"
  }
}

This section defines default parameters for generation:

  • width: Default image width in pixels
  • height: Default image height in pixels
  • steps: Default number of diffusion steps for each model
  • cfg_scale: Default classifier-free guidance scale for each model
  • sampling_method: Default sampling method for each model

Images Configuration

"images": {
  "serve_path": "/images",
  "cache_control": "max-age=3600"
}
  • serve_path: URL path where images will be served (default: "/images")
  • cache_control: Cache-Control header for served images (default: "max-age=3600")

Security Configuration

"security": {
  "api_key_required": false,
  "api_keys": []
}
  • api_key_required: Whether API key authentication is required (default: false)
  • api_keys: List of valid API keys for authentication (default: [])

Environment Variable Overrides

You can override configuration settings with environment variables:

  • DIFFUGEN_OPENAPI_PORT: Override the server port
  • SD_CPP_PATH: Path to the stable-diffusion.cpp directory
  • DIFFUGEN_OUTPUT_DIR: Directory where generated images will be saved
  • DIFFUGEN_CORS_ORIGINS: Comma-separated list of allowed origins
  • DIFFUGEN_RATE_LIMIT: Rate limit in format number/timeunit
  • CUDA_VISIBLE_DEVICES: Control which GPUs are used
  • VRAM_USAGE: VRAM usage strategy
  • GPU_LAYERS: Number of layers to offload to GPU

Command Line Options

You can also specify some configuration options via command line:

python diffugen_openapi.py --host 127.0.0.1 --port 8080 --config custom_config.json
  • --host: Host address to bind to
  • --port: Port to listen on
  • --config: Path to a custom configuration file

OpenWebUI Integration

  1. Open OpenWebUI Settings (gear icon)
  2. Navigate to the "Tools" section
  3. Click the "+" button to add a new tool server
  4. Enter the following details:
  5. Click "Save"

Once added, DiffuGen will appear in the available tools list when clicking the tools icon in the chat interface. The following endpoints will be available:

  • generate_stable_image_generate_stable_post: Generate with Stable Diffusion
  • generate_flux_image_endpoint_generate_flux_post: Generate with Flux Models
  • list_models_models_get: List Available Models

API Endpoints Reference

1. Generate Image with Stable Diffusion Models

POST /generate/stable

Request body:

{
  "prompt": "A beautiful sunset over mountains",
  "model": "sdxl",
  "width": 1024,
  "height": 768,
  "steps": 30,
  "cfg_scale": 7.0,
  "seed": -1,
  "sampling_method": "dpm++2m",
  "negative_prompt": "blurry, low quality"
}

Response:

{
  "file_path": "path/to/generated/image.png",
  "metadata": {
    "model": "sdxl",
    "prompt": "A beautiful sunset over mountains",
    "generation_time": "1.23s",
    "parameters": {
      // Generation parameters used
    }
  }
}

2. Generate Image with Flux Models

POST /generate/flux

Request body:

{
  "prompt": "A cyberpunk cityscape",
  "model": "flux-schnell",
  "width": 512,
  "height": 512,
  "steps": 8,
  "cfg_scale": 1.0,
  "seed": -1,
  "sampling_method": "euler"
}

Response: Same structure as Stable Diffusion endpoint

3. List Available Models

GET /models

Response:

{
  "models": {
    "flux": ["flux-schnell", "flux-dev"],
    "stable_diffusion": ["sdxl", "sd3", "sd15"],
    "default_parameters": {
      // Model-specific default parameters
    }
  }
}

Advanced Configuration Examples

Basic Configuration

{
  "server": {
    "host": "127.0.0.1",
    "port": 8080
  },
  "paths": {
    "output_dir": "/var/diffugen/images"
  },
  "default_model": "flux-schnell"
}

High-Performance Configuration

{
  "hardware": {
    "vram_usage": "maximum",
    "gpu_layers": -1,
    "cuda_device": "0"
  },
  "env": {
    "CUDA_VISIBLE_DEVICES": "0,1"
  },
  "default_params": {
    "steps": {
      "flux-schnell": 12,
      "sdxl": 30
    }
  }
}

Production Configuration with Security

{
  "server": {
    "host": "0.0.0.0",
    "port": 5199,
    "debug": false
  },
  "cors": {
    "allow_origins": ["https://example.com", "https://api.example.com"],
    "allow_methods": ["GET", "POST"]
  },
  "rate_limiting": {
    "rate": "30/minute",
    "enabled": true
  },
  "security": {
    "api_key_required": true,
    "api_keys": ["your-secret-api-key-1", "your-secret-api-key-2"]
  }
}

Model-Specific Recommendations

Flux Models

  • flux-schnell: Best for rapid prototyping

    • Default steps: 8
    • Default cfg_scale: 1.0
    • Sampling method: euler
    • Best for: Quick iterations, concept testing

  • flux-dev: Better quality, slower generation

    • Default steps: 20
    • Default cfg_scale: 1.0
    • Sampling method: euler
    • Best for: Higher quality generations

Stable Diffusion Models

  • sdxl: Highest quality, largest model

    • Default steps: 20-30
    • Default cfg_scale: 7.0
    • Sampling method: dpm++2m
    • Best for: Professional quality images

  • sd3: Good balance of quality and speed

    • Default steps: 20
    • Default cfg_scale: 7.0
    • Sampling method: euler_a
    • Best for: General purpose generation

  • sd15: Classic model, reliable results

    • Default steps: 20
    • Default cfg_scale: 7.0
    • Sampling method: euler_a
    • Best for: Consistent, well-understood results

Error Handling

The API uses standard HTTP status codes:

  • 200: Successful generation
  • 400: Invalid request parameters
  • 404: Model not found
  • 500: Server error

Error responses include detailed messages:

{
  "error": "Invalid parameters",
  "detail": "Width must be between 256 and 2048",
  "code": "INVALID_DIMENSIONS"
}

Output Directory Configuration

The server uses the following directory configuration:

  • Default output directory: /output
  • Environment variable: DIFFUGEN_OUTPUT_DIR
  • Fallback: Creates an output directory in the current working directory if the specified directory is not accessible

Generated images are saved to this directory and served through the /images endpoint. The full URL for accessing generated images will be: http://your-server:port/images/filename.png

CORS Configuration

By default, CORS is enabled for OpenWebUI integration. You can configure CORS settings:

# In your environment or configuration
DIFFUGEN_CORS_ORIGINS="http://localhost:3000,http://localhost:8080"
DIFFUGEN_CORS_METHODS="GET,POST"

Troubleshooting

  1. Server won't start

    • Check if required models are downloaded
    • Verify CUDA installation if using GPU
    • Check port 5199 is available
    • Verify Python environment is activated

  2. Generation fails

    • Check model paths in configuration
    • Verify GPU memory availability
    • Try reducing image dimensions or steps
    • Check server logs for detailed error messages

  3. Poor image quality

    • Increase steps (20-30 for better quality)
    • Adjust cfg_scale (7.0-9.0 for SD models)
    • Use more detailed prompts
    • Try different sampling methods

  4. OpenWebUI Integration Issues

    • Verify server is running and accessible
    • Check CORS settings if using a different domain
    • Ensure URL is correctly formatted in tool settings
    • Check browser console for error messages

Support

For issues and questions: