Skip to content

Latest commit

 

History

History
195 lines (163 loc) · 5.91 KB

COMPONENTS.md

File metadata and controls

195 lines (163 loc) · 5.91 KB

PyAirtable Python Shared Library - Components Overview

Core Packages Implemented

1. Configuration (pyairtable_shared.config)

  • Settings class with nested configurations using pydantic-settings
  • Environment-based configuration with .env support
  • Validation and type safety for all configuration values
  • Safe configuration dumping (excludes sensitive data)
  • Sub-configurations:
    • DatabaseSettings (async PostgreSQL with connection pooling)
    • CacheSettings (Redis with connection pooling)
    • SecuritySettings (JWT, encryption, passwords)
    • LoggingSettings (structured logging with structlog)
    • MetricsSettings (Prometheus metrics)
    • RateLimitingSettings (configurable rate limits)
    • KafkaSettings (event streaming)
    • ServiceDiscoverySettings (Go service URLs)
    • HealthCheckSettings (health monitoring)

2. Database (pyairtable_shared.database)

  • Async SQLAlchemy 2.0 setup with asyncpg driver
  • Connection pooling and health checks
  • Base model class with common functionality
  • Model mixins:
    • UUIDPrimaryKeyMixin (UUID primary keys)
    • TimestampMixin (created_at, updated_at)
    • SoftDeleteMixin (soft delete functionality)
    • TenantMixin (multi-tenant support)
    • UserTrackingMixin (audit trails)
    • SlugMixin (URL-friendly slugs)
    • VersionMixin (optimistic locking)
    • MetadataMixin (JSON metadata fields)
  • BaseRepository with full CRUD operations
  • Database lifecycle management
  • Migration support with Alembic

3. Cache (pyairtable_shared.cache)

  • Async Redis client with connection pooling
  • CacheManager with JSON serialization
  • Cache decorators for function result caching
  • CacheKeyBuilder for consistent key generation
  • Support for:
    • Basic get/set operations
    • Multi get/set operations
    • Sets and counters
    • TTL management
    • Pattern-based cache invalidation
    • User/tenant/workspace scoped keys
    • Rate limiting keys
    • Distributed locks

4. Error Handling (pyairtable_shared.errors)

  • Standardized exception hierarchy:
    • PyAirtableError (base exception)
    • ValidationError (422 status)
    • NotFoundError (404 status)
    • PermissionError (403 status)
    • AuthenticationError (401 status)
    • RateLimitError (429 status)
    • ServiceUnavailableError (503 status)
    • DatabaseError, CacheError, ExternalServiceError
    • BusinessLogicError, ConflictError, TimeoutError
  • Rich error context with details
  • Structured error responses
  • HTTP status code mapping

5. Logging (pyairtable_shared.logger)

  • Structured logging with structlog
  • JSON and console output formats
  • Automatic sensitive data masking
  • Context-aware logging with:
    • Service information
    • Request tracing
    • User/tenant context
  • Configurable log levels
  • Integration with FastAPI middleware

6. Models (pyairtable_shared.models) - Basic Structure

  • BaseModel with common functionality
  • Foundation for User, Tenant, Workspace models
  • Model serialization and validation
  • Integration with database mixins

7. Testing Utilities (pyairtable_shared.testing)

  • AsyncTestCase for async test support
  • Mock helpers and context managers
  • Database testing utilities
  • Cache testing utilities
  • Factory pattern support
  • Test fixtures

Configuration Files

Development and Build

  • pyproject.toml: Modern Python packaging with all dependencies
  • Makefile: Comprehensive development commands
  • .env.example: Complete environment variable template
  • .gitignore: Python-specific ignore patterns
  • LICENSE: MIT license

Code Quality

  • Ruff configuration: Modern Python linting and formatting
  • MyPy configuration: Strict type checking
  • Pytest configuration: Test runners with coverage
  • Pre-commit hooks: Automated code quality checks

Examples

1. Basic Usage (examples/basic_usage.py)

  • Configuration loading
  • Database operations
  • Cache operations
  • Error handling
  • Application lifecycle management

2. FastAPI Integration (examples/fastapi_integration.py)

  • Complete FastAPI application setup
  • Database and cache dependencies
  • Error handling middleware
  • Health check endpoints
  • Structured logging integration
  • Application lifespan management

Architecture Features

Async-First Design

  • All I/O operations are async/await compatible
  • Connection pooling for database and cache
  • Proper resource cleanup and lifecycle management

Type Safety

  • Comprehensive type hints throughout
  • Pydantic models for validation
  • MyPy strict mode compliance

Observability

  • Structured logging with context
  • Health check endpoints
  • Metrics collection ready
  • Error tracking and reporting

Multi-Tenancy Support

  • Tenant-scoped models and caching
  • User tracking and audit trails
  • Permission-aware error handling

Testing Support

  • Async test utilities
  • Mock generators and factories
  • Database test setup/teardown
  • Cache testing helpers

Installation and Usage

# Install package
pip install -e .

# Install with development dependencies
pip install -e .[test,dev]

# Run tests
make test

# Format and lint code
make format lint

# Type checking
make type-check

# Build package
make build

Integration with Microservices

This shared library provides the foundation for Python microservices in the PyAirtable ecosystem:

  1. Consistent configuration across all services
  2. Standardized database patterns with async support
  3. Unified caching strategy with Redis
  4. Common error handling and response formats
  5. Structured logging for observability
  6. Testing utilities for reliable tests
  7. FastAPI integration patterns

Services can extend this library by:

  • Adding service-specific models
  • Implementing business logic repositories
  • Creating custom middleware
  • Adding service-specific error types
  • Extending the configuration with service settings

The library ensures consistency, reduces boilerplate, and provides production-ready components for building scalable Python microservices.