Distributed Multi-Level Cache Module

A comprehensive distributed caching solution that implements a multi-level cache architecture with consistency mechanisms, communication protocols, and monitoring capabilities.

Features

• Multi-level caching: L1 (local), L2 (distributed), L3 (persistent storage)
• Cache consistency: Event-driven invalidation and synchronization
• Communication layer: Support for various messaging systems (Redis Pub/Sub, RabbitMQ)
• Monitoring: Built-in metrics and health checks
• Fault tolerance: Graceful degradation and fallback mechanisms
• High performance: Optimized for low latency and high throughput

Installation

pip install git+https://github.com/your-repo/dist-cache-module.git

Quick Start

from dist_cache.cache.manager import DistributedCacheManager

# Initialize the cache manager
cache_manager = DistributedCacheManager(
    l1_max_size=1000,
    l1_ttl=300,  # 5 minutes
    l2_ttl=1800,  # 30 minutes
    redis_url="redis://localhost:6379"
)

# Get a value from cache
value = await cache_manager.get("my_key")

# Set a value in cache
await cache_manager.set("my_key", "my_value")

# Refresh a specific key
await cache_manager.refresh("my_key")

Architecture

The module implements a three-level cache hierarchy:

┌─────────────────────────────────────────────────────────────┐
│                   Application Layer                         │
└──────────────────────┬──────────────────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────────────────┐
│               DistributedCacheManager                       │
│  - Unified cache interface                                  │
│  - Multi-level cache orchestration                        │
│  - Consistency management                                 │
│  - Monitoring and metrics                                 │
└──────────────────────┬──────────────────────────────────────┘
                       │
        ┌──────────────┼──────────────┐
        │              │              │
        ▼              ▼              ▼
   ┌─────────┐   ┌─────────┐   ┌─────────┐
   │ L1缓存  │   │ L2缓存  │   │ L3数据源│
   │ (本地)  │   │ (Redis) │   │  (API)  │
   └─────────┘   └─────────┘   └─────────┘
        │              │              │
        │              │              │
   TTL: 5分钟    TTL: 30分钟    持久化存储
   容量: 1000条  容量: 无限制   数据源: 外部服务

Configuration

The module supports extensive configuration options:

• Cache sizes and TTLs
• Communication protocols
• Consistency strategies
• Monitoring settings
• Fallback behaviors

Contributing

Contributions are welcome! Please see the contributing guidelines for more information.

License

MIT License