Tempora

Native High Availability Distributed Task Scheduler for Python and Django

Tempora is a production-ready distributed task scheduler with built-in high availability, Raft consensus, and automatic failover. Unlike Celery or other task queues that require external coordination (Redis, RabbitMQ, ZooKeeper), Tempora provides native clustering with no additional infrastructure.

Key Features

• Raft Consensus Protocol - Battle-tested distributed consensus for leader election and log replication
• Automatic Failover - Sub-second leader election with pre-vote optimization
• Split-Brain Prevention - Fencing tokens and distributed locking prevent data corruption
• Zero External Dependencies - No Redis, no ZooKeeper, no RabbitMQ required for coordination
• Django Integration - Native Django ORM support with PostgreSQL persistence
• Production Hardened - TLS 1.3, HMAC-SHA256 authentication, rate limiting, health monitoring
• Flexible Work Distribution - Round-robin, least-loaded, and affinity-based strategies

Why Tempora?

Feature	Tempora	Celery	APScheduler	Django-Q
Native HA	Yes	No (requires Redis Sentinel)	No	No
Raft Consensus	Yes	No	No	No
Zero External Deps	Yes	No	Yes	No
Automatic Failover	Yes	Partial	No	No
Split-Brain Safe	Yes	No	No	No
Django Native	Yes	Partial	Partial	Yes

Installation

pip install tempora-scheduler

Requirements

• Python 3.11+
• Django 4.2+
• PostgreSQL 14+ (for persistence)

Quick Start

1. Add to Django Settings

INSTALLED_APPS = [
    # ...
    'tempora',
]

# Tempora Configuration
TEMPORA_CLUSTER_SECRET = "your-32-byte-cluster-secret-key"
TEMPORA_NODE_ID = "node-1"
TEMPORA_CLUSTER_NODES = [
    {"id": "node-1", "host": "10.0.0.1", "port": 7000},
    {"id": "node-2", "host": "10.0.0.2", "port": 7000},
    {"id": "node-3", "host": "10.0.0.3", "port": 7000},
]

2. Run Migrations

python manage.py migrate tempora

3. Start the Scheduler

from tempora import TemporaCluster

cluster = TemporaCluster(
    node_id="node-1",
    cluster_secret="your-32-byte-cluster-secret-key",
    peers=[
        ("node-2", "10.0.0.2", 7000),
        ("node-3", "10.0.0.3", 7000),
    ]
)

await cluster.start()

4. Schedule Tasks

from tempora import schedule_task
from datetime import timedelta

# One-time task
schedule_task(
    name="send_welcome_email",
    func="myapp.tasks.send_email",
    args={"user_id": 123},
    run_at=timezone.now() + timedelta(hours=1)
)

# Recurring task
schedule_task(
    name="daily_cleanup",
    func="myapp.tasks.cleanup",
    cron="0 2 * * *"  # Every day at 2 AM
)

Architecture

Tempora uses a 3-tier architecture for distributed scheduling:

┌─────────────────────────────────────────────────────────────┐
│                     Coordination Layer                       │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐          │
│  │   Node 1    │  │   Node 2    │  │   Node 3    │          │
│  │  (Leader)   │◄─┤ (Follower)  │──┤ (Follower)  │          │
│  └─────────────┘  └─────────────┘  └─────────────┘          │
│         │              Raft Consensus Protocol               │
└─────────┼───────────────────────────────────────────────────┘
          │
┌─────────▼───────────────────────────────────────────────────┐
│                    Replication Layer                         │
│         Log Replication • Commit Tracking • Snapshots        │
└─────────────────────────────────────────────────────────────┘
          │
┌─────────▼───────────────────────────────────────────────────┐
│                    Persistence Layer                         │
│              PostgreSQL • Django ORM • Migrations            │
└─────────────────────────────────────────────────────────────┘

Configuration Reference

Environment Variables

Variable	Description	Default
TEMPORA_SECRET_KEY	Django secret key	Required
TEMPORA_CLUSTER_SECRET	Cluster authentication secret (32+ bytes)	Required
TEMPORA_NODE_ID	Unique node identifier	Required
TEMPORA_DB_HOST	PostgreSQL host	localhost
TEMPORA_DB_PORT	PostgreSQL port	5432
TEMPORA_DB_NAME	Database name	tempora
TEMPORA_DB_USER	Database user	tempora
TEMPORA_DB_PASSWORD	Database password	Required
TEMPORA_LOG_LEVEL	Logging level	INFO

Cluster Settings

TEMPORA_SETTINGS = {
    # Raft timing
    "election_timeout_min": 150,  # ms
    "election_timeout_max": 300,  # ms
    "heartbeat_interval": 50,     # ms

    # Replication
    "max_entries_per_append": 100,
    "snapshot_threshold": 10000,

    # Security
    "tls_enabled": True,
    "tls_cert_file": "/path/to/cert.pem",
    "tls_key_file": "/path/to/key.pem",

    # Rate limiting
    "max_requests_per_second": 1000,
    "connection_pool_size": 100,
}

Production Deployment

Minimum Cluster Size

For high availability, deploy 3 or 5 nodes (odd numbers prevent split-brain):

• 3 nodes: Tolerates 1 failure
• 5 nodes: Tolerates 2 failures

Health Monitoring

Tempora exposes health endpoints:

from tempora import get_cluster_health

health = await get_cluster_health()
# {
#     "status": "healthy",
#     "leader": "node-1",
#     "term": 42,
#     "commit_index": 15678,
#     "nodes": {
#         "node-1": {"status": "leader", "last_heartbeat": "2024-01-15T10:30:00Z"},
#         "node-2": {"status": "follower", "last_heartbeat": "2024-01-15T10:30:00Z"},
#         "node-3": {"status": "follower", "last_heartbeat": "2024-01-15T10:30:00Z"},
#     }
# }

Graceful Shutdown

# Trigger leadership transfer before shutdown
await cluster.transfer_leadership()
await cluster.stop()

Comparison with Alternatives

vs Celery

Celery requires Redis or RabbitMQ for coordination and doesn't provide native high availability. Tempora eliminates external dependencies and provides built-in Raft consensus for automatic failover.

vs APScheduler

APScheduler is single-node only. Tempora provides distributed scheduling across multiple nodes with automatic leader election.

vs Django-Q

Django-Q requires Redis or Django ORM polling. Tempora uses native TCP coordination with Raft consensus for true distributed scheduling.

Use Cases

• Distributed Cron Jobs - Run scheduled tasks across a cluster with guaranteed execution
• Task Queues - Process background jobs with automatic failover
• Workflow Orchestration - Coordinate multi-step workflows across services
• Event Processing - Handle events with at-least-once delivery guarantees
• Batch Processing - Distribute large batch jobs across worker nodes

API Reference

Core Classes

• TemporaCluster - Main cluster coordinator
• RaftNode - Raft consensus node
• LogReplicator - Log replication manager
• WorkDistributor - Task distribution strategies

Management Commands

# Check cluster status
python manage.py tempora_status

# Trigger leader election
python manage.py tempora_election

# View replication lag
python manage.py tempora_replication

Performance

Benchmarks on 3-node cluster (AWS c5.xlarge):

Metric	Value
Leader election	< 300ms
Task scheduling	10,000+ tasks/sec
Replication lag	< 10ms
Failover time	< 500ms

License

Tempora is dual-licensed:

• MIT License - Free for open source projects
• Commercial License - For proprietary applications

See LICENSE for details.

Contributing

We welcome contributions! See CONTRIBUTING.md for guidelines.

Support

• Documentation: https://tempora.readthedocs.io
• Issues: GitHub Issues
• Discussions: GitHub Discussions

---

Keywords: distributed task scheduler, python task queue, django scheduler, high availability scheduler, raft consensus python, distributed cron, celery alternative, background job processing, task orchestration, distributed systems python, fault-tolerant scheduler, leader election python, distributed consensus, job scheduler django, async task queue