KnowledgeProbe

A production-grade microservice quiz platform that probes your knowledge in Django, Microservices, Redis, RabbitMQ, Docker, Kubernetes, and Terraform.

Why this project? Most microservice tutorials stop at "two services talking over HTTP." This project implements the patterns you'd actually encounter in production: event-driven communication via RabbitMQ, database-per-service isolation, Redis-backed leaderboards with sorted sets, Celery task queues, Kubernetes orchestration with HPAs, and Terraform-managed AWS infrastructure — all wired together end-to-end.

---

Architecture

graph TB
    Client([Client]) --> GW["API Gateway — Nginx :80<br/>Rate Limiting · Routing · TLS"]

    GW --> QS["Quiz Service :8001<br/>Django + DRF"]
    GW --> US["User Service :8002<br/>Django + DRF"]
    GW --> ES["Evaluation Service :8003<br/>Django + DRF"]

    QS <-.->|Events| RMQ{{"RabbitMQ<br/>Topic Exchange"}}
    US <-.->|Events| RMQ
    ES <-.->|Events| RMQ

    QS --- QDB[("quiz_db<br/>PostgreSQL")]
    US --- UDB[("user_db<br/>PostgreSQL")]
    ES --- EDB[("eval_db<br/>PostgreSQL")]

    QS -.-> Redis[("Redis<br/>Cache · Leaderboards<br/>Rate Limiting")]
    ES -.-> Redis

    style GW fill:#2d3748,color:#fff
    style RMQ fill:#ff6600,color:#fff
    style Redis fill:#dc382d,color:#fff
    style QDB fill:#4169e1,color:#fff
    style UDB fill:#4169e1,color:#fff
    style EDB fill:#4169e1,color:#fff

Loading

Services

Service	Port	Purpose
Quiz Service	8001	Quiz/question CRUD, category management, quiz submission
User Service	8002	Registration, JWT authentication, user profiles
Evaluation Service	8003	Scoring, analytics, Redis-powered leaderboards
API Gateway	80	Nginx reverse proxy with rate limiting

Event-Driven Communication

Services communicate asynchronously through RabbitMQ using a topic exchange (knowledge_probe):

Event	Publisher	Consumer	Action
user.registered	User Service	Evaluation Service	Create user stats record
quiz.completed	Quiz Service	Evaluation Service	Score attempt, update leaderboard
score.calculated	Evaluation Service	User Service	Update user profile stats
leaderboard.updated	Evaluation Service	Quiz Service	Invalidate caches

---

Tech Stack

Layer	Technology
Backend	Python 3.11, Django 5.0, Django REST Framework
Auth	JWT (djangorestframework-simplejwt)
Database	PostgreSQL 15 (database-per-service)
Cache	Redis 7 (django-redis)
Message Broker	RabbitMQ 3.12 (pika)
Task Queue	Celery 5.3
API Gateway	Nginx 1.25
Containerization	Docker + Docker Compose
Orchestration	Kubernetes (Deployments, HPAs, Ingress)
Infrastructure	Terraform (AWS: EKS, RDS, ElastiCache, Amazon MQ)

---

Design Decisions

Decision	Alternatives Considered	Why This Approach
RabbitMQ for inter-service messaging	REST calls, gRPC, Kafka	Decouples services temporally — a service being down doesn't break others. Topic exchange enables flexible routing without point-to-point coupling. Kafka would be overkill for this event volume.
Database-per-service (3 PostgreSQL databases)	Shared database, schema-per-service	True data isolation — services can't accidentally couple through shared tables. Mirrors real production microservice boundaries and allows independent schema evolution.
Redis sorted sets for leaderboards	SQL ORDER BY queries, application-level sorting	O(log N) insert + O(log N + M) range queries vs O(N log N) for SQL. Leaderboard reads vastly outnumber writes — Redis gives sub-millisecond response times at scale.
JWT over session-based auth	Server-side sessions, OAuth2, API keys	Stateless authentication works across services without a shared session store. Each service independently validates tokens using the shared secret — no inter-service call needed for auth.
Celery for async task processing	Django signals, threading, Dramatiq	Production-grade task queue with retry logic, rate limiting, and monitoring. RabbitMQ broker already in the stack, so no additional infrastructure required.
Terraform modules per resource	Monolithic Terraform, CloudFormation, Pulumi	Module-per-resource enables reuse across environments. dev/ and prod/ use the same modules with different variable values, avoiding config drift.
Nginx API Gateway	Kong, Traefik, AWS ALB	Lightweight, battle-tested, and sufficient for path-based routing + rate limiting. No need for a full-featured API gateway at this scale.

---

Quick Start

Prerequisites

• Docker and Docker Compose
• Git

One-Command Setup

./scripts/setup.sh

This will:

1. Create .env with generated secrets
2. Build all Docker images
3. Start PostgreSQL, Redis, and RabbitMQ
4. Run database migrations
5. Seed 50+ quiz questions across 7 topics
6. Start all services, workers, and consumers
7. Run health checks

Manual Setup

# 1. Clone and configure
cp .env.example .env
# Edit .env with your own secrets

# 2. Build and start everything
docker-compose up --build -d

# 3. Run migrations (each service runs automatically, or manually)
docker-compose exec quiz_service python manage.py migrate
docker-compose exec user_service python manage.py migrate
docker-compose exec evaluation_service python manage.py migrate

# 4. Seed quiz questions
docker-compose exec quiz_service python manage.py seed_questions

# 5. Verify
curl http://localhost/health

---

API Usage

1. Register a User

curl -X POST http://localhost/api/users/auth/register/ \
  -H "Content-Type: application/json" \
  -d '{
    "username": "john",
    "email": "john@example.com",
    "password": "SecurePass123",
    "password_confirm": "SecurePass123"
  }'

2. Login (Get JWT Token)

curl -X POST http://localhost/api/users/auth/login/ \
  -H "Content-Type: application/json" \
  -d '{
    "username": "john",
    "password": "SecurePass123"
  }'
# Returns: { "access": "eyJ...", "refresh": "eyJ..." }

3. Browse Quiz Categories

curl http://localhost/api/quizzes/categories/

4. Browse Quizzes

# All quizzes
curl http://localhost/api/quizzes/quizzes/

# Filter by category
curl http://localhost/api/quizzes/quizzes/?category=django

# Filter by difficulty
curl http://localhost/api/quizzes/quizzes/?difficulty=advanced

5. Get Quiz Detail (with Questions)

curl http://localhost/api/quizzes/quizzes/1/

6. Submit Quiz Answers

curl -X POST http://localhost/api/quizzes/quizzes/1/submit/ \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <your-access-token>" \
  -d '{
    "answers": [
      {"question_id": 1, "choice_id": 3},
      {"question_id": 2, "choice_id": 7},
      {"question_id": 3, "choice_id": 10}
    ]
  }'

7. View Your Stats

curl http://localhost/api/evaluations/stats/ \
  -H "Authorization: Bearer <your-access-token>"

8. View Leaderboard

# Global leaderboard
curl http://localhost/api/evaluations/leaderboard/

# Category leaderboard
curl http://localhost/api/evaluations/leaderboard/?category=django

9. View Your Quiz Attempts

curl http://localhost/api/evaluations/attempts/ \
  -H "Authorization: Bearer <your-access-token>"

---

API Endpoints Reference

Quiz Service (/api/quizzes/)

Method	Endpoint	Auth	Description
GET	/api/quizzes/categories/	No	List all categories
GET	/api/quizzes/categories/{id}/	No	Category detail
GET	/api/quizzes/quizzes/	No	List quizzes (filterable)
GET	/api/quizzes/quizzes/{id}/	No	Quiz with questions
POST	/api/quizzes/quizzes/{id}/submit/	Yes	Submit answers
GET	/api/quizzes/health/	No	Health check

User Service (/api/users/)

Method	Endpoint	Auth	Description
POST	/api/users/auth/register/	No	Register
POST	/api/users/auth/login/	No	Login (JWT)
POST	/api/users/auth/token/refresh/	No	Refresh token
GET	/api/users/profile/	Yes	Get profile
PUT/PATCH	/api/users/profile/	Yes	Update profile
GET	/api/users/stats/	Yes	User statistics
GET	/api/users/health/	No	Health check

Evaluation Service (/api/evaluations/)

Method	Endpoint	Auth	Description
GET	/api/evaluations/attempts/	Yes	List attempts
GET	/api/evaluations/attempts/{id}/	Yes	Attempt detail
GET	/api/evaluations/leaderboard/	No	Leaderboard
GET	/api/evaluations/stats/	Yes	User stats
GET	/api/evaluations/stats/categories/	Yes	Per-category stats
GET	/api/evaluations/health/	No	Health check

---

Project Structure

.
├── README.md
├── docker-compose.yml           # Local development orchestration
├── .env.example                 # Environment variable template
│
├── services/
│   ├── quiz_service/            # Quiz & question management
│   │   ├── Dockerfile
│   │   ├── config/              # Django settings, URLs, Celery
│   │   └── quizzes/             # Models, views, serializers, tests
│   │       └── management/commands/
│   │           ├── seed_questions.py    # 50+ quiz questions
│   │           └── consume_events.py   # RabbitMQ consumer
│   │
│   ├── user_service/            # Authentication & profiles
│   │   ├── Dockerfile
│   │   ├── config/
│   │   └── accounts/            # Models, JWT auth, views, tests
│   │       └── management/commands/
│   │           └── consume_events.py
│   │
│   └── evaluation_service/      # Scoring & leaderboards
│       ├── Dockerfile
│       ├── config/
│       └── evaluations/         # Models, Redis leaderboard, tests
│           └── management/commands/
│               └── consume_events.py
│
├── shared/                      # Shared utilities
│   ├── messaging.py             # RabbitMQ publish/consume
│   └── auth.py                  # JWT middleware for inter-service auth
│
├── gateway/
│   └── nginx.conf               # API gateway configuration
│
├── k8s/                         # Kubernetes manifests
│   ├── namespace.yaml
│   ├── configmap.yaml
│   ├── secrets.yaml
│   ├── ingress.yaml
│   ├── postgres/                # StatefulSet, PVC, Service
│   ├── redis/                   # Deployment, Service
│   ├── rabbitmq/                # Deployment, Service
│   ├── quiz-service/            # Deployment, Service, HPA
│   ├── user-service/            # Deployment, Service, HPA
│   └── evaluation-service/      # Deployment, Service, HPA
│
├── terraform/                   # AWS infrastructure as code
│   ├── modules/
│   │   ├── vpc/                 # VPC, subnets, NAT, endpoints
│   │   ├── eks/                 # EKS cluster, node groups, IRSA
│   │   ├── rds/                 # PostgreSQL, Secrets Manager
│   │   ├── elasticache/         # Redis cluster
│   │   └── mq/                  # Amazon MQ (RabbitMQ)
│   └── environments/
│       ├── dev/                 # Dev-sized infrastructure
│       └── prod/                # Production infrastructure
│
├── .github/
│   └── workflows/
│       └── ci.yml               # Lint + test pipeline
│
└── scripts/
    ├── setup.sh                 # Full development setup
    ├── run_tests.sh             # Run all test suites
    ├── deploy_k8s.sh            # Kubernetes deployment
    └── init_databases.sql       # PostgreSQL init script

---

Testing

# Run all tests
./scripts/run_tests.sh

# Run specific service tests
./scripts/run_tests.sh quiz
./scripts/run_tests.sh user
./scripts/run_tests.sh eval

# Run tests directly
docker-compose exec quiz_service python manage.py test --verbosity=2

---

Kubernetes Deployment

# Deploy to cluster
./scripts/deploy_k8s.sh

# Check status
kubectl -n knowledge-probe get pods
kubectl -n knowledge-probe get services

# Delete everything
./scripts/deploy_k8s.sh --delete

Before deploying, update:

1. k8s/secrets.yaml — Replace base64 placeholders with real secrets
2. k8s/ingress.yaml — Update hostname and enable TLS
3. Build and push Docker images to your container registry

---

Terraform (AWS Infrastructure)

cd terraform/environments/dev

# Configure
cp terraform.tfvars.example terraform.tfvars
# Edit terraform.tfvars

# Deploy
terraform init
terraform plan
terraform apply

Infrastructure provisions:

• VPC with public/private subnets, NAT Gateways, VPC endpoints
• EKS cluster with managed node groups and IRSA
• RDS PostgreSQL with encryption, backups, monitoring
• ElastiCache Redis with encryption in transit/at rest
• Amazon MQ RabbitMQ broker

---

Key Patterns Demonstrated

Microservice Patterns

• Database per Service — Each service owns its data, enabling independent deployment and scaling
• API Gateway — Single entry point with rate limiting, routing, and TLS termination
• Event-Driven Architecture — Async communication via RabbitMQ prevents temporal coupling
• CQRS-lite — Quiz service writes, Evaluation service reads/aggregates

Infrastructure Patterns

• Redis Sorted Sets — Real-time leaderboard rankings with O(log N) updates
• Redis Caching — Quiz data with event-driven invalidation
• JWT Auth — Stateless authentication across services
• Health Checks — Every service exposes /health/ for orchestrator probes
• HPA — Auto-scaling based on CPU utilization

DevOps Patterns

• Infrastructure as Code — Terraform modules with environment separation (dev/prod)
• Container Orchestration — Kubernetes with rolling updates
• Zero-Downtime Deployments — maxSurge/maxUnavailable strategy
• Pod Anti-Affinity — Spread replicas across nodes for high availability

---

Production Considerations

Things I would add or change for a real production deployment:

• Observability — OpenTelemetry distributed tracing, Prometheus metrics, Grafana dashboards, structured logging with correlation IDs across services
• CI/CD Pipeline — Multi-stage pipeline: lint, test, SAST scan, build images, push to ECR, deploy to staging, integration tests, promote to production
• Schema Registry — Versioned event schemas (e.g., AsyncAPI) to prevent breaking changes across service boundaries
• Circuit Breakers — Retry with exponential backoff and fallback patterns for RabbitMQ/Redis connections (e.g., using tenacity or pybreaker)
• API Versioning — URL-based versioning (/api/v1/) for backward compatibility as the API evolves
• Secret Management — AWS Secrets Manager or HashiCorp Vault with automatic rotation instead of environment variables
• Database Migrations — Blue/green deployment strategy for zero-downtime schema changes across services
• Rate Limiting — Per-user token bucket rate limiting at the gateway level, not just connection-based
• Dead Letter Queues — DLQs for failed RabbitMQ messages with alerting and retry mechanisms

---

Dashboard Access

Service	URL	Credentials
RabbitMQ Management	http://localhost:15672	guest / guest
Quiz Admin	http://localhost:8001/admin/	Create via createsuperuser
User Admin	http://localhost:8002/admin/	Create via createsuperuser
Eval Admin	http://localhost:8003/admin/	Create via createsuperuser