Skip to content

README.md

Latest commit

 

History

History
468 lines (359 loc) · 15.8 KB

README.md

File metadata and controls

468 lines (359 loc) · 15.8 KB

Hankers DevOps

Hankers Logo

Production Infrastructure & Deployment Platform

Enterprise-grade DevOps infrastructure for the Hankers social network

🔗 Live Platform: hankers.tech

📌 Overview

Hankers DevOps is a production-ready infrastructure platform built with Infrastructure as Code (IaC) principles, designed for scalability, reliability, and automation. The project implements modern DevOps practices including container orchestration, CI/CD pipelines, monitoring, and cost-optimized cloud infrastructure.

This codebase demonstrates real-world DevOps engineering: automated deployments, infrastructure versioning, zero-downtime releases, and comprehensive observability.

✨ Key Infrastructure Features

  • 🚀 CI/CD Automation — GitHub Actions for build, test, and deploy
  • ☸️ Kubernetes Orchestration — Azure Kubernetes Service (AKS) for container management
  • 📦 GitOps Deployment — ArgoCD for declarative continuous delivery
  • 🎯 Auto-scaling — Karpenter for intelligent node provisioning
  • 📊 Full Observability — Prometheus, Grafana, Loki for metrics and logs
  • 🔒 Automated SSL/TLS — cert-manager with Let's Encrypt
  • 🏗️ Infrastructure as Code — Terraform for reproducible infrastructure
  • 🐳 Container Registry — GitHub Container Registry (GHCR) + Docker Hub
  • 💾 Managed Database — PostgreSQL on Azure with automated backups
  • 🗄️ Caching Layer — Redis cluster for session management
  • 📦 Object Storage — Azure Blob Storage with CloudFront CDN
  • 🔐 Security First — Network policies, RBAC, encrypted storage

🏗️ Architecture Overview

High-Level Architecture

┌─────────────────────────────────────────────────────────────────┐
│                         Internet Traffic                         │
└──────────────────────────┬──────────────────────────────────────┘
                           │
                           ▼
                   ┌───────────────┐
                   │ Load Balancer │
                   └───────┬───────┘
                           │
                           ▼
              ┌────────────────────────┐
              │   Nginx Ingress        │
              │   (SSL Termination)    │
              └────────┬───────────────┘
                       │
         ┌─────────────┼─────────────┐
         │             │             │
         ▼             ▼             ▼
   ┌─────────┐   ┌─────────┐   ┌─────────┐
   │Frontend │   │ Backend │   │   AI    │
   │  Pods   │   │  Pods   │   │  Bot    │
   └────┬────┘   └────┬────┘   └────┬────┘
        │             │             │
        │             ▼             │
        │      ┌──────────────┐    │
        │      │  PostgreSQL  │    │
        │      │   (Azure)    │    │
        │      └──────────────┘    │
        │                          │
        └──────────┬───────────────┘
                   │
                   ▼
            ┌─────────────┐
            │    Redis    │
            │   Cluster   │
            └─────────────┘

Deployment Pipeline Flow

Developer Push → GitHub → CI/CD Pipeline → Docker Build → Push to Registry
                                                                 │
                                                                 ▼
                                                      Update Helm Charts
                                                                 │
                                                                 ▼
                                                        ArgoCD Detects Change
                                                                 │
                                                                 ▼
                                                    Deploy to Kubernetes (30s)
                                                                 │
                                                                 ▼
                                                      Discord Notification

🛠 Technology Stack

Infrastructure & Orchestration

  • Cloud Provider: Microsoft Azure
  • Container Orchestration: Azure Kubernetes Service (AKS)
  • Infrastructure as Code: Terraform
  • Package Management: Helm
  • GitOps: ArgoCD

CI/CD & Automation

  • CI/CD Platform: GitHub Actions
  • Container Registry: GitHub Container Registry (GHCR), Docker Hub
  • Configuration Management: Ansible
  • Build Tools: Docker, multi-stage builds

Monitoring & Observability

  • Metrics: Prometheus
  • Visualization: Grafana
  • Log Aggregation: Loki
  • Log Shipping: Promtail
  • Alerting: AlertManager

Networking & Security

  • Ingress Controller: Nginx Ingress
  • Certificate Management: cert-manager
  • SSL Provider: Let's Encrypt
  • DNS Management: Azure DNS / Route53

Data Layer

  • Primary Database: PostgreSQL (Azure Database)
  • Cache/Sessions: Redis (Kubernetes-managed)
  • Object Storage: Azure Blob Storage
  • CDN: CloudFront

Auto-scaling & Cost Optimization

  • Node Auto-scaling: Karpenter
  • Pod Auto-scaling: Horizontal Pod Autoscaler (HPA)
  • Spot Instances: Enabled for 50%+ cost savings

🎯 Design Patterns & Best Practices

1. Factory Pattern

  • Location: CI/CD pipeline (runner/.github/workflows/cicd-pipeline.yml)
  • Implementation: Universal pipeline dynamically selects build scripts based on repository
  • Benefit: Single workflow handles backend, frontend, and cross-platform projects

2. Strategy Pattern

  • Location: Deployment scripts (devops/*/deploy.sh)
  • Implementation: Different Helm values files (dev/prod) selected at runtime
  • Benefit: Environment-specific configurations without code duplication

3. Template Method Pattern

  • Location: Helm charts (*/helm-chart/templates/)
  • Implementation: Base Kubernetes manifests with variable substitution
  • Benefit: DRY principle, reusable templates across services

4. Builder Pattern

  • Location: Terraform modules (infra/terraform-prod/modules/)
  • Implementation: Composable infrastructure blocks (VPC, EKS, RDS)
  • Benefit: Testable, reusable infrastructure components

5. Decorator Pattern

  • Location: Docker multi-stage builds
  • Implementation: Layered approach (base → build → runtime)
  • Benefit: Smaller images, better caching, production security

6. Observer Pattern

  • Location: Monitoring stack
  • Implementation: Prometheus scrapes metrics, AlertManager triggers notifications
  • Benefit: Decoupled monitoring, scalable alerting

7. Proxy Pattern

  • Location: Nginx Ingress Controller
  • Implementation: Reverse proxy routing external traffic to internal services
  • Benefit: Centralized SSL, load balancing, single entry point

8. Repository Pattern

  • Location: Multi-repo structure
  • Implementation: Separate repos for backend, frontend, cross-platform, devops
  • Benefit: Independent versioning, team autonomy, clear boundaries

🚀 Quick Start

Prerequisites

  • Azure subscription (or AWS for alternative setup)
  • kubectl, helm, terraform installed locally
  • GitHub account with Personal Access Token (PAT)
  • Docker installed for local testing

Infrastructure Setup

# 1. Clone the repository
git clone https://github.com/SWEProject25/devops.git
cd devops

# 2. Configure Terraform variables
cd infra/terraform
cp terraform.tfvars.example terraform.tfvars
# Edit terraform.tfvars with your values

# 3. Provision infrastructure
terraform init
terraform plan
terraform apply

# 4. Configure kubectl
aws eks update-kubeconfig --region us-east-1 --name hankers-eks

# 5. Install Kubernetes components using Ansible
cd ../setup/ansible
ansible-playbook playbook.yml

# 6. Verify deployment
kubectl get pods -A

Manual Deployment (Optional)

# Deploy a specific service manually
cd backend  # or frontend, ml-model-service
bash deploy.sh

📁 Project Structure

devops/
├── backend/                    # Backend service deployment
│   ├── helm-chart/            # Kubernetes manifests
│   ├── build.sh               # Docker build script
│   ├── deploy.sh              # Helm deployment script
│   └── sonar-project.properties
├── frontend/                   # Frontend service deployment
│   ├── helm-chart/
│   ├── build.sh
│   ├── deploy.sh
│   └── sonar-project.properties
├── cross-platform/            # Flutter mobile app builds
│   ├── build.sh
│   └── sonar-project.properties
├── ml-model-service/          # AI service deployment
│   ├── build.sh
│   └── deploy.sh
├── ai-bot/                    # AI bot service
│   └── build.sh
├── infra/
│   ├── terraform/             # Infrastructure as Code
│   │   ├── main.tf
│   │   ├── modules/          # Reusable Terraform modules
│   │   │   ├── vpc/
│   │   │   ├── eks/
│   │   │   ├── rds/
│   │   │   ├── s3/
│   │   │   ├── karpenter/
│   │   │   └── cloudfront/
│   │   ├── outputs.tf
│   │   └── variables.tf
│   └── setup/
│       ├── ansible/           # Kubernetes setup automation
│       │   ├── playbook.yml
│       │   ├── roles/        # Ansible roles for each component
│       │   │   ├── helm_repos/
│       │   │   ├── metrics-server/
│       │   │   ├── ingress/
│       │   │   ├── cert_manager/
│       │   │   ├── redis/
│       │   │   ├── monitoring/
│       │   │   ├── logging/
│       │   │   ├── karpenter/
│       │   │   └── argocd/
│       │   └── inventory/
│       └── config/            # Configuration files
│           ├── argocd/
│           ├── ingress/
│           ├── karpenter/
│           ├── monitoring/
│           └── logging/
└── README.md

🔧 Configuration Management

Environment Variables

Each service requires specific environment variables. These are managed through:

  1. GitHub Secrets — Sensitive credentials (API keys, passwords)
  2. Helm Values — Service configurations (values.yaml)
  3. ConfigMaps — Non-sensitive application configs
  4. Secrets — Encrypted sensitive data in Kubernetes

Key Configuration Files

  • terraform.tfvars — Infrastructure parameters
  • values.yaml — Helm chart configurations
  • sonar-project.properties — Code quality settings
  • ansible.cfg — Automation configuration

📊 Monitoring & Observability

Access Monitoring Dashboards

Available Dashboards

  1. Kubernetes Cluster Overview — CPU, memory, pods, nodes
  2. Node Exporter — Detailed node metrics
  3. Application Metrics — Custom service metrics
  4. Log Exploration — Loki integration for log analysis

Alerting

AlertManager configured for:

  • High CPU/memory usage
  • Pod crash loops
  • Certificate expiration warnings
  • Database connection issues

🔐 Security Features

  • ✅ Network Policies — Pod-to-pod traffic control
  • ✅ RBAC — Role-based access control for Kubernetes
  • ✅ Encrypted Storage — All persistent volumes encrypted
  • ✅ SSL/TLS — Automatic certificate management
  • ✅ Secret Management — Kubernetes secrets for credentials
  • ✅ Security Scanning — Container image vulnerability scanning
  • ✅ Pod Security Standards — Enforced security policies

💰 Cost Optimization

Implemented Strategies

  1. Karpenter Auto-scaling — Intelligent node provisioning
  2. Spot Instances — 50-70% cost reduction for non-critical workloads
  3. Right-sizing — Automated resource optimization
  4. No NAT Gateway — Saved $45/month by using public subnets
  5. ARM Instances — 20% cheaper than x86 (t4g vs t3)
  6. Storage Optimization — gp3 volumes instead of gp2
  7. Reduced Backup Retention — 1 day instead of 7 for non-prod

Estimated Monthly Costs

  • Kubernetes Cluster: ~$70-100
  • Database (RDS): ~$15-30
  • Load Balancer: ~$20
  • Storage: ~$10
  • Monitoring: ~$5
  • Total: ~$120-165/month

🚦 CI/CD Pipeline

Pipeline Stages

  1. Code Checkout — Pull latest code from repository
  2. Run Tests — Unit tests, integration tests, coverage
  3. Build Docker Image — Multi-stage builds for optimization
  4. Push to Registry — GHCR and Docker Hub
  5. Update Helm Charts — Automated version bump
  6. ArgoCD Sync — Automatic deployment (30 seconds)
  7. Notify Team — Discord webhook notification

Deployment Speed

  • Build Time: ~3-5 minutes
  • Deployment Time: ~30 seconds (ArgoCD)
  • Total Time to Production: ~5 minutes

📚 Documentation

Quick Links

🎓 DevOps Team


Karim Farid

📄 License

This project is for educational and portfolio purposes. All tools used are open-source with commercial-friendly licenses:

  • Kubernetes, Docker, Helm, Terraform: Apache 2.0 License ✅
  • Prometheus: Apache 2.0 License ✅
  • Grafana, Loki: AGPL v3 (acceptable for internal use) ✅
  • GitHub Actions: Free tier for private repos ✅

🤝 Contributing

This is a production infrastructure repository. For collaboration or questions:

  1. Open an issue for discussion
  2. Submit pull requests with detailed descriptions
  3. Follow infrastructure best practices
  4. Test changes in a separate environment first

📞 Support

For infrastructure issues or questions:

🌟 Acknowledgments

Special thanks to the open-source community for the amazing tools that make this infrastructure possible:

  • Kubernetes community
  • HashiCorp (Terraform)
  • Helm maintainers
  • Prometheus & Grafana teams
  • cert-manager contributors
  • Karpenter developers

⭐ If you find this DevOps setup useful, consider starring the repository!