Skip to content

robindevelops/Full-Stack-DevOps-Pipeline

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

3 Commits
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

πŸš€ Full-Stack DevOps Pipeline β€” Amazon Prime Video Clone

A fully automated, production-grade CI/CD pipeline for deploying an Amazon Prime Video clone using modern DevOps practices and cloud-native tooling.


Architecture CI/CD IaC Container Orchestration GitOps Monitoring


Full-Stack DevOps Pipeline Architecture

πŸ“‹ Table of Contents


🎯 Project Overview

This project demonstrates a complete end-to-end DevOps pipeline for an Amazon Prime Video clone application. The entire workflow β€” from code commit to production deployment β€” is fully automated with industry-standard tools.

What This Project Covers

Area Tools Used
Source Control GitHub
CI/CD Automation Jenkins
Code Quality SonarQube
Dependency Management npm
Security Scanning Aqua Trivy
Containerization Docker
Container Registry AWS ECR
Infrastructure as Code Terraform
Kubernetes Orchestration AWS EKS
Package Management Helm
GitOps / Continuous Delivery ArgoCD
Monitoring Prometheus & Grafana

πŸ— Architecture

Developer β†’ GitHub β†’ Jenkins Pipeline
                         β”‚
                         β”œβ”€β”€ 1. Code Checkout
                         β”œβ”€β”€ 2. SonarQube Analysis (Code Quality)
                         β”œβ”€β”€ 3. npm Install & Build
                         β”œβ”€β”€ 4. Trivy Scan (Security)
                         β”œβ”€β”€ 5. Docker Build & Tag
                         β”œβ”€β”€ 6. Push to AWS ECR
                         β”‚
                         β–Ό
                    AWS ECR (Container Registry)
                         β”‚
                         β–Ό
                    ArgoCD (GitOps)
                         β”‚
                         β”œβ”€β”€ Watches Helm chart changes
                         β”œβ”€β”€ Syncs desired state
                         β”‚
                         β–Ό
                    AWS EKS Cluster
                         β”‚
                         β”œβ”€β”€ Pulls image from ECR
                         β”œβ”€β”€ Deploys Pods via Helm
                         β”œβ”€β”€ Manages Services & Ingress
                         β”‚
                         β–Ό
                    Prometheus + Grafana (Monitoring)

High-Level Flow

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  GitHub  │───▢│ Jenkins  │───▢│ SonarQube  │───▢│  npm  │───▢│ Trivy │───▢│ Docker β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”¬β”€β”€β”€β”˜
                                                                                 β”‚
                     β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                     β–Ό
              β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
              β”‚  AWS ECR   │───▢│  ArgoCD  │───▢│ AWS EKS │───▢│ Prometheus  β”‚
              β”‚  (Registry)β”‚    β”‚ (GitOps) β”‚    β”‚  (K8s)  β”‚    β”‚ + Grafana   β”‚
              β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

πŸ›  Tech Stack

Application

Layer Technology Purpose
Frontend React.js Amazon Prime Video clone UI
Backend Node.js REST API server
Styling CSS Responsive design & animations

DevOps & Infrastructure

Category Tool Purpose
Version Control GitHub Source code management & webhooks
CI/CD Jenkins Pipeline orchestration & automation
Code Quality SonarQube Static code analysis & quality gates
Security Aqua Trivy Container & filesystem vulnerability scan
Containerization Docker Application containerization
Registry AWS ECR Private container image registry
IaC Terraform AWS infrastructure provisioning
Orchestration AWS EKS Managed Kubernetes cluster
Packaging Helm Kubernetes manifest templating & management
GitOps ArgoCD Declarative continuous delivery
Monitoring Prometheus Metrics collection & alerting
Visualization Grafana Dashboards & observability

πŸ”„ Pipeline Flow

End-to-End CI/CD Pipeline Stages

 β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
 β”‚                        JENKINS CI/CD PIPELINE                               β”‚
 β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
 β”‚  Stage 1β”‚  Stage 2  β”‚ Stage 3  β”‚ Stage 4  β”‚  Stage 5  β”‚     Stage 6        β”‚
 β”‚         β”‚           β”‚          β”‚          β”‚           β”‚                    β”‚
 β”‚  Clone  β”‚ SonarQube β”‚   npm    β”‚  Trivy   β”‚  Docker   β”‚  Push to ECR      β”‚
 β”‚  Repo   β”‚  Analysis β”‚  Build   β”‚  Scan    β”‚  Build    β”‚  + Deploy via     β”‚
 β”‚         β”‚           β”‚          β”‚          β”‚           β”‚    ArgoCD          β”‚
 β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
  1. πŸ“₯ Clone β€” Pull latest code from GitHub
  2. πŸ” SonarQube Analysis β€” Static code analysis for bugs, vulnerabilities, code smells
  3. πŸ“¦ npm Install & Build β€” Install dependencies and build the application
  4. πŸ›‘οΈ Trivy Security Scan β€” Scan filesystem and Docker images for CVEs
  5. 🐳 Docker Build & Tag β€” Build production Docker image with proper tagging
  6. πŸ“€ Push to AWS ECR β€” Push image to private container registry
  7. πŸš€ ArgoCD Sync β€” ArgoCD detects Helm chart changes and deploys to EKS

πŸ“ Prerequisites

Before setting up this pipeline, ensure you have the following:

Local Development

CI/CD & Infrastructure

AWS Resources

  • AWS Account with IAM permissions for EKS, ECR, VPC, and EC2
  • An S3 bucket for Terraform state (recommended)

πŸ“ Project Structure

full_stack_devops_pipeline/
β”‚
β”œβ”€β”€ πŸ“„ server.js                    # Node.js application entry point
β”œβ”€β”€ πŸ“„ package.json                 # Node.js dependencies & scripts
β”œβ”€β”€ πŸ“„ Dockerfile                   # Multi-stage Docker build
β”œβ”€β”€ πŸ“„ docker-compose.yml           # Local development orchestration
β”œβ”€β”€ πŸ“„ Jenkinsfile                  # CI/CD pipeline definition
β”œβ”€β”€ πŸ“„ .dockerignore                # Docker build exclusions
β”œβ”€β”€ πŸ“„ .gitignore                   # Git exclusions
β”œβ”€β”€ πŸ“„ README.md                    # Project documentation (this file)
β”‚
β”œβ”€β”€ πŸ“‚ src/                         # Application source code
β”‚   β”œβ”€β”€ πŸ“‚ components/              # React components
β”‚   β”œβ”€β”€ πŸ“‚ pages/                   # Page-level components
β”‚   β”œβ”€β”€ πŸ“‚ assets/                  # Static assets (images, icons)
β”‚   └── πŸ“„ App.js                   # Main React application
β”‚
β”œβ”€β”€ πŸ“‚ terraform/                   # Infrastructure as Code
β”‚   β”œβ”€β”€ πŸ“„ main.tf                  # Main Terraform configuration
β”‚   β”œβ”€β”€ πŸ“„ variables.tf             # Input variables
β”‚   β”œβ”€β”€ πŸ“„ outputs.tf               # Output values
β”‚   β”œβ”€β”€ πŸ“„ provider.tf              # AWS provider configuration
β”‚   β”œβ”€β”€ πŸ“‚ modules/
β”‚   β”‚   β”œβ”€β”€ πŸ“‚ vpc/                 # VPC networking module
β”‚   β”‚   β”œβ”€β”€ πŸ“‚ eks/                 # EKS cluster module
β”‚   β”‚   └── πŸ“‚ ecr/                 # ECR repository module
β”‚   └── πŸ“„ terraform.tfvars         # Variable values
β”‚
β”œβ”€β”€ πŸ“‚ helm/                        # Helm chart for Kubernetes deployment
β”‚   β”œβ”€β”€ πŸ“„ Chart.yaml               # Chart metadata
β”‚   β”œβ”€β”€ πŸ“„ values.yaml              # Default configuration values
β”‚   └── πŸ“‚ templates/
β”‚       β”œβ”€β”€ πŸ“„ deployment.yaml      # K8s Deployment manifest
β”‚       β”œβ”€β”€ πŸ“„ service.yaml         # K8s Service manifest
β”‚       β”œβ”€β”€ πŸ“„ ingress.yaml         # K8s Ingress manifest
β”‚       └── πŸ“„ hpa.yaml             # Horizontal Pod Autoscaler
β”‚
β”œβ”€β”€ πŸ“‚ monitoring/                  # Monitoring stack configuration
β”‚   β”œβ”€β”€ πŸ“„ prometheus.yml           # Prometheus configuration
β”‚   └── πŸ“‚ grafana/
β”‚       └── πŸ“‚ dashboards/          # Grafana dashboard JSON files
β”‚
└── πŸ“‚ scripts/                     # Utility scripts
    β”œβ”€β”€ πŸ“„ setup-jenkins.sh         # Jenkins setup automation
    β”œβ”€β”€ πŸ“„ install-tools.sh         # Tool installation script
    └── πŸ“„ cleanup.sh               # Resource cleanup script

πŸš€ Getting Started

1. Clone the Repository

git clone https://github.com/<your-username>/full-stack-devops-pipeline.git
cd full-stack-devops-pipeline

2. Local Development

# Install dependencies
npm install

# Start the development server
npm run dev

# Application will be available at http://localhost:3000

3. Docker Build

# Build the Docker image
docker build -t prime-video-clone .

# Run the container
docker run -d -p 3000:3000 --name prime-clone prime-video-clone

# Or use Docker Compose
docker-compose up -d

4. Infrastructure Provisioning (Terraform)

cd terraform/

# Initialize Terraform
terraform init

# Preview the infrastructure changes
terraform plan

# Apply β€” provisions VPC, EKS Cluster, ECR, and IAM Roles
terraform apply -auto-approve

# Get EKS kubeconfig
aws eks update-kubeconfig --name prime-clone-cluster --region us-east-1

5. CI/CD Pipeline (Jenkins)

Jenkins Setup

  1. Install Jenkins on an EC2 instance or run locally via Docker:

    docker run -d --name jenkins \
      --restart=on-failure \
      -p 8080:8080 \
      -v jenkins_home:/var/jenkins_home \
      -v /var/run/docker.sock:/var/run/docker.sock \
      jenkins/jenkins:lts
  2. Install required Jenkins plugins:

    • Docker Pipeline
    • SonarQube Scanner
    • AWS Credentials
    • Pipeline: AWS Steps
    • NodeJS Plugin
  3. Configure credentials in Jenkins:

    • github-credentials β€” GitHub personal access token
    • sonarqube-token β€” SonarQube authentication token
    • aws-credentials β€” AWS Access Key & Secret Key
    • dockerhub-credentials β€” Docker Hub credentials (if using)
  4. Create a Pipeline job pointing to the Jenkinsfile in this repository.

  5. Set up GitHub Webhook β†’ http://<jenkins-url>:8080/github-webhook/

Jenkinsfile Overview

pipeline {
    agent any

    environment {
        AWS_ACCOUNT_ID   = credentials('aws-account-id')
        AWS_REGION       = 'us-east-1'
        ECR_REPO         = "${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/prime-clone"
        SONAR_HOST       = 'http://<sonarqube-server>:9000'
    }

    stages {
        stage('Checkout')          { steps { git branch: 'main', url: '...' } }
        stage('SonarQube Analysis'){ steps { /* SonarQube scanner */ } }
        stage('npm Build')         { steps { sh 'npm install && npm run build' } }
        stage('Trivy FS Scan')     { steps { sh 'trivy fs --severity HIGH,CRITICAL .' } }
        stage('Docker Build')      { steps { sh 'docker build -t prime-clone .' } }
        stage('Trivy Image Scan')  { steps { sh 'trivy image prime-clone' } }
        stage('Push to ECR')       { steps { /* ECR login & push */ } }
        stage('Deploy via ArgoCD') { steps { /* Update Helm values, ArgoCD syncs */ } }
    }

    post {
        success { echo 'βœ… Pipeline completed successfully!' }
        failure { echo '❌ Pipeline failed. Check logs.' }
    }
}

6. GitOps Deployment (ArgoCD)

# Install ArgoCD on EKS
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

# Access ArgoCD UI
kubectl port-forward svc/argocd-server -n argocd 8080:443

# Get initial admin password
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d

# Create ArgoCD Application (points to Helm chart in this repo)
argocd app create prime-clone \
  --repo https://github.com/<your-username>/full-stack-devops-pipeline.git \
  --path helm/ \
  --dest-server https://kubernetes.default.svc \
  --dest-namespace production \
  --sync-policy automated

7. Monitoring (Prometheus & Grafana)

# Install Prometheus & Grafana via Helm
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

helm install monitoring prometheus-community/kube-prometheus-stack \
  --namespace monitoring \
  --create-namespace

# Access Grafana dashboard
kubectl port-forward svc/monitoring-grafana -n monitoring 3000:80
# Default credentials β€” admin / prom-operator

πŸ”’ Security

This pipeline implements security at every stage:

Layer Tool/Practice Description
Code Quality SonarQube Detects bugs, code smells & vulnerabilities
Dependency Scan Trivy (filesystem mode) Scans node_modules for known CVEs
Image Scan Trivy (image mode) Scans built Docker image for OS/library vulns
Registry AWS ECR (private) Images stored in private, encrypted registry
Secrets Management Jenkins Credentials Store No hardcoded secrets in code
Network AWS VPC + Security Groups Isolated network with least-privilege access
Cluster EKS with RBAC Role-based access control for Kubernetes

πŸ“Š Monitoring & Observability

Component Purpose Access
Prometheus Metrics collection, alerting rules http://<cluster-ip>:9090
Grafana Visualization dashboards http://<cluster-ip>:3000
ArgoCD UI Deployment status & sync health http://<cluster-ip>:8080
Jenkins Pipeline logs & build history http://<jenkins-ip>:8080

Key Metrics Monitored

  • 🟒 Application β€” Request latency, error rates, throughput
  • πŸ”΅ Kubernetes β€” Pod health, resource utilization, node status
  • 🟠 Infrastructure β€” CPU, memory, disk, network I/O
  • πŸ”΄ Alerts β€” Pipeline failures, pod crashes, high error rates

🀝 Contributing

Contributions are welcome! Please follow these steps:

  1. Fork the repository
  2. Create a feature branch: git checkout -b feature/amazing-feature
  3. Commit your changes: git commit -m 'Add amazing feature'
  4. Push to the branch: git push origin feature/amazing-feature
  5. Open a Pull Request

Commit Convention

feat: add new feature
fix: bug fix
docs: documentation update
ci: CI/CD pipeline changes
infra: infrastructure changes
refactor: code refactoring
test: add or update tests

πŸ“„ License

This project is licensed under the MIT License β€” see the LICENSE file for details.


πŸ™ Acknowledgments


⭐ Star this repo if you find it helpful!

Built with ❀️ using modern DevOps practices

About

This project implements a highly resilient, production-grade DevOps ecosystem designed for high availability and security. The architecture transitions from a traditional CI/CD model to a modern GitOps approach, utilizing ArgoCD to synchronize the state of the AWS EKS cluster with the source code. By integrating Aqua Trivy and SonarQube, the pipeli

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors