AtlasDB - Raft-Based Distributed Database

Overview

AtlasDB is a distributed key-value database built from scratch in Java and Spring Boot, implementing the Raft consensus algorithm for leader election, log replication, fault tolerance, and automatic recovery.

The project demonstrates how modern distributed systems such as etcd, Consul, TiKV, and CockroachDB maintain strong consistency across multiple nodes while tolerating failures.

Key Features

Distributed Consensus (Raft)

• Leader Election
• Randomized Election Timeouts
• Heartbeats (AppendEntries RPC)
• Term-based Leadership
• Automatic Leader Failover
• Majority Quorum-Based Decisions

Replication

• Write-Ahead Logging (WAL)
• Log Replication Across Nodes
• Commit Index Management
• Majority Acknowledgement Before Commit
• Incremental Follower Synchronization
• Log Consistency Verification
• Automatic Log Repair using nextIndex

Fault Tolerance

• Follower Crash Recovery
• Persistent Raft State
• Persistent Log Storage
• Snapshot-Based Recovery
• Automatic Cluster Healing
• Node Rejoin Synchronization

Storage Engine

• In-Memory State Machine
• ConcurrentHashMap-Based KV Store
• Write-Ahead Log Persistence
• Snapshot Creation
• Log Compaction

Observability

• Cluster Status Endpoint
• Leader Identification
• Commit Index Tracking
• Election Monitoring

---

Architecture

                    Client
                       |
                  REST API
                       |
          +------------+------------+
          |                         |
      Leader                   Followers
      Node-1                   Node-2
          |                         |
          +------ Raft RPC ---------+
          |                         |
                           Follower
                            Node-3

                 -------------------
                 |     Raft Layer   |
                 -------------------
                           |
                 -------------------
                 |    Raft Log      |
                 -------------------
                           |
                 -------------------
                 | Commit Index     |
                 -------------------
                           |
                 -------------------
                 | State Machine    |
                 -------------------
                           |
                 -------------------
                 | Snapshot Store   |
                 -------------------

---

Raft Workflow

Leader Election

FOLLOWER
    |
Election Timeout
    |
    v
CANDIDATE
    |
RequestVote RPC
    |
Majority Votes
    |
    v
LEADER

Log Replication

Client Write
      |
      v
Leader Append Log
      |
Replicate to Followers
      |
Majority Acknowledgement
      |
Commit Entry
      |
Apply To State Machine

Automatic Recovery

Follower Crash
      |
Follower Restart
      |
Load Snapshot
      |
Load Persistent Log
      |
Incremental Synchronization
      |
Cluster Consistent

---

Project Structure

src/main/java/com/cfs/backend

├── controller
│   ├── KVController
│   ├── VoteController
│   ├── HeartbeatController
│   ├── ReplicationController
│   └── ClusterController
│
├── raft
│   ├── RaftNode
│   ├── RaftLog
│   ├── RaftLogStore
│   ├── ElectionService
│   ├── VoteService
│   ├── HeartbeatService
│   ├── ReplicationService
│   ├── LogReplicationService
│   ├── CommitService
│   ├── ReplicationTracker
│   ├── PersistentRaftState
│   └── RaftStateManager
│
├── storage
│   ├── StateMachine
│   ├── WALManager
│   ├── StateMachineApplier
│   └── RecoveryManager
│
├── snapshot
│   ├── Snapshot
│   └── SnapshotManager
│
├── dto
│   └── ClusterStatus
│
├── exception
│   ├── NotLeaderException
│   └── GlobalExceptionHandler
│
└── model
    ├── LogEntry
    ├── VoteRequest
    ├── VoteResponse
    ├── AppendEntriesRequest
    ├── AppendEntriesResponse
    ├── ReplicationRequest
    ├── ReplicationResponse
    ├── CommitRequest
    ├── CommitResponse
    ├── SyncRequest
    └── SyncResponse

---

Running a 3-Node Cluster

Node 1

mvn spring-boot:run \
-Dspring-boot.run.profiles=node1

Node 2

mvn spring-boot:run \
-Dspring-boot.run.profiles=node2

Node 3

mvn spring-boot:run \
-Dspring-boot.run.profiles=node3

---

API Examples

Put Key

POST /kv

{
  "key":"user",
  "value":"aman"
}

Get Key

GET /kv/user

Delete Key

DELETE /kv/user

Cluster Status

GET /cluster/status

Example Response:

{
  "nodeId":"node-1",
  "role":"LEADER",
  "currentTerm":12,
  "leaderId":"node-1",
  "commitIndex":157
}

---

Failure Recovery Demo

Step 1

Start all three nodes.

Node1 -> Leader
Node2 -> Follower
Node3 -> Follower

Step 2

Insert records.

PUT user=aman
PUT city=delhi

Step 3

Terminate leader.

Node1 Down

Step 4

Automatic election occurs.

Node2 -> Leader
Node3 -> Follower

Step 5

Continue writes successfully.

PUT country=india

Step 6

Restart Node1.

Node1 -> Follower
Node1 -> Synchronizes Logs
Node1 -> Cluster Consistent

---

Concepts Implemented

• Raft Consensus Algorithm
• Distributed Systems Fundamentals
• Strong Consistency
• Quorum-Based Commit
• Leader Election
• Heartbeats
• Log Replication
• Commit Index
• Write-Ahead Logging
• Crash Recovery
• Persistent Metadata
• Snapshotting
• Log Compaction
• Incremental Synchronization
• Conflict Resolution
• Automatic Log Repair
• Fault Tolerance

---

Future Enhancements

• InstallSnapshot RPC
• Linearizable Reads
• Membership Changes
• Dynamic Cluster Scaling
• Prometheus Metrics
• Grafana Dashboards
• Docker Deployment
• Kubernetes Deployment
• gRPC-Based Replication
• Multi-Raft Sharding
• SQL Layer
• Distributed Transactions

---

Inspiration

This project draws inspiration from the design principles used in:

• etcd
• Consul
• TiKV
• CockroachDB
• Apache Ratis

while implementing the core algorithms from scratch for educational and engineering purposes.