Mini Key-Value Store

A production-inspired, multithreaded in-memory key-value database built with Modern C++17 and Linux system programming.

---

Overview

Mini Key-Value Store is a production-inspired backend systems project that implements the core components of an in-memory database from scratch using Modern C++17.

The goal of the project is not to compete with production databases, but to understand how they work internally by building the fundamental pieces yourself.

The server supports multiple concurrent clients over TCP, executes commands using a thread pool, stores data in memory, persists writes through an Append-Only Log (AOF), restores state during startup, supports key expiration (TTL), snapshots the database to disk, and performs graceful shutdown using POSIX signal handling.

This project was built as a systems programming exercise to gain hands-on experience with networking, concurrency, synchronization, persistence, and backend architecture.

---

---

Architecture

---

The server accepts TCP client connections, parses incoming commands, dispatches work to a fixed-size thread pool, and executes operations on a thread-safe sharded storage engine. Write operations are persisted through an Append-Only Log (AOF), snapshots are generated for faster recovery, and TTL management removes expired keys in the background.

Testing And Bug Reports

Thread Sanitizer

./build/app 
FATAL: ThreadSanitizer: unexpected memory mapping 0x61b36c730000-0x61b36c735000
If the above error comes paste the below command and then rerun
echo 0 | sudo tee /proc/sys/kernel/randomize_va_space

Address Sanitizer (ASan)

Detects memory errors: heap overflow, use-after-free, double free, memory leaks.

cmake -DUSE_ASAN=ON ..
make
./build/app

UndefinedBehaviorSanitizer (UBSan)

cmake -DUSE_UBSAN=ON ..
make
./build/app

Features

Networking

• POSIX socket-based TCP server
• Line-based text protocol
• Multiple concurrent client connections
• Per-client receive buffers
• Graceful connection handling

Storage Engine

• Thread-safe in-memory key-value database
• std::unordered_map based storage
• 16-shard architecture to reduce lock contention
• Reader-writer synchronization using std::shared_mutex
• Constant average-time (O(1)) key lookups

Concurrency

• Fixed-size thread pool
• Producer-consumer task queue
• std::mutex
• std::shared_mutex
• std::condition_variable
• RAII-based synchronization

Persistence

• Append-Only Log (AOF)
• Crash recovery
• Snapshot generation
• Log compaction

Expiration

• TTL support
• Lazy expiration
• Background cleanup thread

Memory Management

• Configurable maximum memory limit
• Automatic eviction when memory limit is reached
• FIFO / Random eviction policy
• Memory accounting for stored entries

Reliability

• Graceful shutdown using SIGINT
• Proper thread cleanup
• RAII resource management
• Clean socket shutdown

Development

• Modern C++17
• CMake
• Linux
• GDB
• AddressSanitizer
• ThreadSanitizer

---

Project Structure

mini-kv-store/
│
├── inc/
│   ├── network/
│   ├── database/
|
├── src/
│   ├── network/
│   ├── database/
│   └── main.cpp
│
├── tests/
│
├── data/
│   ├── appendonly.log
│   └── snapshot.db
│
├── CMakeLists.txt
├── README.md
└── .gitignore

Build & Run

Prerequisites

• Linux
• C++17 compatible compiler (GCC or Clang)
• CMake 3.16+
• Make

---

Clone the Repository

git clone https://github.com/arham6606/Mini_kv_store.git
cd Mini_kv_store

---

Build

mkdir build
cd build
cmake ..
make

---

Run the Server

./build/app

The server starts listening on:

127.0.0.1:6379

---

Connect Using Netcat

Open another terminal.

nc 127.0.0.1 6379

---

Supported Commands

Command	Description	Example
PING	Check server availability	PING
SET	Store a key-value pair	SET name Arham
GET	Retrieve a value	GET name
DEL	Delete a key	DEL name
SIZE	Return total number of keys	SIZE
EXIT	Close the client connection	EXIT

---

Example Session

$ nc 127.0.0.1 6379

PING
PONG

SET name Arham
OK

GET name
Arham

EXISTS name
1

SIZE
1

DEL name
1

GET name
(nil)

EXIT

---

Graceful Shutdown

The server supports graceful termination using SIGINT.

Pressing Ctrl + C:

• Stops accepting new client connections
• Notifies worker threads to exit
• Waits for active threads to finish
• Closes all open sockets
• Flushes pending persistent data
• Releases allocated resources through RAII
• Terminates cleanly without leaving orphaned threads

---

Future Improvements

The current implementation focuses on understanding backend infrastructure concepts. Possible future enhancements include:

• Event-driven networking with epoll
• Non-blocking sockets
• RESP-compatible protocol
• LRU/LFU eviction policies
• Replication
• Authentication
• Configuration file support
• Metrics endpoint
• Performance benchmarking
• Asynchronous disk I/O

---

Tech Stack

Category	Technology
Language	C++17
Platform	Linux
Networking	POSIX Sockets
Concurrency	std::thread, std::mutex, std::shared_mutex, std::condition_variable, std::unique_lock
Storage	std::unordered_map
Build System	CMake
Debugging	GDB, AddressSanitizer, ThreadSanitizer

---

Learning Objectives

This project was built to gain hands-on experience with:

• Linux system programming
• TCP/IP networking
• Concurrent programming
• Thread synchronization
• Producer-consumer architecture
• Thread pools
• In-memory database design
• Append-only persistence
• Snapshot-based recovery
• Key expiration (TTL)
• Storage sharding
• Modern C++ resource management (RAII)

---

License

This project is licensed under the MIT License.

See the LICENSE file for details.