Skip to content

Aethel-Systems/AXStorageEngine

Repository files navigation

AX Storage Engine (Enterprise v1.0)

中文

AX is not a traditional database, but a "virtual DDR RAM module" running on disk files.

It discards the complex directory trees and B+ tree indexes of traditional file systems, directly replicating the physical topology of modern computer main memory (DDR RAM) on disk space. Through 4KB fixed-length paging, a PFN (Page Frame Number) database, and a Buddy System, AX elevates the addressing efficiency of persistent storage to the magnitude of memory access.


💎 Core Soul: Lightweight DDR Memory Layout

The storage layout of AX is a streamlined abstraction of a modern Memory Controller. It treats the entire .ax container as a piece of contiguous "physical memory" and divides it into the following core functional areas:

  1. Superblock: Equivalent to the SPD information of a memory module, recording capacity, bit width, and topology.
  2. PFN Database: Replicates the struct page array of the Linux kernel, maintaining the dirty bit, lock status, and type of each 4KB physical page in an extremely streamlined 1-byte format.
  3. Buddy System: Completely replicates the physical allocation algorithm of a memory controller, supporting contiguous physical memory allocation of 1, 2, 4...1024 pages, completely eliminating external fragmentation.
  4. L1/L2 Page Tables: Similar to CPU addressing, it locates data through direct offsets of logical addresses, skipping all complex lookup logic.

Why is it fast? Traditional storage requires "searching," whereas AX only requires "calculating addresses." This O(1) hardware-sensory addressing is the fundamental reason its performance surpasses that of LMDB and RocksDB.


🚀 Performance Benchmark

AX exhibits hair-raising performance in "extreme read/write" tests. Especially in Volatile (Hot Cache) mode, it almost touches the theoretical limit of the system bus.

Industrial-Grade Comparison Data

Storage Engine Architecture Type Read Performance (ops/sec) Write Performance (ops/sec) Persistence Guarantee Typical Scenario
AX (Volatile + Hot Cache) DDR Simulated Layout 17,800,000 ✨ 440,000 ⚠️ None (Maintained by OS) Publicity demos, real-time memory computing
AX (Volatile + Cache Cleared) DDR Simulated Layout 5,800,000 450,000 ⚠️ None (Data lost on power off) Industrial-grade caching, temporary hotspots
AX (Balanced Default) DDR Simulated Layout 121,000 42,000 ✅ Crash Consistency Production default, general scenarios
AX (Durable) DDR Simulated Layout ~100,000 ~15,000 ✅ Physically Forced Flush Critical configuration, financial transaction logs
LMDB B+tree + mmap 100k - 300k 30k - 100k ✅ Strong Consistency Embedded, read-heavy
RocksDB LSM Tree 16k - 500k+ 14k - 300k+ ✅ Strong Write Distributed storage, big data
SQLite (In-memory DB) B-tree + SQL 1,000,000+ 150,000 ❌ Not Persistent In-memory computing, SQL needs
Pure In-memory HashMap Hash Table 5,000,000+ 2,000,000+ ❌ Not Persistent Pure in-process caching

Test Environment: macOS 26.4.1 m5Pro 48GB RAM, Test files: test_volatile_nocache.c, test_volatile.c, test_fast.c


🛠️ Key Technical Features

1. Physical Addressing and Pinning (Hardware Locking)

Through the ax_pin_page interface, AX can command the operating system to forcibly fix specific disk-mapped pages in RAM. This means core business data is "welded" onto the physical memory module, never experiencing latency jitter caused by swapping.

2. Buddy System Allocator

AX does not use heap allocation internally. It possesses its own physical page management logic capable of allocating contiguous disk space at O(log N) speed. This design ensures space allocation efficiency under high-frequency writes, just as tight as DDR chip management.

3. Copy-on-Write (CoW) Transactions

While maintaining the DDR layout, AX introduces a lightweight CoW mechanism. Each update generates a new page mapping and atomically switches the superblock, ensuring that even if power is lost suddenly, the entire "virtual memory space" will not be corrupted.

4. Zero-Copy Reading

Read operations do not pass through user-space buffers; instead, they return the virtual memory address directly. For downstream algorithms, AX functions just like a native C language pointer array that can be persisted.


📦 Quick Installation and Usage

Compilation

 mkdir build && cd build                                                         
cmake ..
cmake --build . --config Release

CLI Tool: Managing Storage Like Operating a BIOS

# Format an AX storage volume with a capacity of 128MB
./ax_cli format disk.ax 128

# View the physical topology of the storage volume (PFN status, root page table, etc.)
./ax_cli info disk.ax

# Operate objects just like writing data to memory addresses
./ax_cli write disk.ax 1024 "Hello DDR-AX"

# Force trigger defragmentation to compact physical space
./ax_cli compact disk.ax

🎯 Why Choose AX?

Traditional storage solutions build a "library" on disk, whereas AX builds a "memory warehouse" directly on disk.

If your application scenario involves:

  • Extreme sensitivity to nanosecond/microsecond latency jitter
  • Need for a large-scale, high-performance persistent hash table
  • Industrial-grade real-time data acquisition and playback
  • Zero-copy data sharing on edge computing devices

Then, AX is that "never-power-off" DDR memory module of yours.

About

17M+ ops/s Ultra-high performance storage engine ( CPU cache). OS-style 9-9-12 paging & Buddy/SLAB allocation. Virtual memory-level zero-copy persistence.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors