Aether-Stream is a local C++20 message-broker toolkit for studying and demonstrating low-latency-oriented messaging primitives in a carefully bounded environment. It combines a single-producer/single-consumer lock-free queue, broker-style APIs, mmap-backed write-ahead logging (WAL), replay tools, metrics, CLI demos, benchmarks, and CI verification.
It is intentionally local-first: no networking, no distributed cluster, no live cross-process broker service, and no production-readiness claim. The value of the project is in the correctness-focused implementation of queueing, persistence, diagnostics, benchmark discipline, and build automation.
Local library and CLI toolkit. Explicit limitations. No unsupported benchmark numbers.
Overview · Quick Start · Architecture · Benchmarks · Docs
- What Aether-Stream does
- Technical depth
- What is included vs intentionally not claimed
- Architecture at a glance
- Core feature tour
- Quick Start
- Benchmarks
- Docs and deep dives
- Testing and CI
- Limitations and honesty notes
- Project structure
- Contributing
- License
- Acknowledgements
Aether-Stream provides local messaging building blocks that can be read, tested, benchmarked, and extended without introducing service infrastructure. The repository contains a reusable C++20 library, examples, CLI tools, CTest coverage, benchmark targets, and documentation for the queueing, WAL, broker, metrics, and build systems.
For non-technical reviewers: this project shows how a local message can move through a bounded queue, optionally be written to a local log file first, be replayed later, and be measured with diagnostics. For technical reviewers: the interesting parts are the SPSC synchronization contract, raw slot lifetime management, WAL record validation, mmap resource handling, typed replay constraints, relaxed metrics counters, and repeatable verification workflow.
| Capability | Codebase-grounded behavior |
|---|---|
| Lock-free SPSC queue | Uses acquire/release publication, cache-line padding, raw slot lifetime management, and stress tests. |
| Broker APIs | Provides in-memory, batch, and WAL-backed wrappers over the SPSC queue. |
| WAL persistence | Appends validated fixed-format records before queue publication in the persistent broker. |
| mmap file abstraction | Wraps POSIX memory-mapped file I/O (mmap) behind RAII (automatic, exception-safe resource cleanup) and reports unsupported behavior where native support is not implemented. |
| Metrics and diagnostics | Provides counters, snapshots, latency histogram, and CLI summaries. |
| Benchmark discipline | Uses benchmark executables, a raw-output runner, and published redacted local results with limitations. |
| Build verification | CMake options cover tests, examples, tools, apps, benchmarks, sanitizers, clang-tidy, and install/export checks. |
| Area | Included | Not claimed |
|---|---|---|
| Queue | SpscRingBuffer<T, Capacity> and experimental ZeroCopySpsc<T, Capacity>. |
MPSC/MPMC queues or blocking wait primitives. |
| Broker | Broker, BatchBroker, and PersistentBroker local APIs. |
Distributed broker semantics or live inter-process subscriptions. |
| WAL | Fixed-size mmap-backed append/read/replay with CRC32 validation (a checksum used to detect corrupted records). | WAL rotation, repair tooling, schema evolution, or production crash recovery. |
| CLI | aether_bench, aether_pub, aether_sub, aether_replay, aether_inspect_wal. |
Network clients, daemons, auth, TLS, or service discovery. |
| Benchmarks | Benchmark executables, canonical raw-output runner, and redacted local result documentation. | Production guarantees or distributed-system comparisons. |
| CI | Format, build, CTest, sanitizer, clang-tidy, benchmark smoke, and package smoke workflows. | Proof of production readiness. |
| Platform support | Linux/macOS-oriented development path; WSL2 recommended for Windows users. | Fully verified native Windows mmap/test behavior. |
The diagram below traces a call from a CLI app or example down through the broker layer to the queue, WAL, and metrics components, all of which share the same core types.
flowchart LR
User["Developer / CLI user"] --> Apps["CLI apps<br/>aether_bench / pub / sub / replay / inspect_wal"]
User --> Examples["Examples<br/>basic_spsc / broker_basic / persistent_broker"]
Apps --> Broker["Broker APIs<br/>Broker / BatchBroker / PersistentBroker"]
Examples --> Broker
Broker --> Queue["SPSC queue<br/>SpscRingBuffer / ZeroCopySpsc"]
Broker --> Metrics["Metrics<br/>counters / snapshots / latency histogram"]
Broker --> WAL["WAL layer<br/>writer / reader / replay"]
WAL --> Mmap["mmap file layer<br/>MmapFile"]
Queue --> Core["Core types<br/>Status / Expected / Config / MessageView"]
WAL --> Core
Metrics --> Core
Bench["Benchmark suite"] --> Queue
Bench --> Broker
CI["GitHub Actions<br/>CI / sanitizers / benchmark smoke"] --> Build["CMake build<br/>tests / examples / apps / package install"]
The sequence diagram below shows a message being appended to the WAL before it is published to the queue, and a separate reader later replaying that WAL file.
sequenceDiagram
participant P as Producer
participant PB as PersistentBroker
participant W as WalWriter
participant Q as SPSC Queue
participant C as Consumer
participant R as WalReader / Replay
P->>PB: try_publish(event)
PB->>PB: validate broker and queue capacity
PB->>W: append serialized event record
W-->>PB: Status::ok or WAL error
PB->>Q: publish only after WAL append succeeds
Q-->>C: consume event in process
R->>R: open WAL file later
R->>R: validate magic, version, size, checksum
R-->>P: replay typed records for same-program payloads
SpscRingBuffer<T, Capacity> is a header-only queue for exactly one producer and one consumer. It uses monotonic logical counters, power-of-two slot masking, acquire/release publication, cache-line separation, and raw storage lifetime management.
Broker<T, Capacity> provides a local publish/consume wrapper over the queue. BatchBroker<T, Capacity> adds batch-oriented publish/consume APIs. PersistentBroker<T, Capacity> composes the queue with the WAL writer and reader.
The WAL layer stores records with a 40-byte little-endian v1 header, payload bytes, and CRC32 validation. Persistent broker publishing uses WAL-before-queue semantics: a value is published to the in-process queue only after the WAL append succeeds.
The CLI apps demonstrate local benchmark-style broker flow, WAL publishing, typed replay, raw replay, and WAL inspection. aether_sub is a local subscriber/replay demo, not a network subscriber.
Broker metrics use relaxed-atomic counters and immutable snapshots. The latency histogram supports diagnostic and benchmark summaries without introducing an external observability dependency.
Benchmark targets cover SPSC throughput, SPSC latency, payload-size comparison, broker end-to-end flow, batch publishing, zero-copy SPSC, and spin-wait primitives. Reported numbers must come from preserved benchmark evidence.
BatchBroker, experimental ZeroCopySpsc, SpinWait, cpu_relax, and CPU affinity helpers are comparison and exploration APIs. They are not a latency guarantee and do not change the project into a production messaging system.
This is the fast path for a local build, test run, smoke example, and CLI demo. For the complete beginner-friendly setup, testing, CLI, sanitizer, install, benchmark, and troubleshooting walkthrough, see the Full setup and CLI guide.
macOS:
brew install git cmake ninja llvmLinux:
sudo apt-get update
sudo apt-get install -y build-essential cmake ninja-build gitWindows:
Use WSL2, then follow the Linux commands inside the WSL distribution. Native Windows mmap behavior is not the main verified path.
git clone https://github.com/AyushmanRaha/Aether-Stream.git
cd Aether-Stream
cmake -S . -B build/debug -G Ninja \
-DCMAKE_BUILD_TYPE=Debug \
-DAETHER_BUILD_TESTS=ON \
-DAETHER_BUILD_EXAMPLES=ON \
-DAETHER_BUILD_TOOLS=ON \
-DAETHER_BUILD_APPS=ON \
-DAETHER_BUILD_BENCHMARKS=OFF
cmake --build build/debug
ctest --test-dir build/debug --output-on-failure
./build/debug/examples/smokeExpected smoke output:
Aether-Stream 0.1.0
A successful CTest run should end with 100% tests passed, 0 tests failed out of 20.
cmake -S . -B build/release -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DAETHER_BUILD_TESTS=ON \
-DAETHER_BUILD_EXAMPLES=ON \
-DAETHER_BUILD_TOOLS=ON \
-DAETHER_BUILD_APPS=ON
cmake --build build/release
mkdir -p data
./build/release/apps/aether_bench --messages 100000 --payload-size 64 --capacity 1024
./build/release/apps/aether_pub --wal data/sample.wal --messages 1000
./build/release/apps/aether_inspect_wal --wal data/sample.wal
./build/release/apps/aether_replay --wal data/sample.wal --limit 10
./build/release/apps/aether_sub --wal data/sample.wal --limit 10Notes:
aether_benchis a local demo benchmark, not official performance data.aether_pubcreates a local WAL file atdata/sample.wal.aether_inspect_wal,aether_replay, andaether_sub --walinspect or replay that WAL.- There is no networking, daemon, or live remote broker subscription.
For the full chronological workflow, command-by-command explanations, sanitizer checks, package install smoke test, benchmark workflow, and troubleshooting notes, read the Full setup and CLI guide.
Aether-Stream includes local benchmark executables for SPSC throughput, SPSC latency, payload-size behavior, broker end-to-end flow, batch publishing, zero-copy SPSC, and spin-wait primitives. See the benchmark methodology for the publication rules.
Published local benchmark results are available in Performance results, with the detailed consolidated evidence in M1 MacBook Air benchmark run — 2026-06-29. The detailed file preserves the benchmark values in a single redacted Markdown transcript rather than adding separate raw .txt or .json files.
These are local synthetic measurements from a redacted Apple M1 MacBook Air run. They are useful for understanding implementation tradeoffs, but they are not production guarantees and do not measure networking, distributed messaging, or cross-process service behavior.
./scripts/run_benchmarks.sh| Document | Purpose |
|---|---|
| Repository guide | Current repository layout, targets, options, and verification map. |
| Concepts guide | Technical primer for the main C++20, SPSC, WAL, broker, metrics, benchmark, and build concepts. |
| Architecture | System overview and component flow diagrams. |
| Ring buffer design | SPSC queue API, slot lifecycle, memory ordering, and tests. |
| Memory ordering | Acquire/release rationale for queue publication and reuse. |
| mmap notes | POSIX-oriented MmapFile behavior and non-goals. |
| WAL format | Record layout, checksum policy, reader behavior, and replay limits. |
| Broker API | In-memory, batch, and persistent broker usage and limitations. |
| CLI guide | Complete beginner-friendly setup, CLI usage, testing, sanitizer, install, benchmark, and troubleshooting walkthrough. |
| Metrics | Counters, snapshots, latency histogram, and CLI summaries. |
| Benchmark methodology | How to run, preserve, and publish benchmark results honestly. |
| Performance results | Summarized local benchmark results and caveats. |
| M1 MacBook Air benchmark run — 2026-06-29 | Consolidated redacted benchmark evidence for the published local run. |
| Low-latency tuning | Practical tuning notes for batching, zero-copy, spin waits, and affinity. |
| Low-latency design notes | Design tradeoffs and limits for latency-oriented primitives. |
| Limitations | Explicit project boundaries and unsupported behavior. |
| Release checklist | Pre-tag verification checklist. |
| Check | Command or workflow |
|---|---|
| Format | ./scripts/format_all.sh --check |
| Debug tests | cmake -S . -B build/debug -G Ninja -DCMAKE_BUILD_TYPE=Debug -DAETHER_BUILD_TESTS=ON -DAETHER_BUILD_EXAMPLES=ON -DAETHER_BUILD_TOOLS=ON -DAETHER_BUILD_APPS=ON && cmake --build build/debug && ctest --test-dir build/debug --output-on-failure |
| ASAN/UBSAN | cmake -S . -B build/asan -G Ninja -DCMAKE_BUILD_TYPE=Debug -DAETHER_BUILD_TESTS=ON -DAETHER_ENABLE_ASAN=ON -DAETHER_ENABLE_UBSAN=ON && cmake --build build/asan && ctest --test-dir build/asan --output-on-failure |
| TSAN | cmake -S . -B build/tsan -G Ninja -DCMAKE_BUILD_TYPE=Debug -DAETHER_BUILD_TESTS=ON -DAETHER_ENABLE_TSAN=ON && cmake --build build/tsan && ctest --test-dir build/tsan --output-on-failure |
| Package install | cmake -S . -B build/package -G Ninja -DCMAKE_BUILD_TYPE=Release -DAETHER_BUILD_TESTS=ON -DAETHER_ENABLE_INSTALL=ON && cmake --build build/package && ctest --test-dir build/package --output-on-failure && cmake --install build/package --prefix install/aether |
| CI workflow | .github/workflows/ci.yml |
| Sanitizer workflow | .github/workflows/sanitizer.yml |
| Benchmark smoke workflow | .github/workflows/benchmark-smoke.yml |
- The queue APIs are SPSC only.
- There is no networking, daemon, cluster, live IPC broker service, replication, auth, TLS, or service discovery.
- WAL storage is fixed-size and local; there is no rotation, compaction, repair, or production recovery tooling.
- Typed replay is intended for trivially copyable same-program/same-platform payloads.
- CPU affinity behavior is platform-dependent.
- Native Windows support is not presented as fully verified; WSL2 is the recommended Windows path.
- Benchmark smoke checks and stress tests are not official performance results.
- Published benchmark results are local synthetic measurements with redacted environment details; they are not production guarantees.
Glossary for newcomers (click to expand)
- SPSC (single-producer/single-consumer): exactly one thread may produce and one may consume. See docs/ring-buffer-design.md.
- Lock-free: progress is made using atomic operations instead of OS-level locks.
- WAL (write-ahead log): a local append-only file that records each message before it becomes visible to the consumer. See docs/wal-format.md.
- mmap: an OS call that maps a file into memory so it can be read and written like an array instead of through read()/write() calls. See docs/mmap-notes.md.
- RAII: a C++ pattern where a resource is acquired in a constructor and automatically released in a destructor.
- CRC32: a checksum algorithm used to detect corrupted WAL records.
- Broker: the local publish/consume API layered on top of the SPSC queue. See docs/broker-api.md.
- Replay: reading a WAL file back from disk to reconstruct the events it contains.
- Sanitizer (ASAN/UBSAN/TSAN): a compiler-instrumented build that detects memory errors, undefined behavior, or data races at runtime.
- CTest: the test runner bundled with CMake, used to run this project's automated test suite.
For a complete expanded tree, see docs/project-structure.md.
.
├── apps/ CLI applications
├── benchmarks/ Benchmark executables
├── cmake/ Reusable CMake modules
├── docs/ Architecture, API, benchmark, and limitation docs
├── examples/ Small runnable examples
├── include/aether/ Public C++20 headers
├── scripts/ Local build, test, format, and benchmark helpers
├── src/ Library implementation files
├── tests/ CTest executables
└── tools/ Manual validation tools
See CONTRIBUTING.md. Keep changes focused, run relevant checks, preserve benchmark honesty, and update documentation when behavior or limitations change.
Aether-Stream is released under the MIT License. See LICENSE.
This project uses C++20, CMake, Ninja-compatible build flows, CTest, the Google Benchmark library, clang-format, clang-tidy, sanitizers, and GitHub Actions to demonstrate practical systems engineering habits around a deliberately scoped local message-broker toolkit.