SUCO

SUper COmpiler Grid – Distributed C/C++ Compilation and Caching System for Local Networks.

Compile once. Cache forever. Distributed builds with zero configuration.

---

⚡ At a Glance

SUCO is a high-performance, lightweight alternative to expensive proprietary solutions like IncrediBuild or legacy systems like Icecream/distcc. It is designed for maximum speed with minimal setup:

• 🌐 Heterogeneous Multi-Node Cross-Compilation – A Windows client (suco-cl++.exe) dispatches compilation tasks to parallel Linux worker nodes running MinGW cross-compilers (x86_64-w64-mingw32-g++), returning native PE-x86-64 binary objects.
• 🤖 CI-Verified Multi-Node Grid – Tested in GitHub Actions CI via a dedicated multi-runner network workflow (multi-runner-grid.yml) verifying multi-node distribution across isolated virtual hosts.
• 📈 Prometheus & Grafana Telemetry (:9001/metrics) – Open HTTP metrics exporter supplying real-time job counts, cache hit rates, active worker slots, and coordinator uptime.
• 🖥️ Native Qt 6 Control Center & Windows Installer – Modern desktop GUI (suco-gui.exe) and automated NSIS Windows Installer (suco.nsi) for effortless setup.
• 🔍 Zero-Config Auto-Discovery – Workers automatically discover the coordinator via UDP Broadcast.
• 💾 Intelligent SSD Cache – SHA-256-based LRU cache with metadata tracking and versioned keys.
• 📊 Live Web Dashboard – Real-time monitoring of all workers, CPU cores, cache hit rate, and compile history.
• 🚀 Direct Data Path (Direct Dispatch) – Compilation data streams directly from clients to workers, avoiding coordinator bottlenecks.
• ⚖️ Load-Aware CPU Scheduling – Heartbeat-based CPU load monitoring dynamically scores workers to prevent node saturation.
• 🔄 Least-Recently-Assigned Scheduling – Fair Round-Robin tie-breaking distributes parallel compile threads uniformly.
• 🛡️ Transparent Grid Failover – If a worker goes offline, the coordinator immediately reschedules the jobs.
• ↩️ Resilient Client Fallback – If the coordinator fails, the client seamlessly falls back to local compilation in <100ms.
• 🛠️ CMake & IDE Integration – Easy integration via SUCO.cmake and automatic compile_commands.json wrapper prefix cleaning.
• 🧼 Grid-Wide Cache Clearing – Clean all local and remote caches via suco cache clear.
• 🤝 Path Normalization – Cross-directory cache hits via path mapping and -ffile-prefix-map integration.

---

🏆 Benchmark Results

🗄️ Real-world: RocksDB (342 translation units, 4-node grid, idle machine)

Head-to-head against Icecream under identical conditions (same session, same flags, -j32), building the full librocksdb.a:

Build System	Cold build	Warm rebuild
🍦 Icecream	101.9s	101.9s (no cache — every rebuild is a full compile)
🚀 SUCO	100.7s (parity)	24.7s (4.1× faster)

Cold builds are on par with Icecream; warm rebuilds are ~4× faster because SUCO serves unchanged objects straight from its content-addressed cache instead of recompiling them. Independently reproduced across two idle runs; three consecutive cold builds each completed all 342 objects.

⏱️ Distributed GoogleTest Build Suite (108 C++ Files, -j17)

Tested on a real grid with 4× Nodes (parallelized using -j17):

Build System	Duration	Speedup vs Native	Notes / Description
🖥️ Native g++	50.83s	1.00x (Baseline)	Pure local compilation.
🍦 Icecream	55.34s	0.92x	Distributed compilation routed via iceccd daemon.
❄️ SUCO Cold	84.61s	0.60x	First run on a cleared cache (includes preprocessor & network overhead).
🔥 SUCO Hot	13.25s 🚀	3.84x ⚡	100% Cache Hits. Bypasses compiler phases completely via global SSD caching!

⏱️ Small Project Build Benchmark (40 C++ Files, -j8)

Tested on the same grid (parallelized using -j8):

Build System	Duration	Speedup vs Native	Notes / Description
🖥️ Native g++	1.13s	1.00x (Baseline)	Local native build.
🍦 Icecream	1.22s	0.93x	Distributed via iceccd.
❄️ SUCO Cold	3.42s	0.33x	Initial cold run.
🚀 SUCO Warm	1.12s ⚡	1.01x	Skips compilation entirely and pulls .o objects from coordinator.

Why is the Cache so fast?

Normal Build:             SUCO Cache Hit:
┌──────────────┐         ┌──────────────┐
│ Preprocessor │ ~0.1s   │ Preprocessor │ ~0.1s
├──────────────┤         ├──────────────┤
│ Parser/AST   │ ~3.0s   │ SHA-256 Hash │ ~0.001s
├──────────────┤         ├──────────────┤
│ Optimization │ ~15.0s  │ Cache Lookup │ ~0.002s
├──────────────┤         ├──────────────┤
│ Codegen      │ ~7.0s   │ SSD → Client │ ~0.5s
└──────────────┘         └──────────────┘
     ~25s                    ~0.7s

The client performs only the preprocessing phase locally and computes a SHA-256 hash. On a cache hit, CPU-intensive compiler phases (parsing, optimization, codegen) are completely skipped — the finished .obj file is fetched directly from the SSD cache.

---

🏗️ Architecture

SUCO consists of several native C++20 components:

                   ┌────────────────────────────────────────────────────────┐
                   │                     DEVELOPER MACHINE                  │
                   │  ┌───────────────┐                                     │
                   │  │     suco      │ (Entry Point / Build Wrapper)       │
                   │  └───────┬───────┘                                     │
                   │          │ Sets CC/CXX to suco-cl/suco-cl++            │
                   │          ▼                                             │
                   │  ┌───────────────┐     ┌────────────────────────────┐  │
                   │  │ suco-cl/cl++  │────▶│      SUCO COORDINATOR      │  │
                   │  │ (Compiler Wrp)│◀───│  ┌──────────┐ ┌──────────┐  │  │
                   │  └───────────────┘     │  │SSD Cache │ │Dashboard │  │  │
                   │                        │  │(5GB LRU) │ │  :9001   │  │  │
                   │                        │  └──────────┘ └──────────┘  │  │
                   └────────────────────────┴───────┬────────────────────┘  │
                                                    │ TCP :9000
                                    ┌───────────────┼───────────────┐
                                    ▼               ▼               ▼
                           ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
                           │  WORKER #1   │ │  WORKER #2   │ │  WORKER #3   │
                           │  HP Mini G2  │ │  HP Mini G2  │ │  HP Mini G2  │
                           │  4 Cores     │ │  4 Cores     │ │  4 Cores     │
                           └──────────────┘ └──────────────┘ └──────────────┘
                                    UDP Auto-Discovery :9002

Components

Component	Description
suco (Wrapper)	The main build wrapper. Intercepts build commands (make, ninja, cmake), overrides environment variables CC/CXX and delegates to suco-cl/suco-cl++.
suco-cl / suco-cl++	The actual compiler wrappers. Preprocesses source files locally, calculates SHA-256 hashes and interacts with the coordinator.
suco-coordinator	Central grid hub. Manages the SSD LRU cache, distributes jobs (least-loaded scheduling), and hosts the live web dashboard.
suco-worker	Compilation node. Registers via UDP, compiles preprocessed sources, and returns compiled binaries.

🛠️ Wrapper Subcommands

The suco wrapper includes a clean OOP-based subcommand system:

• suco setup: Interactive setup assistant (configures Coordinator host, port, slots, log level, detects local compilers and performs connection verification).
• suco status: Shows grid coordinator state, cache hit rate, and real-time slots usage.
• suco workers: Lists all registered worker nodes grouped by compilers, tools and Qt versions.
• suco cache clear: Clears local caches and triggers PCH/object cache cleanup across the entire grid.
• suco config show: Displays currently active client configurations and paths.
• suco help: Prints usage instructions for available subcommands.

Cache Key Format

The cache uses a versioned, metadata-rich hash key:

v3:⟨0x1F⟩Target⟨0x1F⟩CompilerVersion⟨0x1F⟩Standard⟨0x1F⟩Defines⟨0x1F⟩Includes⟨0x1F⟩Flags⟨0x1F⟩NormalizedSource

• Versioning: v3: prefix for schema migrations (updated for path normalization and compression flags).
• Metadata: Target architecture, compiler version, language standard, sorted defines & include paths.
• Source Normalization: Strips #line directives, empty lines, and normalizes paths (optimized using memchr for 300k+ lines in <5ms).
• Separator: ASCII 0x1F (Unit Separator) — guaranteed not to conflict with normal code contents.

📦 Precompiled Headers (PCH) Support

SUCO includes a hybrid PCH processing mechanism:

• PCH Detection: Parses common PCH options for MSVC (/Yu, /Yc, /Y-, /Fp) and GCC/Clang (-include-pch, -fpch-preprocess, .gch, .pch).
• Hybrid Distribution:
  • PCH Creation (e.g. compiling stdafx.h to .gch): Executed locally on the client to avoid copying large header sets over the network.
  • PCH Usage (compiling .cpp source files using the PCH): Executed remotely in the grid. The client strips PCH flags and preprocessor directives, and uploads fully expanded source code to ensure portability on grid workers.

⚡ Thread-Safe Logging

Client, coordinator, and worker use a unified thread-safe logging library:

• Granular Log Levels: Configured via the SUCO_LOG_LEVEL environment variable (DEBUG, INFO, WARN, ERROR - default is INFO).
• Performance Optimized: Inline compile-time guards prevent std::format string construction for disabled log levels.
• Timestamps: Console output is structured as: [YYYY-MM-DD HH:MM:SS] [LEVEL] Message.

---

🐧 Linux: Official APT Repository (Ubuntu / Debian)

Install directly from the official SUCO APT repository:

# 1. Download & register the official GPG keyring
curl -fsSL https://micbur.github.io/suco/suco-archive-keyring.asc | sudo tee /etc/apt/keyrings/suco.asc >/dev/null

# 2. Add the SUCO APT repository source list
echo "deb [signed-by=/etc/apt/keyrings/suco.asc] https://micbur.github.io/suco stable main" | sudo tee /etc/apt/sources.list.d/suco.list >/dev/null

# 3. Update and install
sudo apt update && sudo apt install -y suco

Dependencies

Platform	Packages
Linux	build-essential, cmake, libssl-dev, libzstd-dev
Windows (MinGW, recommended)	MinGW-w64 GCC ≥ 13 (e.g. MSYS2 or the Qt toolchain), CMake ≥ 3.15, Ninja, OpenSSL, SQLite3, zstd
Windows (MSVC)	Visual Studio Build Tools (MSVC), CMake ≥ 3.15, OpenSSL + zstd + sqlite3 (via vcpkg)

Windows: prebuilt binaries

Don't want to build? Grab the self-contained suco-<version>-windows-x64.zip from the latest release — the six .exe plus the runtime DLLs they need, no toolchain required. Run clients with SUCO_NO_DAEMON=1.

Building

# Linux / WSL
cmake -B build_linux -S . -DCMAKE_BUILD_TYPE=Release
cmake --build build_linux

# Windows (MSYS2 MinGW64 shell)
pacman -S --needed mingw-w64-x86_64-gcc mingw-w64-x86_64-cmake mingw-w64-x86_64-ninja \
                   mingw-w64-x86_64-openssl mingw-w64-x86_64-sqlite3 mingw-w64-x86_64-zstd
cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build

# Windows (Qt MinGW toolchain, PowerShell) — build zstd once into thirdparty/, see docs
cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH="C:/Qt/Tools/mingw1310_64/opt"
cmake --build build --config Release

# Windows (MSVC) — vcpkg install openssl zstd sqlite3 --triplet x64-windows first
cmake -B build_msvc -S . -G "Visual Studio 17 2022" -A x64 -DCMAKE_TOOLCHAIN_FILE=C:/vcpkg/scripts/buildsystems/vcpkg.cmake
cmake --build build_msvc --config Release

🪟 Windows notes: run clients with SUCO_NO_DAEMON=1 (the IPC daemon uses Unix sockets). The header-set/PCH optimization works on Windows: MinGW system headers are recognized, workers cache the header text, build PCHs and serve stripped TUs (~KB instead of MB payloads). Windows clients dispatch MinGW jobs under the target-qualified name x86_64-w64-mingw32-g++, so Linux workers can serve them once mingw-w64 is installed there — nodes without it are skipped safely (the client compiles locally).

Installation (Linux)

📖 Full setup walkthrough (grid, verify, use, auth, troubleshooting): docs/INSTALL.md

Option A — one line (recommended): sets up the signed APT repo and installs suco. Works on Debian and Ubuntu.

curl -fsSL https://micbur.github.io/suco/install.sh | sudo sh

Then enable the role each node should play (nothing starts automatically):

sudo systemctl enable --now suco-worker                    # compile node
sudo systemctl enable --now suco-coordinator suco-worker   # head node
suco --version                                             # verify

Option A′ — the same thing by hand (if you'd rather not pipe to a shell)

curl -fsSL https://micbur.github.io/suco/suco-archive-keyring.asc \
  | sudo tee /etc/apt/keyrings/suco.asc >/dev/null
echo "deb [signed-by=/etc/apt/keyrings/suco.asc] https://micbur.github.io/suco stable main" \
  | sudo tee /etc/apt/sources.list.d/suco.list >/dev/null
sudo apt update && sudo apt install suco

Installs binaries to /usr/bin/, systemd units to /usr/lib/systemd/system/ (shipped disabled — a fresh install never silently joins a running grid), dashboard to /usr/share/suco/. To enable grid-wide auth, set SUCO_SECRET in the unit (or an EnvironmentFile) on every node and restart.

Option B — build the .deb yourself:

cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build -j
( cd build && cpack -G DEB )
sudo apt install ./build/suco_*_amd64.deb

Option B — interactive installer:

sudo bash install.sh
# Choose: 1) Coordinator + Worker   2) Worker Only

The installer installs binaries to /usr/local/bin/, registers systemd daemons, and opens ports in the firewall.

---

⚙️ Usage

1. Start Coordinator

./suco-coordinator
# Dashboard: http://localhost:9001
# Cache: ~/.cache/suco/ (Default: 5 GB SSD cache with LRU eviction)

2. Start Worker (on grid nodes)

# Auto-Discovery (discovers coordinator automatically via UDP)
./suco-worker --slots 4

# Manual Address config
./suco-worker --coordinator 192.168.0.200:9000 --slots 8

3. Compile via SUCO

# Linux (GCC)
suco g++ -O3 -std=c++20 -c myfile.cpp -o myfile.o

# Windows (MinGW, PowerShell)
$env:SUCO_NO_DAEMON = "1"
.\suco-cl++.exe -O2 -std=c++20 -c myfile.cpp -o myfile.o

# Windows (MSVC)
suco cl.exe /O2 /EHsc /c myfile.cpp /Fo myfile.obj

# In CMake Lists
set(CMAKE_CXX_COMPILER_LAUNCHER suco)

---

📊 Live Web Dashboard

The built-in dashboard on port 9001 visualizes:

• 🟢 Worker Nodes Status and CPU core utilization.
• 📈 Cache Hit Rate via animated circular gauges.
• ⚡ Active Compilations with runtime indicators.
• 📋 Job History with hit/miss indicators.

---

🔧 Network Ports

Port	Protocol	Purpose
9000	TCP	Remote compilations (Client ↔ Coordinator ↔ Worker)
9001	TCP	Live dashboard and REST stats API
9002	UDP	Grid auto-discovery broadcasts

---

🤝 Comparison with Alternatives

Feature	SUCO	IncrediBuild	Icecream	distcc
Price	✅ Free	❌ ~$2000/license	✅ Free	✅ Free
SSD Caching	✅ SHA-256 LRU	✅ Proprietary	❌ No	❌ No
Auto-Discovery	✅ UDP Broadcast	✅ Central Broker	✅ mDNS	❌ Manual config
Web Dashboard	✅ Glassmorphism	✅ GUI Client	⚠️ Icemon	❌ No
Windows + Linux	✅ Native	⚠️ Windows-only	✅ Yes	✅ Yes
Zero-Config Worker	✅ Yes	❌ Agent needed	✅ Yes	❌ Manual config
Resilient Fallback	✅ <100ms	⚠️ Slow fallback	❌ No	❌ No
Setup Time	✅ ~2 mins	❌ ~30 mins	⚠️ ~10 mins	⚠️ ~15 mins

🗺️ Roadmap & Milestones

• ✅ v0.12.0 (Current / Released): Prometheus & Grafana Telemetry Exporter (:9001/metrics), Multi-Node Distributed CI Verification Grid Workflow (multi-runner-grid.yml), Heterogeneous Windows-to-Linux Parallel Cross-Compilation, Linux Service Deployment (k3master), Native Qt 6 Desktop Control Center (suco-gui.exe).
• 🚀 v1.0.0 (Production Target): Worker Sandboxing (bubblewrap), Remote Preprocessing, TLS/mTLS Encryption, macOS ARM64 Support.

For detailed technical specifications, view the full Project Roadmap and Portfolio Showcase.

---

📜 License

MIT License – See LICENSE for details.

---

_{Created by Michael Burzlaff. Developed with ⚡ by MicBur}