#15 - Add reusable C++ bench infrastructure for per-platform performance reports

[RamyaGuru]

Summary

First of a 3-PR stack replacing #72 (DGX Spark performance report). This PR contains only the platform-agnostic bench plumbing — no Spark-specific tooling, no perf doc.

• TokenBucketPacer in examples/raw_bench_common.{cpp,h}, plus a --target-gbps G CLI flag that wires it into the raw / RDMA / socket benches.
• --seconds N CLI flag for time-bounded runs (replaces the previous YAML iterations requirement for smoke runs).
• Atomic stringstream prints so the multi-thread bench threads don't garble their RX complete: / TX complete: lines.
• Drop-count propagation — DPDK manager xstats summary, RDMA CQ error lines, and socket bench sent_packets/sent_bytes keys are all emitted in a grep-friendly format that the upstream sweep wrapper consumes.
• examples/bench_capture_environment.sh — one-shot capture of slow-moving host state (kernel, NIC firmware, IOMMU, hugepages, GPU info) into a results directory.

Used by the upcoming DGX Spark, IGX, and x86 platform performance reports (issues #15, #16, and follow-on platform PRs).

Stack

This PR → #β (Spark sweep tooling) → #γ (Spark perf doc) — opening next once their test plans pass.

Replaces #72 (which had all three concerns bundled into one ~1,545-line draft).

Test plan

• Clean build: cmake -S . -B build -DBUILD_SHARED_LIBS=ON -DDAQIRI_BUILD_PYTHON=OFF -DDAQIRI_MGR="dpdk socket rdma" && cmake --build build -j — no warnings.
• --seconds honored on DPDK QSFP loopback (daqiri_bench_raw_gpudirect against daqiri_bench_raw_tx_rx_spark.yaml) — TX/RX complete lines print cleanly, RX packets > 0, exits in ~5 s.
• TokenBucketPacer rate-limits: --target-gbps 10 measured within ±20% of the target on the same cable.
• Socket bench output format (Server complete: sent_packets=... sent_bytes=...) — deferred to PR β's regression test, which exercises the same field-name path through the sweep wrapper.

🤖 Generated with Claude Code

[RamyaGuru]

Force-pushed to add a missing BUILD_RPATH on daqiri_bench_socket in examples/CMakeLists.txt. The original commit linked raw_bench_common.cpp + CUDA::cudart into the socket bench (matching what was done for daqiri_bench_rdma) but missed the matching BUILD_RPATH property. Without it, the binary runtime-links against /opt/daqiri/lib/libdaqiri.so — the installed snapshot, which can be older than the source tree (it was, in this case: missing Cliff's set_connection_id accessors from #88). Found while running PR #97's regression test, which crashed with an undefined-symbol error until I added the RPATH. daqiri_bench_rdma already had the same property, so this is just bringing socket in line.

[greptile-apps]

Greptile Summary

Adds platform-agnostic bench plumbing: TokenBucketPacer for --target-gbps rate limiting, --seconds time-bounded runs, atomic stats printing, drop-count propagation in structured output fields, and bench_capture_environment.sh for host/NIC/GPU state capture.

• TokenBucketPacer (raw_bench_common.{h,cpp}): software token-bucket pacer wired into DPDK, RDMA, and socket bench TX workers; rdma/socket benches each give server and client their own pacer instance, but raw_gpudirect_bench.cpp shares a single tx_pacer across all TX threads — the mutex is held for the full sleep loop, which serializes parallel TX workers when --target-gbps is active and multiple queues are configured.
• Structured output fields (seconds=, send_bytes=, recv_bytes=, etc.) are emitted consistently across all three benches in a grep-friendly name=value format suitable for the upstream sweep wrapper.
• bench_capture_environment.sh: new one-shot script capturing kernel, NIC firmware, hugepages, IOMMU, and GPU state into a timestamped results directory.

Confidence Score: 4/5

Safe to merge for single-queue configs; multi-queue DPDK TX with --target-gbps will serialize TX threads due to the lock held during sleep.

The pacer mutex is held across the entire sleep loop in wait_for_bytes. In raw_gpudirect_bench.cpp a single tx_pacer is shared among all TX threads, so any multi-queue DPDK bench run with --target-gbps set will block all TX threads behind one sleeping thread rather than pacing them in parallel. The rdma and socket benches are unaffected because each role gets its own pacer. Fixing this requires releasing the mutex before the sleep loop.

examples/raw_bench_common.cpp (wait_for_bytes) and examples/raw_gpudirect_bench.cpp (shared tx_pacer)

Important Files Changed

Filename	Overview
examples/raw_bench_common.cpp	Adds TokenBucketPacer implementation; mutex is held across the sleep loop, serializing threads when a shared pacer is used.
examples/raw_bench_common.h	Declares TokenBucketPacer and updates print_queue_stats signature to include elapsed seconds; clean.
examples/raw_gpudirect_bench.cpp	Wires TokenBucketPacer (shared single instance across all TX threads) and per-thread elapsed timing into tx_worker; pacer serialization bug surfaces here under multi-queue configs.
examples/rdma_bench.cpp	Adds per-role pacers (server/client each own a separate TokenBucketPacer), byte tracking, and structured output; no contention issue since pacers are not shared across threads.
examples/socket_bench.cpp	Adds pacer, byte tracking, and structured output; pre-existing send_tx_burst failure leak and TX vs RX byte accounting mismatch are flagged in prior review threads.
examples/CMakeLists.txt	Adds raw_bench_common.cpp and CUDA::cudart to rdma and socket bench targets; also adds missing BUILD_RPATH for socket bench.
examples/bench_capture_environment.sh	New one-shot host/NIC/GPU state capture script; correctly handles missing commands and uses section headers for easy grepping.

_{Reviews (3): Last reviewed commit: "#15 - Add reusable C++ bench infrastruct..." | Re-trigger Greptile}

[greptile-apps]

TX vs RX byte metrics use different accounting bases

sent_bytes is accumulated as cfg.message_size per packet (user-payload only), while received_bytes uses get_burst_tot_byte(burst) (actual bytes delivered by the DAQIRI burst, which may include framing). In a zero-loss loopback the "Server complete" line will show recv_bytes > sent_bytes, causing the upstream sweep wrapper — which parses both fields to compute efficiency — to report spurious byte loss or negative efficiency. rdma_bench.cpp consistently uses cfg.message_size for both sides; socket_bench should do the same, or switch both to the actual API measurement.

[greptile-apps]

Doc-sync required for new CLI flags and output format

This PR introduces the TokenBucketPacer class, the --target-gbps flag, and rewrites every bench's final summary line to use name=value fields (adding seconds=, send_bytes=, recv_bytes=, etc.). It also adds examples/bench_capture_environment.sh. Per the repo's doc-sync rule, changes under examples/*.cpp require updating docs/tutorials/benchmarking_examples.md, docs/tutorials/configuration-walkthrough.md, and the benchmark table in AGENTS.md. None of those appear to be updated in this PR.

Rule Used: DAQIRI has no automated doc-sync gate beyond mkdoc... (source)