Skip to content

Publish a measured performance report with a regeneration script#124

Merged
weiminglong merged 1 commit into
mainfrom
feat/performance-report
Jul 6, 2026
Merged

Publish a measured performance report with a regeneration script#124
weiminglong merged 1 commit into
mainfrom
feat/performance-report

Conversation

@weiminglong

Copy link
Copy Markdown
Owner

Summary

The repo claimed performance characteristics without a single attributed measurement. Now:

  • docs/PERFORMANCE.md — measured Criterion medians for every hot-path stage (wire deserialize, dispatcher normalize + shadow-book cross-check, WAL codec encode / append+flush / fdatasync-per-record, book delta/snapshot/top-of-book, read-model snapshot), stamped with CPU, RAM, OS, toolchain, commit, and date, plus the full 49-benchmark table. Headline medians from the committed run (Apple M3, 8 cores): book delta apply 9 ns, top-of-book read 1 ns, WAL codec encode 87 ns, WAL append+flush 509 ns/record, fdatasync-every-record 4.04 ms, dispatcher normalize 484 ns/entry (2.1 M entries/s), read-model snapshot 154 ns.
  • scripts/bench-report.py — regenerates the report from target/criterion estimates (batch benches get a per-item column via encoded batch sizes); just bench-report runs the suite and rewrites the file.
  • README's benchmarks section now leads with the measured numbers and links the report.

The report is explicit about methodology limits: single-machine wall-clock medians for order-of-magnitude reasoning; CI compiles benches but does not gate on timings.

Verification

  • Generated from a full cargo bench --workspace run on current main (49 benchmarks collected)
  • Script re-run is idempotent; hygiene gate clean; doc-only + script + justfile changes (no Rust code touched)

docs/PERFORMANCE.md carries Criterion medians for every hot-path stage
— wire deserialization, dispatcher normalize with the shadow-book
cross-check, WAL codec encode and append, book operations, and the
read-model snapshot — stamped with the CPU, RAM, OS, toolchain, commit,
and date they were measured on, plus the full 49-benchmark result table.
scripts/bench-report.py regenerates it from target/criterion output and
`just bench-report` wraps the run. The README benchmarks section now
leads with the measured numbers instead of bare commands.

Numbers replace the order-of-magnitude estimates that docs previously
carried without attribution; CI continues to compile (not time) the
benches, since shared runners are too noisy for statistical gating.
@weiminglong
weiminglong force-pushed the feat/performance-report branch from bdc8be6 to 71bace4 Compare July 6, 2026 12:08
@weiminglong
weiminglong merged commit c62b64c into main Jul 6, 2026
19 checks passed
@weiminglong
weiminglong deleted the feat/performance-report branch July 6, 2026 12:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant