Skip to content

feat(bench): pin ferrflow to single thread + report parallel stat with runner cores#145

Merged
BryanFRD merged 1 commit into
mainfrom
feat/single-thread-fair-bench
Jun 17, 2026
Merged

feat(bench): pin ferrflow to single thread + report parallel stat with runner cores#145
BryanFRD merged 1 commit into
mainfrom
feat/single-thread-fair-bench

Conversation

@BryanFRD

Copy link
Copy Markdown
Contributor

Closes #144. Depends on FerrLabs/FerrFlow#597 (the --jobs flag).

What

Since FerrLabs/FerrFlow#476 made per-package planning parallel, the competitive benchmark's ferrflow numbers scaled with the runner's core count — non-reproducible and unfair against single-threaded competitors.

  • The competitive comparison now runs ferrflow with --jobs 1, so its numbers are single-threaded, reproducible, and apples-to-apples with semantic-release / changesets / standard-version / commit-and-tag-version.
  • A second, all-cores pass runs per fixture/command and is recorded separately under ferrflow_parallel (kept out of the benchmarks object and the comparison table), with a top-level runner_cores so the real-world speedup is visible without distorting the fair comparison.

latest.json additions

{
  "runner_cores": 4,
  "benchmarks": { /* ferrflow pinned to --jobs 1, unchanged shape */ },
  "ferrflow_parallel": {
    "mono-large-release-dry": { "median_ms": 72.0, "stddev_ms": 1.2 }
  }
}

Parallel raw files live in raw/parallel/ so they don't leak into the existing benchmarks glob or the markdown table. Additive only — compare.sh / format-release.sh and their bats tests are unaffected (bash -n clean). The ferrlabs.com perf page rendering of ferrflow_parallel + runner_cores follows in a FerrFlow-Cloud PR.

@BryanFRD BryanFRD enabled auto-merge (squash) June 17, 2026 11:21
@BryanFRD BryanFRD merged commit 8c99a93 into main Jun 17, 2026
12 checks passed
@BryanFRD BryanFRD deleted the feat/single-thread-fair-bench branch June 17, 2026 11:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bench: pin ferrflow to single thread for fair comparison + report parallel stat

1 participant