feat(bench): pin ferrflow to single thread + report parallel stat with runner cores#145
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #144. Depends on FerrLabs/FerrFlow#597 (the
--jobsflag).What
Since FerrLabs/FerrFlow#476 made per-package planning parallel, the competitive benchmark's ferrflow numbers scaled with the runner's core count — non-reproducible and unfair against single-threaded competitors.
--jobs 1, so its numbers are single-threaded, reproducible, and apples-to-apples with semantic-release / changesets / standard-version / commit-and-tag-version.ferrflow_parallel(kept out of thebenchmarksobject and the comparison table), with a top-levelrunner_coresso the real-world speedup is visible without distorting the fair comparison.latest.json additions
{ "runner_cores": 4, "benchmarks": { /* ferrflow pinned to --jobs 1, unchanged shape */ }, "ferrflow_parallel": { "mono-large-release-dry": { "median_ms": 72.0, "stddev_ms": 1.2 } } }Parallel raw files live in
raw/parallel/so they don't leak into the existingbenchmarksglob or the markdown table. Additive only —compare.sh/format-release.shand their bats tests are unaffected (bash -nclean). The ferrlabs.com perf page rendering offerrflow_parallel+runner_coresfollows in a FerrFlow-Cloud PR.