Skip to content

Fix compute_timer_precision to not always take 1 second#307

Merged
ionelmc merged 1 commit intoionelmc:masterfrom
akx:faster-calibration
Mar 26, 2026
Merged

Fix compute_timer_precision to not always take 1 second#307
ionelmc merged 1 commit intoionelmc:masterfrom
akx:faster-calibration

Conversation

@akx
Copy link
Copy Markdown
Contributor

@akx akx commented Mar 25, 2026

I was viztracering pytest-benchmark (because I had given up on fastidiously running tests locally because it was slow 😄) and noticed compute_timer_precision() always took a suspicious exactly-one-second.

Looks like the code had a little logic issue so it always took up to the timeout, even if there were enough data points to go on with.

This PR proposes to fix that logic bug (and instead of or), and then increases the minimum number of data points to 100.

(master) $ python3 -m timeit -s 'from pytest_benchmark.timers import compute_timer_precision, default_timer' 'compute_timer_precision(default_timer)'
1 loop, best of 5: 1e+03 msec per loop
(faster-calibration) $ python3 -m timeit -s 'from pytest_benchmark.timers import compute_timer_precision, default_timer' 'compute_timer_precision(default_timer)'
10000 loops, best of 5: 22.7 usec per loop

This means that for each of the pytest subprocesses spawned by the test suite, there's about 1 second less of waiting...

@akx akx force-pushed the faster-calibration branch from 8f84b70 to 247a4cf Compare March 25, 2026 19:28
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 25, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.72%. Comparing base (cf97f26) to head (247a4cf).
⚠️ Report is 5 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #307   +/-   ##
=======================================
  Coverage   90.72%   90.72%           
=======================================
  Files          28       28           
  Lines        3052     3052           
  Branches      336      336           
=======================================
  Hits         2769     2769           
  Misses        208      208           
  Partials       75       75           
Flag Coverage Δ
py310-pytest84-nodist-cover (macos/arm64) 86.40% <0.00%> (ø)
py310-pytest84-nodist-cover (ubuntu/x64) 86.89% <0.00%> (ø)
py310-pytest84-nodist-cover (windows/x64) 86.17% <0.00%> (ø)
py310-pytest84-xdist-cover (macos/arm64) 86.73% <0.00%> (ø)
py310-pytest84-xdist-cover (ubuntu/x64) 87.02% <0.00%> (-0.20%) ⬇️
py310-pytest84-xdist-cover (windows/x64) 86.50% <0.00%> (ø)
py310-pytest90-nodist-cover (macos/arm64) 86.36% <0.00%> (ø)
py310-pytest90-nodist-cover (ubuntu/x64) 86.86% <0.00%> (+0.19%) ⬆️
py310-pytest90-nodist-cover (windows/x64) 86.14% <0.00%> (+0.19%) ⬆️
py310-pytest90-xdist-cover (macos/arm64) 86.69% <0.00%> (?)
py310-pytest90-xdist-cover (ubuntu/x64) 86.99% <0.00%> (ø)
py310-pytest90-xdist-cover (windows/x64) 86.46% <0.00%> (ø)
py311-pytest84-nodist-cover (macos/arm64) 86.40% <0.00%> (ø)
py311-pytest84-nodist-cover (ubuntu/x64) 86.89% <0.00%> (+0.19%) ⬆️
py311-pytest84-nodist-cover (windows/x64) 86.17% <0.00%> (ø)
py311-pytest84-xdist-cover (macos/arm64) 86.73% <0.00%> (ø)
py311-pytest84-xdist-cover (ubuntu/x64) 87.02% <0.00%> (-0.04%) ⬇️
py311-pytest84-xdist-cover (windows/x64) 86.30% <0.00%> (ø)
py311-pytest90-nodist-cover (macos/arm64) 86.36% <0.00%> (ø)
py311-pytest90-nodist-cover (ubuntu/x64) 86.89% <0.00%> (+0.22%) ⬆️
py311-pytest90-nodist-cover (windows/x64) 85.94% <0.00%> (-0.20%) ⬇️
py311-pytest90-xdist-cover (macos/arm64) 86.69% <0.00%> (ø)
py311-pytest90-xdist-cover (ubuntu/x64) 87.18% <0.00%> (-0.04%) ⬇️
py311-pytest90-xdist-cover (windows/x64) 86.46% <0.00%> (+0.19%) ⬆️
py312-pytest84-nodist-cover (macos/arm64) 86.40% <0.00%> (ø)
py312-pytest84-nodist-cover (ubuntu/x64) 86.89% <0.00%> (ø)
py312-pytest84-nodist-cover (windows/x64) 85.97% <0.00%> (-0.20%) ⬇️
py312-pytest84-xdist-cover (macos/arm64) 86.73% <0.00%> (ø)
py312-pytest84-xdist-cover (ubuntu/x64) 87.05% <0.00%> (ø)
py312-pytest84-xdist-cover (windows/x64) 86.50% <0.00%> (+0.19%) ⬆️
py312-pytest90-nodist-cover (macos/arm64) 86.36% <0.00%> (ø)
py312-pytest90-nodist-cover (ubuntu/x64) 86.69% <0.00%> (ø)
py312-pytest90-nodist-cover (windows/x64) 86.14% <0.00%> (+0.19%) ⬆️
py312-pytest90-xdist-cover (macos/arm64) 86.69% <0.00%> (ø)
py312-pytest90-xdist-cover (ubuntu/x64) 87.02% <0.00%> (ø)
py312-pytest90-xdist-cover (windows/x64) 86.46% <0.00%> (ø)
py313-pytest84-nodist-cover (macos/arm64) 86.20% <0.00%> (-0.20%) ⬇️
py313-pytest84-nodist-cover (ubuntu/x64) 86.92% <0.00%> (+0.22%) ⬆️
py313-pytest84-nodist-cover (windows/x64) 85.97% <0.00%> (-0.20%) ⬇️
py313-pytest84-xdist-cover (macos/arm64) 86.73% <0.00%> (ø)
py313-pytest84-xdist-cover (ubuntu/x64) 87.22% <0.00%> (ø)
py313-pytest84-xdist-cover (windows/x64) 86.50% <0.00%> (ø)
py313-pytest90-nodist-cover (macos/arm64) 86.17% <0.00%> (-0.20%) ⬇️
py313-pytest90-nodist-cover (ubuntu/x64) 86.89% <0.00%> (ø)
py313-pytest90-nodist-cover (windows/x64) 85.94% <0.00%> (ø)
py313-pytest90-xdist-cover (macos/arm64) 86.69% <0.00%> (ø)
py313-pytest90-xdist-cover (ubuntu/x64) 87.22% <0.00%> (+0.19%) ⬆️
py313-pytest90-xdist-cover (windows/x64) 86.27% <0.00%> (ø)
py314-pytest84-nodist-cover (macos/arm64) 89.60% <100.00%> (?)
py314-pytest84-nodist-cover (ubuntu/x64) 90.00% <100.00%> (-0.10%) ⬇️
py314-pytest84-nodist-cover (windows/x64) 89.28% <100.00%> (-0.07%) ⬇️
py314-pytest84-xdist-cover (macos/arm64) 89.74% <100.00%> (?)
py314-pytest84-xdist-cover (ubuntu/x64) 90.13% <100.00%> (-0.27%) ⬇️
py314-pytest84-xdist-cover (windows/x64) 89.67% <100.00%> (ø)
py314-pytest90-nodist-cover (macos/arm64) 89.60% <100.00%> (ø)
py314-pytest90-nodist-cover (ubuntu/x64) 89.83% <100.00%> (-0.04%) ⬇️
py314-pytest90-nodist-cover (windows/x64) 89.34% <100.00%> (+0.19%) ⬆️
py314-pytest90-xdist-cover (macos/arm64) 89.93% <100.00%> (ø)
py314-pytest90-xdist-cover (ubuntu/x64) 90.23% <100.00%> (-0.20%) ⬇️
py314-pytest90-xdist-cover (windows/x64) 89.67% <100.00%> (+0.19%) ⬆️
pypy311-pytest84-nodist-cover (macos/arm64) 85.58% <0.00%> (ø)
pypy311-pytest84-nodist-cover (ubuntu/x64) 85.87% <0.00%> (ø)
pypy311-pytest84-nodist-cover (windows/x64) 85.15% <0.00%> (ø)
pypy311-pytest84-xdist-cover (macos/arm64) 85.91% <0.00%> (ø)
pypy311-pytest84-xdist-cover (ubuntu/x64) 86.23% <0.00%> (ø)
pypy311-pytest84-xdist-cover (windows/x64) 85.48% <0.00%> (ø)
pypy311-pytest90-nodist-cover (macos/arm64) 85.58% <0.00%> (?)
pypy311-pytest90-nodist-cover (ubuntu/x64) 85.87% <0.00%> (-0.04%) ⬇️
pypy311-pytest90-nodist-cover (windows/x64) 85.35% <0.00%> (ø)
pypy311-pytest90-xdist-cover (macos/arm64) 85.91% <0.00%> (ø)
pypy311-pytest90-xdist-cover (ubuntu/x64) 86.43% <0.00%> (+0.19%) ⬆️
pypy311-pytest90-xdist-cover (windows/x64) 85.68% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@akx
Copy link
Copy Markdown
Contributor Author

akx commented Mar 25, 2026

This branch:

py314-pytest90-nodist-nocov: OK (447.03=setup[18.04]+cmd[428.99] seconds)
congratulations :) (452.23 seconds)

cf97f26:

py314-pytest90-nodist-nocov: OK (485.05=setup[21.09]+cmd[463.96] seconds)
congratulations :) (486.91 seconds)


Helps a bit!

@ionelmc ionelmc merged commit 908cfc0 into ionelmc:master Mar 26, 2026
221 of 223 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants