Skip to content

chore: add startup benchmark script#30

Open
sacchen wants to merge 1 commit into
mainfrom
chore/startup-bench
Open

chore: add startup benchmark script#30
sacchen wants to merge 1 commit into
mainfrom
chore/startup-bench

Conversation

@sacchen

@sacchen sacchen commented Apr 11, 2026

Copy link
Copy Markdown
Owner

Summary

  • Adds scripts/bench.sh to measure warm startup latency (p50/p95 over 12 runs)
  • Covers four invocations: :version, 2+2, d(x^2,x), solve(x^2-4,x)

Current baseline (this machine)

:version                      p50=  223ms  p95=  246ms
2+2                           p50=  225ms  p95=  231ms
d(x^2, x)                     p50=  292ms  p95=  323ms
solve(x^2-4, x)               p50=  243ms  p95=  285ms

Findings from profiling (see issue #21 comment)

The dominant cost is sympy.polys (~118ms of SymPy's ~245ms total), loaded because solve needs polynomial algorithms. The urllib.request claim in the issue is not meaningful — its import is <0.1ms and the network call is already guarded by isatty().

Realistic floor with current feature set: ~200ms. The <150ms target in #21 is not achievable without removing solve from the namespace.

Closes #21 investigation phase.

Measures warm p50/p95 over 12 runs for four representative invocations.
Use before/after any startup optimization to capture baselines.

Closes #21 investigation phase. See issue for profiling findings.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Benchmark and optimize phil startup latency

1 participant