Summary
Implement median-based repeat aggregation for oracletrace run -- pytest, matching the behavior already available when tracing a script directly.
Problem
--repeat is defined on _BASE_PARSER (shared by both entry points) but aggregation logic exists only in _run_target(). When using oracletrace run --repeat 3 -- pytest, pytest runs once and repeat is silently ignored.
Proposed Solution
- Extract the repeat loop +
FunctionAggregate median logic from _run_target() into a shared helper (e.g. _aggregate_repeated_runs())
- Update
_run_pytest() to invoke pytest N times when --repeat > 1 and aggregate trace data
- Preserve current semantics: skip
show_results when runs > 1 (same as script mode)
- Define exit-code precedence when combining
--repeat, pytest failures, and --fail-on-regression
- Add tests and update
docs/docs/cli-reference.md
Use Case
CI pipelines using oracletrace run -- pytest tests/ need median aggregation to reduce OS-level noise — the primary motivation for --repeat in v3.0.0. Without this, CI users cannot benefit from repeat when tracing test suites.
Example (optional)
oracletrace run --repeat 5 --json current.json --compare baseline.json --fail-on-regression -- pytest tests/ -q
# pytest runs 5 times; exported/compared trace uses median total_time and call_count per function
Alternatives Considered
- Document that
--repeat is script-only and reject it on run — simpler but worse UX since the flag is already accepted.
- Require users to shell-loop pytest manually — workable but duplicates logic already in
_run_target().
Additional Context
Verified locally: oracletrace run --repeat 3 -- pytest calls pytest.main() once.
Relevant files:
oracletrace/cli.py — _run_target() (lines 227–253), _run_pytest() (lines 297–318)
oracletrace/tracer.py — FunctionAggregate
tests/test_cli.py — existing --repeat and run pytest tests
docs/docs/cli-reference.md — CI pytest workflow section
Checklist
Summary
Implement median-based repeat aggregation for
oracletrace run -- pytest, matching the behavior already available when tracing a script directly.Problem
--repeatis defined on_BASE_PARSER(shared by both entry points) but aggregation logic exists only in_run_target(). When usingoracletrace run --repeat 3 -- pytest, pytest runs once and repeat is silently ignored.Proposed Solution
FunctionAggregatemedian logic from_run_target()into a shared helper (e.g._aggregate_repeated_runs())_run_pytest()to invoke pytest N times when--repeat > 1and aggregate trace datashow_resultswhenruns > 1(same as script mode)--repeat, pytest failures, and--fail-on-regressiondocs/docs/cli-reference.mdUse Case
CI pipelines using
oracletrace run -- pytest tests/need median aggregation to reduce OS-level noise — the primary motivation for--repeatin v3.0.0. Without this, CI users cannot benefit from repeat when tracing test suites.Example (optional)
Alternatives Considered
--repeatis script-only and reject it onrun— simpler but worse UX since the flag is already accepted._run_target().Additional Context
Verified locally:
oracletrace run --repeat 3 -- pytestcallspytest.main()once.Relevant files:
oracletrace/cli.py—_run_target()(lines 227–253),_run_pytest()(lines 297–318)oracletrace/tracer.py—FunctionAggregatetests/test_cli.py— existing--repeatandrun pytesttestsdocs/docs/cli-reference.md— CI pytest workflow sectionChecklist