fix(loadtest): measure parent-side wall in the multiprocess sweep#97
Merged
Conversation
The multi-process fan-out gathered each slice's samples but called summarize(..., wall_seconds=0.0), so the reported throughput was always 0.0 (count/wall collapses to 0.0 when wall is non-positive). Latencies, retries, oversell, and the oracle verdict were already meaningful; only throughput was degenerate. Time the parent-side wall around the ProcessPoolExecutor run — from dispatch until the last slice returns — and pass it to summarize, mirroring how the single-process drive() reports its wall. The parent-side span is the correct denominator for an aggregate throughput across overlapping workers; summing per-slice walls would double-count concurrent time. The comparative table's published numbers still come from the single-process run. Closes #96. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The multi-process fan-out (
loadtest.multiprocess.multiprocess_sweep) gathered each slice'sCommandSamples but calledsummarize("guarded-mp", samples, wall_seconds=0.0), so the reported throughput was always0.0(count/wallcollapses to0.0whenwallis non-positive). Correctness signals —oversell,retries, latencies, and the oracle verdict — were already meaningful; onlythroughputwas degenerate.This threads a real aggregate wall through the sweep: it times the parent-side wall around the
ProcessPoolExecutorrun (from dispatch until the last slice returns) and passes it tosummarize, mirroring how the single-processdrive()reports its wall.Why parent-side wall
The parent-side span is the correct denominator for an aggregate throughput across overlapping workers — summing per-slice walls would double-count concurrent time. The comparative table's published throughput/p99 numbers still come from the single-process run; this is the multi-process path's own end-to-end figure.
Test
TDD: extended
test_multiprocess_guarded_stays_cleanwithassert report.metrics.throughput > 0.0. Confirmed it fails first (assert 0.0 > 0.0, withcount=48and valid latencies), then passes after the fix. Fullmake verifyis green (591 passed, 99.14% coverage; ruff, mypy --strict, import-linter, pip-audit clean).Closes #96.
🤖 Generated with Claude Code