Audit polish: sim-vs-analytical validation, LP/CPM tests, AI caveat#3
Open
Gavand1969 wants to merge 1 commit into
Open
Audit polish: sim-vs-analytical validation, LP/CPM tests, AI caveat#3Gavand1969 wants to merge 1 commit into
Gavand1969 wants to merge 1 commit into
Conversation
Job-readiness audit polish: * README: fix stale test count (57 → 131), remove broken docs/screenshot.png reference, add "Why it exists" framing and an AI explanation caveat that makes the LLM-vs-solver responsibility split explicit. * README: surface the new validation test + runnable demo with copy-paste commands so a hiring manager can verify correctness in 30 seconds. * tests/test_simulation_vs_analytical.py: 8 cross-validation tests that check the Monte-Carlo W_q lies within the 95% CI of the analytical W_q for M/M/1, M/M/c, and that M/D/1 < M/M/1 empirically. * tests/test_optimization.py: 12 new tests pinning LP correctness against the textbook Wyndor Glass problem (optimal value 36, binding constraints, slack), minimisation, infeasible/unbounded detection, and CPM critical path/float/PERT variance. * examples/queue_design_walkthrough.py: ~60-line narrated demo that sizes an M/M/c call centre to hit P(wait) ≤ 30%, then validates against a 15-replication simulation (analytical 0.71 min vs simulated 0.70 ± 0.05). * .env.example: align RATE_LIMIT_PER_MINUTE default with the server (60). Result: 131 passing tests (was 111), zero behaviour changes to solvers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR is the safe-fixes output of a deep job-readiness / product-credibility audit of Sim2Sim. The solvers themselves were not modified — they are already textbook-correct. The changes are all polish and trust-building:
README
57→131) and remove brokendocs/screenshot.pngreference.RATE_LIMIT_PER_MINUTEdocumentation with the server default (60) and documentALLOWED_ORIGIN.tests/test_simulation_vs_analytical.py (new, 8 tests)
The most important credibility test: pin the Monte-Carlo simulator against the closed-form analytical answer. Asserts the empirical
W_qlies within 1.5 × CI half-width of the analyticalW_qfor M/M/1 (three (λ, μ) settings) and M/M/c (two settings), checks empirical utilisation accuracy, CI shrinkage with replication count, and that M/D/1 < M/M/1 empirically.tests/test_optimization.py (new, 12 tests)
The LP and CPM solvers previously had no direct unit coverage — only indirect API integration tests. New tests pin LP correctness against the textbook Wyndor Glass problem (optimal value 36, optimal
(x1, x2) = (2, 6), binding constraints, slack values), minimisation, infeasible / unbounded detection, plus CPM critical path / float / PERT variance.examples/queue_design_walkthrough.py (new)
A ~60-line narrated end-to-end demo: size an M/M/c call centre to hit
P(wait) ≤ 30%, then validate against a 15-replication simulation. Output: analytical 0.71 min vs simulated 0.70 ± 0.05 min. Intended as the script to read first when evaluating the project..env.example
Align
RATE_LIMIT_PER_MINUTEdefault with the actual server default (60).Test results
Previously: 111 tests. This PR adds 20 (+18%).
Test plan
pytest -q— confirm all 131 passpytest tests/test_simulation_vs_analytical.py -v— confirm cross-validation tests passpython -m examples.queue_design_walkthrough— confirm demo prints with sim Wq inside CI🤖 Generated with Claude Code