Enhance benchmarking capabilities and improve performance metrics#9
Merged
Conversation
Contributor
Author
|
@copilot resolve the merge conflicts in this pull request |
There was a problem hiding this comment.
Pull request overview
This PR broadens QMC.jl’s benchmarking and test infrastructure while introducing several performance-focused internal changes (notably in Gaussian transforms and replicated QMC integration), alongside API extensions (new kernels, AcceptanceRejectionReal, and set_tolerance!) and extensive formatting/documentation updates.
Changes:
- Added a standalone benchmarking workflow (Makefile targets +
benchmark/environment/scripts) and expanded CI/test runners with timing + coverage artifacts. - Improved numerical/performance paths in core components (Gaussian transform fast paths, replicated QMC integration sampling, periodization with Jacobian weights).
- Expanded the public surface area (new kernels, new true measure, exported helper APIs) and added/expanded tests and READMEs across the source tree.
Reviewed changes
Copilot reviewed 86 out of 92 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| test/test_utils.jl | New unit tests for FWHT utilities (natural/sequency/2D). |
| test/test_multilevel.jl | Formatting cleanup and reduced noisy test output. |
| test/test_kernels.jl | Adds tests for new kernel types and tightens kernel test coverage. |
| test/test_integration.jl | Formatting + readability improvements in end-to-end integration tests. |
| test/test_discrete_distributions.jl | Adds lattice order-alias tests and formatting normalization. |
| test/test_aqua.jl | Formatting normalization for Aqua quality checks. |
| test/runtests.jl | Centralizes test file list and prints per-file timing/total duration. |
| test/run_notebooks.jl | Adds notebook timing + summary reporting improvements. |
| test/README.md | Documents test entry points and local test/coverage commands. |
| src/util/transforms.jl | Minor kwarg formatting normalization. |
| src/util/README.md | Documents purpose and contents of src/util/. |
| src/util/periodization.jl | Adds _periodize_with_weight to preserve integrals under periodization. |
| src/util/mlmc_test.jl | Formatting-only refactor for printouts and expressions. |
| src/util/gpu_backend.jl | Minor style refactor to expression-bodied _gpu_available. |
| src/util/diagnostics.jl | Formatting improvements for IterationLog and its display. |
| src/util/bernoulli.jl | Minor refactors to short-form functions and kwarg spacing. |
| src/true_measure/zero_inflated_exp_uniform.jl | Parameterizes struct for better type stability; formatting cleanup. |
| src/true_measure/uniform.jl | Parameterizes struct for better type stability; formatting cleanup. |
| src/true_measure/uniform_triangle.jl | Parameterizes struct; simplifies show method. |
| src/true_measure/triangular.jl | Parameterizes struct; simplifies show method. |
| src/true_measure/student_t.jl | Parameterizes struct; clamps inputs for inverse-CDF stability; simplifies show. |
| src/true_measure/README.md | Documents purpose and contents of src/true_measure/. |
| src/true_measure/matern_gp.jl | Parameterizes struct; minor formatting tweaks. |
| src/true_measure/lebesgue.jl | Parameterizes struct; formatting tweaks in show. |
| src/true_measure/kumaraswamy.jl | Parameterizes struct; formatting cleanup. |
| src/true_measure/johnsons_su.jl | Parameterizes struct; clamps inputs for inverse-CDF stability; simplifies show. |
| src/true_measure/geometric_brownian_motion.jl | Parameterizes struct; vectorized transform for performance. |
| src/true_measure/gaussian.jl | Replaces Normal quantile broadcast with erfinv-based path + diagonal fast path + open-interval clamp. |
| src/true_measure/distributions_wrapper.jl | Parameterizes struct; formatting cleanup. |
| src/true_measure/brownian_motion.jl | Parameterizes struct; formatting cleanup and argument checks. |
| src/true_measure/bernoulli_cont.jl | Parameterizes struct; simplifies show method. |
| src/true_measure/acceptance_rejection.jl | Parameterizes struct; adds new AcceptanceRejectionReal true measure for ℝ^d DAR. |
| src/stopping_criterion/README.md | Documents stopping-criterion folder layout and scope. |
| src/stopping_criterion/pf_gp_ci.jl | Parameterizes struct; keeps “not implemented” stub behavior. |
| src/stopping_criterion/cub_qmc_rep_student_t.jl | Improves replicated sampling path; adds resume-key compatibility and :n_total/:n_per_rep metadata. |
| src/stopping_criterion/cub_qmc_net_g.jl | Adds resume-key compatibility and richer data fields (:n_total, :n_per_rep). |
| src/stopping_criterion/cub_qmc_lattice_g.jl | Adds resume-key compatibility and richer data fields (:n_total, :n_per_rep). |
| src/stopping_criterion/cub_qmc_bayes_net_g.jl | Uses periodization weights; fixes mean extraction; improves resume-key handling + data fields. |
| src/stopping_criterion/cub_qmc_bayes_lattice_g.jl | Uses periodization weights; fixes mean extraction; improves resume-key handling + data fields. |
| src/stopping_criterion/cub_mlqmc.jl | Parameterizes criterion type; formatting tweaks; minor stat kwarg normalization. |
| src/stopping_criterion/cub_mlqmc_cont.jl | Parameterizes criterion type; minor refactors to short-form helpers and formatting. |
| src/stopping_criterion/cub_mlmc.jl | Parameterizes criterion type; formatting-only refactors and show formatting. |
| src/stopping_criterion/cub_mlmc_cont.jl | Parameterizes criterion type; formatting-only refactors and show formatting. |
| src/stopping_criterion/cub_mc_g.jl | Parameterizes criterion type; formatting-only refactors and show formatting. |
| src/stopping_criterion/cub_mc_clt.jl | Parameterizes criterion type; adds :n_total to result metadata. |
| src/stopping_criterion/cub_mc_clt_vec.jl | Parameterizes criterion type; formatting-only refactors and show formatting. |
| src/README.md | Documents src/ layout and main subdirectories. |
| src/QMC.jl | Exports new APIs/types (e.g. kernels, AcceptanceRejectionReal, set_tolerance!); minor init style. |
| src/kernel/shift_invariant.jl | Minor formatting + short-form show. |
| src/kernel/README.md | Documents kernel folder purpose and file categories. |
| src/kernel/multitask.jl | Parameterizes KernelMultiTask for type stability; formatting cleanup. |
| src/kernel/matern.jl | Adds KernelRationalQuadratic and KernelSquaredExponential; minor refactors and show updates. |
| src/kernel/dig_shift_invariant.jl | Minor formatting normalization. |
| src/integrand/umbridge_wrapper.jl | Parameterizes wrapper’s true_measure field; improves error formatting. |
| src/integrand/sin1d.jl | Parameterizes integrand’s true_measure; simplifies show. |
| src/integrand/sensitivity_indices.jl | Parameterizes inner/base integrand; minor formatting. |
| src/integrand/README.md | Documents integrand categories and file organization. |
| src/integrand/multimodal2d.jl | Parameterizes true_measure; formatting cleanup. |
| src/integrand/ml_integrand.jl | Refactors default methods to short form; minor formatting normalization. |
| src/integrand/linear0.jl | Parameterizes true_measure; simplifies constructor and show. |
| src/integrand/keister.jl | Vectorizes evaluate; replaces exact formula with stable hypergeometric series. |
| src/integrand/ishigami.jl | Parameterizes true_measure; adds closed-form exact Sobol reference helper + export. |
| src/integrand/hartmann6d.jl | Parameterizes true_measure; formatting cleanup for constants. |
| src/integrand/genz.jl | Vectorizes oscillatory evaluation; improves exact formulas (oscillatory + corner_peak). |
| src/integrand/four_branch2d.jl | Parameterizes true_measure; simplifies show. |
| src/integrand/financial_option.jl | Adds European fast path for GBM-transformed inputs; adds digital exact value; formatting cleanup. |
| src/integrand/financial_option_ml.jl | Parameterizes true_measure; formatting cleanup and minor refactors. |
| src/integrand/custom_fun.jl | Parameterizes callable type; forwards kwargs in sample_and_evaluate (enables n_start, etc.). |
| src/integrand/box_integral.jl | Parameterizes true_measure; short-form show. |
| src/integrand/bayesian_lr_coeffs.jl | Parameterizes true_measure; formatting cleanup and improved argument checks. |
| src/integrand/asian_option.jl | Parameterizes true_measure; adds exact value for geometric Asian; formatting cleanup. |
| src/discrete_distribution/README.md | Documents discrete-distribution folder layout and generators. |
| src/discrete_distribution/lattice.jl | Canonicalizes order tokens/aliases (QMCPy parity); parameterizes RNG type; window validation tweaks. |
| src/discrete_distribution/kronecker.jl | Minor formatting normalization. |
| src/discrete_distribution/iid_std_uniform.jl | Parameterizes RNG type; short-form show. |
| src/discrete_distribution/halton.jl | Parameterizes RNG type; formatting cleanup. |
| src/discrete_distribution/digital_net_b2.jl | Parameterizes RNG type; adds fused C-path when available to reduce allocations. |
| src/discrete_distribution/digital_net_any_bases.jl | Parameterizes RNG type; formatting cleanup and clearer checks. |
| src/data/README.md | Documents purpose of static data tables. |
| src/abstract_types.jl | Adds set_tolerance!, _open_unit_interval, and improves QMCResult display. |
| README.md | Updates benchmark instructions and adds repo-layout pointers + coverage notes. |
| Makefile | Adds bench/coverage targets, label helpers, and Python auto-detection for QMCPy comparisons. |
| docs/src/README.md | Documents doc-source page layout. |
| docs/src/ci-testing.md | Updates CI/coverage workflow documentation and local commands. |
| docs/README.md | Documents local docs build process. |
| devtools/README.md | Documents purpose of devtools helpers. |
| devtools/formatter/README.md | Documents the formatter environment. |
| community.md | Adds Julia learning link and expands background references. |
| benchmark/results/README.md | Documents benchmark output conventions and intended git-ignoring. |
| benchmark/README.md | Full benchmark workflow guide (Julia-only, label comparison, Julia vs QMCPy). |
| benchmark/Project.toml | Adds benchmark-only environment for BenchmarkTools/PkgBenchmark tooling. |
| benchmark/compare_labels.jl | New script to compare two saved Julia benchmark labels and produce a report. |
| .JuliaFormatter.toml | Updates formatter rules (kwarg spacing, long→short defs, line joining). |
| .gitignore | Ignores Python bytecode caches and benchmark outputs while keeping README tracked. |
| .github/workflows/README.md | Documents workflow files and their roles. |
| .github/workflows/format.yml.disabled | Adds a disabled formatting workflow for reference. |
| .github/workflows/ci.yml | Runs tests with coverage, uploads lcov.info artifact, and uploads to Codecov. |
| .github/README.md | Documents GitHub-specific repository automation structure. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+28
to
+32
| function set_tolerance!( | ||
| sc::AbstractStoppingCriterion; | ||
| abs_tol=nothing, | ||
| rel_tol=nothing, | ||
| rmse_tol=nothing, |
Comment on lines
+46
to
+51
| if rmse_tol !== nothing | ||
| hasfield(typeof(sc), :rmse_tol) || | ||
| throw(ArgumentError("$(typeof(sc)) has no rmse_tol field")) | ||
| rmse_tol >= 0 || throw(ArgumentError("rmse_tol must be >= 0")) | ||
| sc.rmse_tol = Float64(rmse_tol) | ||
| end |
Comment on lines
+60
to
+63
| const _OPEN01_LOW = eps(Float64) | ||
| const _OPEN01_HIGH = 1.0 - eps(Float64) | ||
| @inline _open_unit_interval(u::Float64) = clamp(u, _OPEN01_LOW, _OPEN01_HIGH) | ||
|
|
Contributor
Resolved the merge conflicts and committed the fix in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.