Optimize ASCI workflow#464
Open
v-boqinzhang wants to merge 38 commits into
Open
Conversation
…osoft/qdk-chemistry into feature/omp_asci_search_opt
v-boqinzhang
requested review from
ConradJohnston,
nabbelbabbel,
sobolevnrm and
wavefunction91
as code owners
Contributor
There was a problem hiding this comment.
Pull request overview
This PR optimizes the MACIS ASCI workflow (and its QDK integration) by reducing integral precompute/memory, improving determinant search/build performance, and adding incremental Hamiltonian-build and warm-start capabilities to speed iterative ASCI growth/refinement.
Changes:
- Remove storage of full antisymmetrized two-electron integrals (
G_pqrs) and compute antisymmetrized terms on-the-fly fromV_pqrs. - Add incremental (patched) Hamiltonian build support, plus ASCI warm-start heuristics and oscillation stabilization during refinement.
- Introduce/enable additional Hamiltonian build algorithms (Residue Arrays / Dynamic Bit Masking), parallel pair processing/sort improvements, and build-system support for ips4o.
Reviewed changes
Copilot reviewed 31 out of 32 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| external/macis/tests/asci.cxx | Update test calls for new asci_iter signature (cache arg). |
| external/macis/src/macis/hamiltonian_generator/base.ipp | Stop building full G_pqrs; compute reduced intermediates from V (but adds stdout logging). |
| external/macis/src/macis/CMakeLists.txt | Fetch/use ips4o include path; add atomic link on non-Apple platforms. |
| external/macis/include/macis/solvers/selected_ci_diag.hpp | Add incremental-cache API and patched-operator path; adjust MPI reporting and caching. |
| external/macis/include/macis/solvers/incremental_h_build.hpp | New incremental (patched) operator + cache state implementation. |
| external/macis/include/macis/mcscf/cas.hpp | Update selected_ci_diag call signature. |
| external/macis/include/macis/hamiltonian_generator/sorted_double_loop.hpp | Add build/RDM/entropy timing logs. |
| external/macis/include/macis/hamiltonian_generator/residue_arrays.hpp | New diagonal-block Hamiltonian build via residue arrays; pair-based RDM override. |
| external/macis/include/macis/hamiltonian_generator/dynamic_bit_masking.hpp | New diagonal-block Hamiltonian build via dynamic bit masking. |
| external/macis/include/macis/hamiltonian_generator/connection_build_utils.hpp | New shared utilities for pair enumeration and CSR construction. |
| external/macis/include/macis/hamiltonian_generator/matrix_elements.hpp | Compute same-spin antisymmetrized values from V_pqrs on the fly. |
| external/macis/include/macis/hamiltonian_generator/base.hpp | Remove G_pqrs storage/getter. |
| external/macis/include/macis/bitset_operations.hpp | Faster ffs/fls via word scanning for word-aligned bitsets. |
| external/macis/include/macis/asci/iteration.hpp | Add warm-start projection; plumb incremental cache into diagonalization. |
| external/macis/include/macis/asci/grow.hpp | Add grow-phase CI tolerance override; taper grow factor; carry/clear incremental H cache. |
| external/macis/include/macis/asci/refine.hpp | Add oscillation stabilization via union rediagonalization; carry incremental H cache. |
| external/macis/include/macis/asci/pt2.hpp | Remove G_pqrs parameter; adjust worker/thread logging. |
| external/macis/include/macis/asci/mask_constraints.hpp | Add buffer-reuse overloads; compute same-spin antisym from V; add constraint-gen debug logs. |
| external/macis/include/macis/asci/determinant_sort.hpp | Add parallel sort+accumulate for ASCI contribution arrays (ips4o/OpenMP). |
| external/macis/include/macis/asci/determinant_search.hpp | Remove G_pqrs usage; improve constraint search threading and logging; use parallel S&A. |
| external/macis/include/macis/asci/determinant_contributions.hpp | Switch same-spin doubles contributions to on-the-fly antisym from V. |
| external/macis/examples/src/standalone_driver.cxx | Update PT2 call signature (remove G()/G_pqrs). |
| external/macis/cmake/macis-ips4o.cmake | Avoid linking atomic on Apple. |
| docs/source/_static/examples/python/example_circuit.circuit.json | Add serialized circuit example JSON. |
| docs/source/_static/examples/python/example_circuit.circuit.h5 | Add serialized circuit example HDF5 artifact. |
| cpp/tests/test_macis.cpp | Adjust ASCI test settings (larger determinant cap; enable refine; tighter tolerances). |
| cpp/src/qdk/chemistry/data/basis_set.cpp | Avoid repeated tar extraction when JSON already exists (parallel-run race mitigation). |
| cpp/src/qdk/chemistry/algorithms/microsoft/macis_base.hpp | Add timing/info logs around wavefunction/RDM/entropy construction; update orbitals support note. |
| cpp/src/qdk/chemistry/algorithms/microsoft/macis_base.cpp | Plumb new ASCI settings (warm-start, patch overlap, taper factor, etc.). |
| cpp/src/qdk/chemistry/algorithms/microsoft/macis_asci.hpp | Add new settings defaults and documentation strings. |
| cpp/src/qdk/chemistry/algorithms/microsoft/macis_asci.cpp | Select Hamiltonian generator implementation via h_build_algo; expand info logging. |
| cpp/src/qdk/chemistry/algorithms/microsoft/cholesky_hamiltonian.cpp | Batch shell-pair processing and orthogonalization to reduce overhead. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
📊 Coverage Summary
Detailed Coverage ReportsC++ Coverage DetailsPython Coverage DetailsPybind11 Coverage Details |
wavefunction91
requested changes
May 13, 2026
wavefunction91
requested changes
May 20, 2026
📊 Coverage Summary
Detailed Coverage ReportsC++ Coverage DetailsPython Coverage DetailsPybind11 Coverage Details |
v-agamshayit
reviewed
Jun 4, 2026
| #include <fcntl.h> | ||
| #include <qdk/chemistry/scf/config.h> | ||
| #include <sys/file.h> | ||
| #include <unistd.h> |
Contributor
There was a problem hiding this comment.
This is needed for flock, but not all OSs have this file
| break; | ||
| } | ||
|
|
||
| // Detect oscillation: sign change in energy delta with similar magnitude |
Contributor
There was a problem hiding this comment.
Is there a test that goes through this oscillation detection?
Comment on lines
+188
to
+190
| const size_t ndets = std::distance(dets_begin, dets_end); | ||
| const bool incremental = cache != nullptr; | ||
|
|
Comment on lines
+10
to
+15
| #pragma once | ||
| #include <array> | ||
| #include <cassert> | ||
| #include <macis/hamiltonian_generator/connection_build_utils.hpp> | ||
| #include <macis/hamiltonian_generator/sorted_double_loop.hpp> | ||
|
|
Comment on lines
+93
to
+98
| const size_t ndets = std::distance(dets_begin, dets_end); | ||
| const WfnType* dets = &*dets_begin; | ||
|
|
||
| detail::SpinCache<WfnType> cache; | ||
| cache.build({dets, ndets}); | ||
|
|
Comment on lines
+155
to
+160
| const size_t ndets = std::distance(dets_begin, dets_end); | ||
| detail::SpinCache<WfnType> cache; | ||
| cache.build({&*dets_begin, ndets}); | ||
|
|
||
| const size_t n_elec = dets_begin->count(); | ||
| const size_t residues_per_det = n_elec * (n_elec - 1) / 2; |
Comment on lines
+533
to
+542
| } else if constexpr (N % 64 == 0) { | ||
| // FNV-1a inspired hash over 64-bit words | ||
| size_t h = 14695981039346656037ULL; | ||
| auto words = reinterpret_cast<const uint64_t*>(&bs); | ||
| constexpr auto nwords = N / 64; | ||
| for (size_t i = 0; i < nwords; ++i) { | ||
| h ^= words[i]; | ||
| h *= 1099511628211ULL; | ||
| } | ||
| return h; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.