From e3852cc629ea49d4b1c1f73502a48005af6c9824 Mon Sep 17 00:00:00 2001 From: Roger-luo Date: Thu, 16 Jul 2026 23:47:52 -0400 Subject: [PATCH 1/2] docs(design): document traits 2 refactor --- docs/design/tableau-data-structure.md | 329 ++++++++++++ .../traits-2-configuration-and-hashing.md | 508 ++++++++++++++++++ docs/design/word-data-structures.md | 357 ++++++++++++ 3 files changed, 1194 insertions(+) create mode 100644 docs/design/tableau-data-structure.md create mode 100644 docs/design/traits-2-configuration-and-hashing.md create mode 100644 docs/design/word-data-structures.md diff --git a/docs/design/tableau-data-structure.md b/docs/design/tableau-data-structure.md new file mode 100644 index 00000000..7b283359 --- /dev/null +++ b/docs/design/tableau-data-structure.md @@ -0,0 +1,329 @@ +# Contiguous tableau data structure + +Status: design sketch + +## Purpose + +The tableau should be a specialized bit-matrix data structure. It should not +be represented as `Vec>`, and its internal rows should not +affect the shared PPVM trait system. + +This design separates: + +- the logical stabilizer-tableau model; +- its physical, contiguous memory layout; +- orientation changes used to accelerate different operations; and +- structural hashing used when a tableau is a classical-mixture key. + +Gate, noise, measurement, reset, and `Indexable` traits observe tableau +behavior. They do not expose matrix blocks, row values, strides, alignment, or +orientation. + +## Goals + +- Store the X/Z tableau bits in contiguous, aligned, bit-packed memory. +- Make one- and two-qubit column operations efficient. +- Permit temporary transposition for row-oriented elimination and collapse. +- Keep phases and optional loss state independently addressable and hashable. +- Hash the logical tableau independently of its current physical orientation. +- Avoid parameterizing the trait system by a row type or Pauli-word storage. +- Leave room for SIMD-width and padding changes without changing public + behavioral traits. + +## Non-goals + +- Reusing the standalone `PauliWord` representation for tableau rows. +- Exposing borrowed tableau rows as the primary public interface. +- Standardizing a general-purpose matrix trait in `ppvm-traits-2`. +- Maintaining both row-major and column-major copies before benchmarks show + that the memory and synchronization cost is worthwhile. +- Deciding in this document whether PPVM should store a forward or inverse + tableau. Orientation and inversion are independent design choices. + +## Logical model + +For `n` qubits, a stabilizer/destabilizer tableau has `2n` generators. Each +generator contains an X bit and a Z bit for every qubit, plus a phase: + +```text + qubit + 0 1 2 ... n-1 +generator 0 x/z x/z x/z +generator 1 x/z x/z x/z + ... +generator 2n-1 x/z x/z x/z + +phase one value per generator +``` + +The logical state consists of: + +- an X matrix of shape `(2n, n)`; +- a Z matrix of shape `(2n, n)`; +- a phase plane of length `2n`; and +- for lossy variants, a loss plane of length `n` or another explicitly defined + logical shape. + +This model does not imply a physical row object. + +## Physical storage + +The first prototype should use one aligned contiguous allocation for the bit +planes, divided by computed offsets: + +```rust +pub struct TableauData { + blocks: AlignedVec, + x_offset: usize, + z_offset: usize, + phase_offset: usize, + loss_offset: Option, + major_stride: usize, + n_qubits: usize, + orientation: Orientation, +} +``` + +`Block` is an internal implementation choice such as `u64` or a SIMD-width +block. It is not an associated type of a public tableau trait. Offsets and +strides account for alignment and padding, while logical accessors enforce the +actual `(2n, n)` dimensions. + +Keeping all planes in one allocation improves cloning and locality and avoids +one allocation per generator. It also lets a mixture branch copy a single +contiguous region. If benchmarks favor separate aligned allocations for X and +Z, that change remains internal to `TableauData`. + +Padding must either be kept zero or excluded from equality and hashing. Zeroed +padding is preferable because it permits bulk comparison and hashing of +canonical ranges. + +## Column-major orientation + +The default orientation should make the generator dimension contiguous for a +fixed qubit. In other words, the X and Z planes are column-major with respect +to the logical `(generator, qubit)` matrix: + +```text +qubit 0: X bits for generators 0..2n, then padding +qubit 1: X bits for generators 0..2n, then padding +... + +qubit 0: Z bits for generators 0..2n, then padding +qubit 1: Z bits for generators 0..2n, then padding +... +``` + +This layout makes a selected qubit column contiguous. Single-qubit gates can +load the X and Z columns, update them with bitwise operations, and update the +affected phase bits. Two-qubit gates operate on two pairs of contiguous +columns. Measurement can scan the selected anticommutation column without +stepping across separately allocated row objects. + +Stim uses aligned SIMD bit tables and documents its tableau layout as +column-major, with output-observable iteration following contiguous memory. It +also provides an explicit quadrant transpose and a transposition guard for +operations that need the opposite orientation: + +- [Stim `Tableau`](https://github.com/quantumlib/Stim/blob/main/src/stim/stabilizers/tableau.h) +- [Stim `TableauSimulator`](https://github.com/quantumlib/Stim/blob/main/src/stim/simulators/tableau_simulator.h) +- [Stim `simd_bit_table`](https://github.com/quantumlib/Stim/blob/main/src/stim/mem/simd_bit_table.h) + +Column-major storage is only one part of Stim's performance strategy. PPVM +should benchmark its own gate, measurement, and sampling workloads instead of +assuming the same total performance from layout alone. + +## Temporary transposition + +Row multiplication and elimination need long generator rows and therefore +prefer the opposite orientation. The initial design should transpose the X/Z +quadrants temporarily instead of storing two permanent copies: + +```rust +pub enum Orientation { + ColumnMajor, + RowMajor, +} + +pub struct TransposedTableau<'a> { + tableau: &'a mut Tableau, +} +``` + +Creating the guard transposes the bit matrices and marks the physical +orientation. Dropping it restores the canonical column-major orientation: + +```rust +impl Drop for TransposedTableau<'_> { + fn drop(&mut self) { + self.tableau.restore_column_major(); + } +} +``` + +Operations that require row-major access receive the guard, making the +orientation precondition explicit. Public methods return with the tableau in +canonical orientation. Panic safety must be preserved by the guard's `Drop` +implementation. + +Transposition is a physical reordering of the same logical bits. It does not +invalidate structural hashes or change equality. Hashing should occur only +through logical access or while the tableau is in its public canonical state. + +Maintaining both orientations with dirty flags remains a future alternative +for workloads that switch frequently enough to amortize the doubled storage. + +## Tableau API boundary + +The public API exposes logical operations instead of storage slices: + +```rust +impl Tableau { + pub fn n_qubits(&self) -> usize; + + pub fn h(&mut self, qubit: usize); + pub fn cnot(&mut self, control: usize, target: usize); + pub fn measure_z(&mut self, qubit: usize) -> bool; + + pub fn x_bit(&self, generator: usize, qubit: usize) -> bool; + pub fn z_bit(&self, generator: usize, qubit: usize) -> bool; + pub fn phase(&self, generator: usize) -> Phase; +} +``` + +Logical bit accessors are useful for tests, serialization, debugging, and +interoperability. They are not intended as the hot gate implementation path. +Bulk import/export methods may use a canonical serialized representation +without exposing the in-memory layout. + +There should be no `stabilizers_mut() -> &mut [Row]` or equivalent escape hatch +that bypasses hash invalidation and orientation invariants. Specialized +internal row operations operate through `TableauData` or a transposition guard. + +## Gate access patterns + +The implementation should categorize mutations by physical access and hash +effect: + +| Operation | Preferred access | X/Z changed | Phase changed | +| --- | --- | --- | --- | +| Pauli `X`, `Y`, `Z` | column | no | possibly | +| `H`, `S` | one column pair | yes | possibly | +| `CNOT`, `CZ` | two column pairs | yes | possibly | +| Find measurement pivot | column | no | no | +| Row multiplication | transposed row | yes | possibly | +| Collapse/elimination | column scan + transposed rows | yes | yes | +| Physical transpose | bulk matrix | logically no | no | + +This table describes logical mutations. A gate may determine that no phase bit +changed and preserve the phase cache in that special case, but conservative +component invalidation is correct for the first implementation. + +## Structural hashing + +A tableau used in a classical mixture is an `Indexable` key. It owns its own +hasher and cache representations; neither is inherited from `PauliWord`. + +The structural hash is composed from independent logical components: + +```text +tableau hash = combine(xz hash, phase hash[, loss hash]) +``` + +The X and Z planes share an `xz_hash` cache because most Clifford mutations +update them together. The phase plane has a separate cache so Pauli +conjugations and sign changes do not force a matrix rehash. Loss should use a +third cache if it is independently mutable and part of key identity. + +```rust +pub struct Tableau { + data: TableauData, + xz_hash: XzHashCache, + phase_hash: PhaseHashCache, + loss_hash: Option, + rng: SmallRng, +} +``` + +The cache types above are concrete choices made by the tableau author. The +shared `Indexable::HashCache` associated type may name a small aggregate or +another representation chosen by the implementation. + +Equality and hashing include logical qubit count, generator order, all logical +X/Z bits, phases, and loss state when present. They exclude: + +- RNG state; +- allocation capacity; +- alignment padding; +- cache values and validity flags; and +- current physical orientation. + +The component invalidation rules are: + +| Mutation | X/Z cache | Phase cache | Loss cache | +| --- | --- | --- | --- | +| Pauli `X`, `Y`, `Z` | preserve | invalidate if changed | preserve | +| Direct phase change | preserve | invalidate | preserve | +| `H`, `S`, `CNOT`, `CZ` | invalidate | invalidate if changed | preserve | +| Row multiplication | invalidate | invalidate if changed | preserve | +| Mark or clear loss | preserve | preserve | invalidate | +| Physical transpose | preserve | preserve | preserve | +| RNG update | preserve | preserve | preserve | + +The current `ppvm-tableau-sum` split between `word_fingerprint` and +`phase_loss_hash` is evidence that component hashing matters. The new tableau +owns these components directly and separates loss as well when the logical +model permits it. + +## Cloning and mixture use + +Classical-mixture branching can clone tableaus frequently. Contiguous backing +storage makes cloning a bulk memory copy. A clone may copy valid hash caches +because it initially has identical logical contents. Subsequent mutation of +the branch invalidates only the affected components. + +Copy-on-write backing storage may be evaluated later, but it should not be part +of the initial design: most branches are mutated immediately, which may turn +reference counting and deferred copying into overhead. + +An indexable tableau must not be structurally mutated while stored as a map +key. Mixture storage removes or clones a key before applying gates and inserts +the resulting tableau under its updated hash. + +## Sampling implications + +Column-major X/Z planes make fixed-qubit queries and bit-parallel gate updates +efficient. They are also compatible with scanning many generators during a +measurement. Temporary transposition makes elimination and row products +contiguous when required. + +This layout should be evaluated separately from higher-level sampling +algorithms. Stim also uses an inverse tableau and reference-frame sampling; +those algorithmic choices are not consequences of column-major storage and do +not belong in the PPVM trait system. + +## Prototype validation + +The prototype should include: + +- property tests comparing gates and measurements with the existing tableau; +- round-trip tests for column-major -> row-major -> column-major transpose; +- equality and hash tests across physical orientations; +- tests proving phase-only changes preserve the X/Z hash cache; +- tests proving padding never affects equality or hashing; +- benchmarks for one- and two-qubit Clifford gates; +- benchmarks for deterministic and random measurement paths; +- benchmarks for clone-and-mutate mixture branching; and +- benchmarks comparing permanent row-major, permanent column-major, and + temporary-transpose variants on representative circuits. + +## Open questions + +1. What block width and alignment should the first implementation use? +2. Should phases occupy one or two bits per generator in the tableau model? +3. Should loss be stored per qubit, per generator, or outside the core tableau? +4. Which operations should receive a transposition guard versus performing + column-strided work directly? +5. Does a dual-orientation representation outperform temporary transposition + for PPVM's measurement-heavy workloads? +6. Should PPVM ultimately store a forward or inverse tableau? diff --git a/docs/design/traits-2-configuration-and-hashing.md b/docs/design/traits-2-configuration-and-hashing.md new file mode 100644 index 00000000..5d242a2a --- /dev/null +++ b/docs/design/traits-2-configuration-and-hashing.md @@ -0,0 +1,508 @@ +# `ppvm-traits-2`: type composition, indexable values, and cached hashing + +Status: design sketch + +## Motivation + +The current `ppvm_traits::Config` bundles choices from several unrelated +layers: + +- coefficient type; +- packed Pauli storage; +- Pauli-word representation; +- key hasher; +- truncation strategy; and +- concrete map implementation. + +This makes the foundational configuration specific to `PauliSum`, even though +the gate and noise traits are shared by other algorithms. In particular, a map +and a truncation strategy are not properties of quantum data. They are choices +made by a particular algorithm. + +The second trait-system experiment should separate: + +1. the coefficient type, passed directly as a generic parameter; +2. concrete quantum-data representations; +3. the hashing contract of values that can be used as keys; and +4. algorithm-specific storage and policy choices. + +The redesign should remain compile-time generic. It should not introduce +runtime trait objects or runtime dispatch for these choices. + +This proposal was compared against the current definitions in +`ppvm-traits/src/config.rs`, `traits/map.rs`, `traits/strategy.rs`, and +`traits/word_trait.rs`; `ppvm-pauli-sum/src/sum/data.rs`; the concrete word +types in `ppvm-pauli-word`; and `ppvm-tableau-sum`'s `EntryStore`, `VecStorage`, +and `MapStorage`. Existing names are retained below unless the abstraction or +responsibility changes. + +## Type-composition layers + +### Coefficient type + +There is no algorithm-agnostic `Config` trait. Algorithms use the coefficient +type directly: + +```rust +pub struct SomeAlgorithm { + // ... +} +``` + +A trait containing only `type Coeff` adds indirection without providing useful +composition. Maps, policies, Pauli-word storage, Pauli-word +implementations, and hashers are also selected independently rather than being +collected into a replacement global configuration trait. + +### Representation types + +There is no separate global storage configuration, and representation storage +does not appear as an associated type. A concrete value encapsulates its own +fields; generic algorithms use behavioral methods instead of naming or +inspecting the backing memory. + +`Word` is the common concept for an indexed algebraic monomial. The old +Pauli-word operations are not Pauli-specific: every supported word has a site +alphabet, an indexed extent, site access and mutation, and a weight: + +```rust +pub trait Word { + type Site; + + fn n_sites(&self) -> usize; + fn get(&self, index: usize) -> Self::Site; + fn set(&mut self, index: usize, site: Self::Site); + fn weight(&self) -> usize; +} +``` + +`Site` selects the operator alphabet without introducing `PauliWord` or +`FermionWord` subtraits. For example, the relevant concrete types may be: + +```rust +pub enum Pauli { + I, + X, + Y, + Z, +} + +pub enum LossySite { + Present(S), + Lost, +} + +pub struct FermionSite { + pub mode: usize, + pub action: FermionAction, +} +``` + +Thus an ordinary packed Pauli word implements `Word`, a lossy +wrapper implements `Word>`, and a future ordered +fermionic product can implement `Word`. A fermionic word's +index denotes factor order; `FermionSite` carries the physical mode. For a +dense Pauli word, the index is the qubit and `n_sites()` is its width. +`weight()` is the number of non-identity factors according to the concrete site +alphabet; an ordered representation that stores no explicit identities may +therefore have `weight() == n_sites()`. Implementations of `set()` preserve +their representation invariants and invalidate the affected hash components. + +The concrete `WithLoss` family provides loss mutation and +`loss_weight()` as inherent methods. Loss channels and loss-specific +truncation specialize directly on `WithLoss`; there is no one-implementation +`LossyPauliWord` capability trait. + +Concrete packed Pauli, lossy, phased, and hash-cache layouts are described in +[`word-data-structures.md`](word-data-structures.md). None of those layouts is +visible through `Word`. + +A tableau is an independent concrete representation. It does not contain a +public row type, implement `Word`, or select an associated word +implementation. Its X/Z matrices, phases, orientation, and contiguous backing +allocation are private implementation details described in +[`tableau-data-structure.md`](tableau-data-structure.md). + +### Behavioral traits + +Shared traits describe operations, not representation layout. Clifford gates +need no coefficient parameter. Operations that consume numeric parameters use +the coefficient type directly: + +```rust +pub trait Clifford { + fn h(&mut self, qubit: usize); + fn cnot(&mut self, control: usize, target: usize); + // ... +} + +pub trait RotationOne { + fn rx(&mut self, qubit: usize, theta: C); + // ... +} + +pub trait PauliError { + fn pauli_error(&mut self, qubit: usize, probabilities: [C; 3]); +} +``` + +The same concrete tableau may implement a numeric trait for every supported +coefficient type without storing that coefficient type. Measurement and reset +traits likewise expose behavior and result types without exposing tableau +rows, packed matrix blocks, or matrix orientation. + +There is no shared `TableauStorage` trait in the first design. If multiple +tableau implementations are later useful, each concrete type can implement +the same behavioral traits. A storage abstraction should only be introduced +after two implementations demonstrate a common interface. + +### Algorithm and storage parameters + +An algorithm should take its independent choices as direct type parameters. +An associated-type bundle is not useful merely because it replaces two type +parameters with one. In particular, there is no `PauliSumAlgorithm` trait that +bundles a term map with a policy: storage layout and policy are orthogonal +choices. + +The reusable sparse-sum shape is: + +```rust +pub struct OperatorSum +where + C: Coefficient, + W: Word + Indexable, + S: SumStorage, + P: Policy, +{ + storage: S, + policy: P, + n_sites: usize, +} +``` + +Here `C`, `W`, `S`, and `P` respectively mean coefficient domain, algebraic +word, concrete sparse-sum storage engine, and algorithm policy. Each +parameter has an independent meaning. Propagation methods select their algebra +through the site type, for example `W: Word` or +`W: Word`. + +`Policy` is the proposed name for the current `Strategy` concept. It retains +the current responsibilities: predicting initial capacity and truncating the +sum. Existing concrete strategies become policies without otherwise changing +their meaning; `NoStrategy` and `CombinedStrategy` become `NoPolicy` and +`CombinedPolicy`, while `MaxPauliWeight` and `CoefficientThreshold` keep their +established names: + +```rust +pub trait Policy: Default + Clone + Copy +where + W: Word + Indexable, + C: Coefficient, +{ + fn capacity(&self, n_sites: usize) -> usize; + + fn truncate(&self, map: &mut M) + where + M: ACMap; +} +``` + +`Policy` and its concrete implementations belong to the sparse-sum crate. This +removes the current split where the `Strategy` trait lives in `ppvm-traits` but +its concrete strategies live in `ppvm-pauli-sum`; the policy is not an +algorithm-agnostic `ppvm-traits-2` concern. + +`ACMap` remains the name of the associative coefficient-map capability already +implemented by `HashMap`, `IndexMap`, and `DashMap`. Its generic signature can +be simplified after `PauliStorage` and the separately supplied build hasher are +removed, but coefficient accumulation, iteration, insertion, retention, and +consumption are the same concept. `ACMap` moves with the sparse-sum engine (the +existing `ppvm-pauli-sum` initially) rather than being renamed or kept in the +algorithm-agnostic trait crate. Its existing capability names—such as +`ACMapBase`, `ACMapIter`, `ACMapAddAssign`, `ACMapInsert`, `ACMapRetain`, and +`ACMapConsume`—should also remain unless implementation work shows that two +capabilities should actually be merged or split. + +A `SumStorage` is a new abstraction extracted from the fields currently owned +directly by `PauliSum`: its maps and reusable workspace. It is an actual value, +not a marker configuration: + +```rust +pub trait SumStorage: Clone +where + W: Word + Indexable, + C: Coefficient, +{ + type Map: ACMap; + + fn data(&self) -> &Self::Map; + fn data_mut(&mut self) -> &mut Self::Map; + + fn map_insert(&mut self, f: F) + where + F: Fn(&W, &mut C) -> Option<(W, C)>; + + fn map_insert_multiple(&mut self, f: F) + where + F: Fn(&W, &mut C) -> Option>; + + fn map_add(&mut self, f: F) + where + F: Fn(&W, &C) -> (W, C); + + fn consume(&mut self); +} +``` + +The exact closure bounds and support for multiple produced terms remain an +implementation detail for the prototype. The important boundary is that the +trait preserves the current semantic operation names without exposing physical +auxiliary maps or scratch buffers. + +The generalized engine may be named `OperatorSum`, but the Pauli specialization +retains the existing domain-facing `PauliSum` name. This is a new internal +generalization, not a requirement to rename Pauli call sites. + +A classical tableau mixture follows the same principle and takes its +`EntryStore` directly rather than introducing a one-associated-type +`TableauMixtureAlgorithm` bundle: + +```rust +pub struct GeneralizedTableauSum +where + C: Coefficient, + T: Indexable, + S: EntryStore, +{ + entries: S, +} +``` + +`GeneralizedTableauSum` and `EntryStore` retain their current names because the +proposal does not change their underlying roles. `GeneralizedTableauSum` and +`OperatorSum` are both sparse linear combinations of indexable keys, so they +may eventually share an implementation. This iteration deliberately keeps them +separate: their mutation, branching, normalization, and storage requirements +have not yet been reduced to a proven common interface. The next design +iteration should look for the smallest useful common factor and merge only that +factor, rather than assuming that the two complete algorithms are identical. + +Every keyed store must use the build hasher associated with its key type. + +### Compatibility with current names + +The redesign is not a vocabulary reset. The following names are retained or +changed according to whether their underlying responsibility changes: + +| Current implementation | Proposal | Rationale | +| --- | --- | --- | +| `Config` | removed | The bundle itself is removed; this is not a rename. | +| `PauliWordTrait` | `Word` plus `Indexable` where used as a key | Word operations are generalized through `Word::Site`; hashing becomes a separate capability. | +| `n_qubits`, `get`, `set`, `weight` | `n_sites`, `get`, `set`, `weight` | Only the Pauli-specific extent name changes; the other operation names stay. | +| concrete `PauliWord` | `PauliWord` | The packed X/Z word is the same domain concept. | +| concrete `LossyPauliWord` | `LossyPauliWord` alias over `WithLoss` | The public concept stays; only the implementation becomes a generic wrapper. | +| `PhasedPauliWord` | `PhasedPauliWord` alias over `Phased` | The public Pauli name stays; the implementation becomes a word-generic wrapper. | +| `rehash` | `invalidate_hash` for lazy modes | Recalculation changes from eager mutation-time work to lazy demand-time work. | +| `Strategy` | `Policy` | Intentional terminology change requested for this redesign. | +| `ACMap` | `ACMap` | The associative coefficient map has the same role. | +| `PauliSum::data`, `map_insert`, `map_add` | same method names on `SumStorage` | These semantic operations already match the proposed boundary. | +| `PauliSum` map pair and `scratch` fields | `SumStorage` | A new abstraction is extracted from currently unnamed storage state. | +| `PauliSum` | `PauliSum` over generalized `OperatorSum` machinery | Pauli-facing code keeps its established name; `OperatorSum` names the new cross-algebra engine. | +| `GeneralizedTableauSum` | `GeneralizedTableauSum` | The classical mixture algorithm remains the same concept. | +| `EntryStore`, `VecStorage`, `MapStorage` | unchanged | The proposal uses the existing storage boundary and implementations. | +| `BuildHasher`, `HashFinalize` | unchanged | The hasher concepts remain, but ownership moves from `Config` to each indexable key type. | +| `PauliStorage` | removed | Packed backing storage becomes private to the concrete word representation. | + +Names such as `Word`, `Indexable`, `SumStorage`, `WithLoss`, and +`OperatorSum` are therefore new because they denote abstractions that do not +exist in the current implementation, not because the existing API is being +renamed wholesale. + +### Trait admission rule + +A proposed trait belongs in the design only when generic code consumes it and +there are multiple meaningful implementation families, or when it is an +established behavioral boundary implemented by different backends. A trait is +not justified merely to name inherent methods on one generic struct. + +This rule keeps: + +- `Indexable`, consumed by keyed stores and implemented by hash-enabled words + and tableaus; +- `Word`, consumed by the shared sparse-sum engine and propagation algorithms; +- gate and noise traits, implemented across propagation and tableau backends; +- `SumStorage` and `ACMap`, implemented by genuinely different storage engines + and collections; +- `EntryStore`, already implemented by `VecStorage` and `MapStorage`; and +- `Policy`, implemented by independent capacity and truncation behaviors. + +It rejects the removed global `Config`, `PauliSumAlgorithm`, +`TableauMixtureAlgorithm`, and `TableauStorage` traits, as well as word +subtraits named `PauliWord`, `FermionWord`, or `LossyPauliWord`. Their +distinctions are expressed by `Word::Site` or by concrete wrapper types instead +of one-alphabet subtraits; the concrete `PauliWord` and `LossyPauliWord` type +names remain available. + +### Sparse-sum branch staging + +A propagation rule can turn one term into multiple terms. For example, a +Pauli rotation may produce: + +```text +c P -> c cos(theta) P + c sin(theta) P' +``` + +The existing entry can be updated while the active map is traversed, but a new +key cannot generally be inserted into that same map during mutable iteration. +New terms must therefore be staged and merged after the traversal. New keys +may collide with each other or with existing keys, so the merge accumulates +their coefficients. + +Only one staging mechanism is required for correctness. The current engine +uses two because they serve different performance paths: + +- an auxiliary map supports whole-map rewrites, combines output collisions as + they are produced, and retains its allocation across operations; and +- a reusable `Vec<(W, C)>` scratch buffer stages additional terms when existing + entries remain in place, avoiding an auxiliary-map insertion followed by a + second map probe during the merge. + +Conceptually: + +```text +whole-map rewrite: active map -> auxiliary map -> swap +in-place branching: active map + scratch buffer -> merge into active map +``` + +Neither physical mechanism is part of the public storage contract. A default +`SumStorage` implementation may privately contain both maps and the reusable +vector, matching the current `PauliSum` layout. A simpler backend may implement +both `map_insert` and `map_add` using only an auxiliary map; another backend +may use per-thread buffers. Whether retaining both mechanisms is worthwhile is +a benchmark decision, not a type-system requirement. + +## Indexable values + +Hash-enabled `Word` values and `Tableau` can both be expensive, mostly-stable +map keys. Their hashing contract should be expressed independently of any map. + +The tentative common capability is: + +```rust +pub trait Indexable: Clone + Eq + Hash { + type BuildHasher: std::hash::BuildHasher + Clone + Default + HashFinalize; + type HashCache; + + fn invalidate_hash(&mut self); +} +``` + +The important points are: + +- `BuildHasher` retains the current associated-type name and moves from + `Config` to the key type; +- the build hasher is associated with the key type, not with a configuration + bundle; +- the hash cache is an associated type, not a framework-provided concrete + `HashCache` structure; +- the author of the concrete data type owns the cache layout and invalidation + implementation; and +- equality and hashing cover only structural key identity, never cache fields + or incidental runtime state. + +For example, a word implementation may choose `u64`, `Option`, an atomic +representation, or another type. `ppvm-traits-2` does not prescribe the +fields that store it. + +### Lazy hashing and interior mutability + +Rust's `Hash::hash` receives `&self`. A cache that is first populated from +inside `Hash::hash` therefore requires interior mutability. Possible concrete +representations include: + +- `Cell` plus `Cell` for single-threaded keys; +- atomics for keys that must remain `Sync`; or +- a plain `u64` plus `bool` when cache preparation is eager or explicitly + performed through `&mut self` before hashing. + +This is deliberately a representation-level decision. A cache using interior +mutability will generally prevent the containing value from being `Copy`. + +Multiple concurrent readers may compute the same invalid hash more than once; +that is acceptable as long as they publish the same structural hash safely. +Structural mutation still requires `&mut self`. + +### Key mutation invariant + +An indexable value must not be structurally mutated while it is stored as a map +key. Cache invalidation makes a value correct for its next insertion or lookup; +it cannot make in-place mutation of an existing map key valid. + +Structural fields should therefore be private in the new representations. +Mutation must go through operations that invalidate the affected cache, or +through mutation guards that invalidate on completion. + +## Concrete word hashing + +Every concrete `Word` owns its `BuildHasher`, cache representation, structural +hash algorithm, and invalidation logic through `Indexable`. Pauli words hash +their X/Z content, lossy words compose Pauli and loss components, and future +fermion words hash their ordered factors. Factor order is part of fermionic +identity. + +The trait layer does not expose packed storage to support hashing. Concrete +implementations hash their private fields and may apply hasher-specific +finalization internally. Detailed layouts and component invalidation rules are +in [`word-data-structures.md`](word-data-structures.md). + +## Tableau indexability + +A tableau may itself be used as a key by a classical-mixture algorithm, so the +concrete tableau implements `Indexable` directly and owns a tableau-specific +hasher and cache representation. This does not imply that a tableau is a +`Word`; they only share the `Indexable` key capability. + +The tableau's structural hash is composed from its logical X/Z matrix, phase +plane, and optional loss plane. It excludes RNG, padding, cache state, and +physical matrix orientation. Separate component caches allow phase-only +changes to avoid rehashing the X/Z matrix. Physical transposition is a layout +change, not a logical mutation, and does not invalidate the structural hash. + +The concrete memory layout, component invalidation table, and canonical hash +order live in [`tableau-data-structure.md`](tableau-data-structure.md) so they +do not leak into the shared trait-system design. + +## Expected generic composition + +The intended composition is explicit: + +```rust +OperatorSum +Tableau +GeneralizedTableauSum +``` + +Domain-specific aliases or wrappers can preserve `PauliSum` and introduce +`FermionSum` without rebuilding a monolithic configuration trait. + +## Non-goals for the first prototype + +- Migrating the existing crates to `ppvm-traits-2` immediately. +- Merging `GeneralizedTableauSum` and `OperatorSum` in this iteration; only a + smaller proven common factor should be considered in the next iteration. +- Defining one collection interface shared by all algorithms. +- Requiring every sparse-sum storage backend to physically contain both an + auxiliary map and a scratch buffer. +- Selecting a single cache representation for every key type. +- Preserving `Copy` at the expense of correct lazy caching. +- Adding runtime dispatch for storage, hashing, or algorithm policies. + +## Open design questions + +1. What is the minimal method surface of `Indexable` beyond the associated + `BuildHasher` and `HashCache` types? Cache mechanics should remain owned by + the concrete type, but algorithms may need a common invalidation operation. +2. Which cache representations must support `Send` and `Sync` in the first + prototype? +3. Do benchmarks justify retaining both the auxiliary-map and vector-staging + fast paths in the default sparse-sum storage backend? diff --git a/docs/design/word-data-structures.md b/docs/design/word-data-structures.md new file mode 100644 index 00000000..5ece8e5c --- /dev/null +++ b/docs/design/word-data-structures.md @@ -0,0 +1,357 @@ +# Word data structures + +Status: design sketch + +## Purpose + +This document describes concrete data structures for standalone algebraic +words. The shared trait design is specified in +[`traits-2-configuration-and-hashing.md`](traits-2-configuration-and-hashing.md). + +The trait-level `Word` is representation-free but defines the common indexed +product operations: + +```rust +pub trait Word { + type Site; + + fn n_sites(&self) -> usize; + fn get(&self, index: usize) -> Self::Site; + fn set(&mut self, index: usize, site: Self::Site); + fn weight(&self) -> usize; +} +``` + +There is no `Word::Storage` associated type. Packed arrays, loss masks, ordered +factors, hash fields, and validity flags are private fields of concrete types. +Generic propagation and collection code uses behavioral traits and never names +the backing memory. + +`Word` does not extend `Indexable`. A word used as an `ACMap` key implements +both traits, while a mutable intermediate such as `Phased<_, NoHash>` can still +implement `Word` without pretending to be a valid map key. This separation +replaces the current `REHASH = false` use case with an explicit non-key mode. + +This document initially focuses on: + +- packed Pauli words; +- lossy Pauli words; +- phased-word composition; and +- lazy structural hash caches. + +Fermion-word storage will receive a separate design when fermionic propagation +is implemented. It will use the same `Word` interface with a different `Site`: +the word index records product order, while the fermionic site value records +the physical mode and creation or annihilation action. + +`weight()` counts non-identity factors according to the selected site type. +For a representation that stores only non-identity factors, it may equal +`n_sites()`. `set()` is the shared structural mutation boundary and must +preserve concrete invariants and invalidate the affected hash components. + +## Logical Pauli model + +An ordinary Pauli word is a fixed-width tensor product over `I`, `X`, `Y`, and +`Z`. Each site is represented by two logical bits: + +| Pauli | X bit | Z bit | +| --- | --- | --- | +| `I` | 0 | 0 | +| `X` | 1 | 0 | +| `Z` | 0 | 1 | +| `Y` | 1 | 1 | + +A lossy Pauli word adds a `Lost` state. Loss is exclusive with the four Pauli +states; a lost site has canonical bits `(x, z, lost) = (0, 0, 1)`. + +The exact Pauli alphabet and its lossy extension are site types rather than +word subtraits: + +```rust +pub enum Pauli { + I, + X, + Y, + Z, +} + +pub enum LossySite { + Present(S), + Lost, +} +``` + +An ordinary word implements `Word`. `WithLoss` implements +`Word>` and returns `Lost` for marked sites. This keeps +the word interface independent of the chosen operator alphabet. + +## `PauliWord` packed representation + +The initial packed implementation stores parallel fixed-size X and Z arrays: + +```rust +pub struct PauliWord { + xbits: BitArray, + zbits: BitArray, + nqubits: usize, + hash_cache: M, + _phantom: PhantomData, +} +``` + +`A` is an implementation parameter such as `[u8; N]` or `[usize; N]`. It is +not exposed through `Word`. The implementation validates that `nqubits` fits +the arrays and ignores or canonicalizes unused high bits. + +The structural identity is: + +```text +(nqubits, logical X bits, logical Z bits) +``` + +Equality and hashing exclude unused capacity, cache contents, cache validity, +and the `PhantomData` marker. + +All fields are private. Mutations go through methods that preserve unused-bit +invariants and invalidate the hash when a logical Pauli site changes. + +## Lossy Pauli word + +Loss is best modeled as an orthogonal wrapper around a Pauli word: + +```rust +pub struct WithLoss +where + W: Word, +{ + word: W, + lbits: BitArray, + loss_hash: M, +} +``` + +`A` is the packed bit-array backing selected by the concrete type. The public +alias retains the current `LossyPauliWord` name: + +```rust +pub type LossyPauliWord = WithLoss, A>; +``` + +`WithLoss` is a justified new implementation name because the proposal changes +the current standalone lossy representation into a generic wrapper. The +established domain-facing alias does not need to change. + +Neither `A` nor the underlying word's array type appears in `Word`. + +### Canonical loss invariant + +A lost site must contain identity in the underlying Pauli word: + +```text +lost[q] = 1 => word[q] = I +``` + +`set_lost(q)` first sets the underlying word to `I`, then sets the loss bit. +`set(q, LossySite::Present(p))` clears the loss bit and then writes `p`. This +prevents multiple physical encodings from representing the same logical lossy +word. + +The structural identity is: + +```text +(underlying Pauli word, logical loss bits) +``` + +`weight()` counts `X`, `Y`, `Z`, and `Lost`; `loss_weight()` counts only lost +sites. + +### Loss-specific behavior + +Generic lossy Pauli propagation sees `LossySite::Lost` through `Word` and +preserves or skips the site according to the operation's semantics. Operations +that create, clear, or count loss use inherent `WithLoss` methods: + +```rust +impl WithLoss +where + W: Word, +{ + pub fn is_lost(&self, qubit: usize) -> bool; + pub fn set_lost(&mut self, qubit: usize); + pub fn clear_loss(&mut self, qubit: usize); + pub fn loss_weight(&self) -> usize; +} +``` + +Loss channels and maximum-loss-weight truncation specialize directly on the +generic wrapper family: + +```rust +impl LossChannel + for OperatorSum, S, P> +where + W: Word, +{ + // ... +} +``` + +There are no traits named `PauliWord`, `LossyPauliWord`, or `FermionWord`. +`PauliWord` and `LossyPauliWord` remain concrete domain type names. Algorithms +select the algebra through `Word::Site`; loss-only operations remain inherent +to `WithLoss`. + +## Phased words + +Phase is another orthogonal wrapper: + +```rust +pub struct Phased +where + W: Word, + M: HashMode, +{ + word: W, + phase: Phase, + hash_state: M::State, +} +``` + +`Phased` is the generalized wrapper implementation. Pauli-facing code retains +the established name through an alias: + +```rust +pub type PhasedPauliWord = Phased; +``` + +For Pauli use, `Phase` represents `+1`, `+i`, `-1`, and `-i`. The wrapper may +also be useful for other word algebras, but algebra-specific multiplication is +implemented only under the appropriate specialized word bound. + +Loss and phase compose without a new combined representation: + +```rust +PhasedPauliWord, A>, Mode> +``` + +The wrapper fields are private so phase mutation cannot bypass component-cache +invalidation. + +## Hash ownership + +Every hash-enabled word implements `Indexable` and selects: + +- a build hasher; +- an `Indexable::HashCache` associated type; +- concrete cache fields; and +- the algorithm that hashes its private structural data. + +The associated cache type does not require every implementation to use the +same representation. Examples include: + +- a plain unsigned integer with a separate validity flag for eager or explicit + recomputation; +- `Cell` plus `Cell` for lazy single-threaded hashing; +- atomics for a `Sync` key; and +- `()` for a mode that deliberately provides no cache. + +Because `Hash::hash` receives `&self`, a cache populated lazily from that method +requires interior mutability. Such a representation will generally prevent +the containing word from being `Copy`. + +## Component hashes + +Hash composition follows the logical wrappers: + +```text +packed Pauli hash = hash(nqubits, X bits, Z bits) +lossy hash = combine(Pauli hash, loss hash) +phased hash = combine(inner-word hash, phase hash) +phased lossy hash = combine(Pauli hash, loss hash, phase hash) +``` + +`combine` must be ordered and domain-separated. It must not be an +unqualified XOR of arbitrary component digests. + +The phase has only four values, so `CompositeHash` can normally compute its +contribution from a small table or mixer without another cache. `CachedHash` +is justified only if profiling identifies a caller that benefits from caching +the combined phased value. + +Loss masks may be large, so `WithLoss` can cache the loss component separately +from the Pauli component. A loss-only mutation then avoids rehashing X/Z. + +## Invalidation rules + +| Mutation | Pauli component | Loss component | Phase component | +| --- | --- | --- | --- | +| Change ordinary Pauli site | invalidate | preserve | preserve | +| Mark identity site lost | preserve | invalidate | preserve | +| Mark nonidentity site lost | invalidate | invalidate | preserve | +| Clear loss to identity | preserve | invalidate | preserve | +| Replace loss with Pauli | invalidate if nonidentity | invalidate | preserve | +| Change phase | preserve | preserve | recompute or invalidate | + +Constructors compute caches eagerly or mark them invalid according to the +selected cache mode. Cloning copies a valid cache because the clone initially +has identical structural contents. + +## Hash modes + +Wrappers that are sometimes used only as mutable intermediate values should be +generic over hash behavior: + +```rust +pub trait HashMode { + type State: Clone; +} + +pub struct NoHash; +pub struct CompositeHash; +pub struct CachedHash(PhantomData); +``` + +- `NoHash` stores no wrapper cache and does not implement `Hash` or + `Indexable` for the wrapper. +- `CompositeHash` implements `Hash` by combining cached inner components on + demand, with no combined cache. +- `CachedHash` stores a lazily or eagerly maintained combined cache. + +The concrete policy protocol should be finalized while implementing the first +two modes. It should not expose cache state to propagation algorithms. + +## Ordering and serialization + +`Eq`, `Ord`, `Hash`, display, parsing, and serialization must agree on logical +identity: + +- Pauli sites compare in a documented order. +- Loss participates after the underlying Pauli content or through an explicit + `LossySite` ordering. +- Phase participates only when the phased wrapper's identity includes it. +- Unused bits and cache state never participate. + +Serialization uses logical symbols and lengths, not raw native-word memory, so +it remains stable across storage widths and platforms. + +## Prototype validation + +The prototype should include: + +- round-trip parsing tests for ordinary and lossy symbols; +- property tests comparing packed operations with a simple symbol vector; +- tests enforcing `lost => underlying I` after every mutator; +- equality/hash agreement tests for all wrappers and modes; +- tests showing loss-only changes preserve the Pauli hash component; +- tests showing phase-only changes preserve Pauli and loss components; +- tests proving unused high bits do not affect identity; +- `Send`/`Sync` assertions for cache modes intended for concurrent maps; and +- benchmarks comparing eager, lazy, composite, and uncached hashing. + +## Open questions + +1. Should `WithLoss` permit a loss-mask storage width different from its inner + word's packed width? +2. Does any real phased-word caller benefit from a combined `CachedHash` mode? +3. Does `Indexable::HashCache` have a generic consumer, or should the cache + type also remain entirely private like word storage? From f2a5e7f2c94052c487d601eae3bf3e9f5a699924 Mon Sep 17 00:00:00 2001 From: David Plankensteiner Date: Thu, 23 Jul 2026 12:35:27 +0200 Subject: [PATCH 2/2] docs(design): reconcile traits 2 decisions --- docs/design/tableau-data-structure.md | 130 +++++++++-- .../traits-2-configuration-and-hashing.md | 127 ++++++----- docs/design/word-data-structures.md | 201 +++++++----------- 3 files changed, 254 insertions(+), 204 deletions(-) diff --git a/docs/design/tableau-data-structure.md b/docs/design/tableau-data-structure.md index 7b283359..7d5379dc 100644 --- a/docs/design/tableau-data-structure.md +++ b/docs/design/tableau-data-structure.md @@ -24,7 +24,7 @@ orientation. - Store the X/Z tableau bits in contiguous, aligned, bit-packed memory. - Make one- and two-qubit column operations efficient. - Permit temporary transposition for row-oriented elimination and collapse. -- Keep phases and optional loss state independently addressable and hashable. +- Keep phases and per-qubit loss state independently addressable and hashable. - Hash the logical tableau independently of its current physical orientation. - Avoid parameterizing the trait system by a row type or Pauli-word storage. - Leave room for SIMD-width and padding changes without changing public @@ -61,8 +61,7 @@ The logical state consists of: - an X matrix of shape `(2n, n)`; - a Z matrix of shape `(2n, n)`; - a phase plane of length `2n`; and -- for lossy variants, a loss plane of length `n` or another explicitly defined - logical shape. +- a per-qubit loss plane of length `n`. This model does not imply a physical row object. @@ -77,7 +76,7 @@ pub struct TableauData { x_offset: usize, z_offset: usize, phase_offset: usize, - loss_offset: Option, + loss_offset: usize, major_stride: usize, n_qubits: usize, orientation: Orientation, @@ -98,6 +97,47 @@ Padding must either be kept zero or excluded from equality and hashing. Zeroed padding is preferable because it permits bulk comparison and hashing of canonical ranges. +## Loss ownership + +The existing concrete `Tableau` always owns the per-qubit loss plane. This is a +capability of the same tableau type, not a `LossyTableau` variant or a +`Tableau` parameter: + +```rust +pub struct Tableau { + data: TableauData, + lost_count: usize, + // hash caches and RNG +} + +pub struct GeneralizedTableau { + tableau: Tableau, + coefficients: S, + // threshold and measurement record +} +``` + +`GeneralizedTableau` therefore has no separate `is_lost: Vec`. Its gate, +noise, and measurement algorithms query and mutate the loss plane owned by the +inner tableau. + +`lost_count` is derived metadata used to preserve a fast +`lost_count == 0` path. It is excluded from equality and hashing; debug builds +should verify that it equals the population count of the logical loss plane. +When no qubit is lost, gate kernels enter their existing lossless bulk path. +When loss is present, a one-qubit gate skips a lost target and a two-qubit gate +skips the operation if either target is lost, matching the current generalized +tableau semantics. Batch kernels should mask or skip lost targets without +allocating filtered target vectors. + +This ownership enables a pure Clifford-plus-loss simulation with the same +`Tableau`. Loss events use the pure Clifford collapse procedure before marking +the affected qubit lost. Generalized loss events still use the +coefficient-aware generalized measurement procedure before marking the loss; +moving the bit does not move that algorithm. The pure path covers loss models +whose conditional trajectory remains a stabilizer state; faithful +non-Clifford survival back-action remains generalized. + ## Column-major orientation The default orientation should make the generator dimension contiguous for a @@ -183,11 +223,12 @@ impl Tableau { pub fn h(&mut self, qubit: usize); pub fn cnot(&mut self, control: usize, target: usize); - pub fn measure_z(&mut self, qubit: usize) -> bool; + pub fn measure_z(&mut self, qubit: usize) -> Option; pub fn x_bit(&self, generator: usize, qubit: usize) -> bool; pub fn z_bit(&self, generator: usize, qubit: usize) -> bool; pub fn phase(&self, generator: usize) -> Phase; + pub fn is_lost(&self, qubit: usize) -> bool; } ``` @@ -200,6 +241,44 @@ There should be no `stabilizers_mut() -> &mut [Row]` or equivalent escape hatch that bypasses hash invalidation and orientation invariants. Specialized internal row operations operate through `TableauData` or a transposition guard. +## Measurement algorithms + +The Rust behavioral boundary is one loss-aware trait: + +```rust +pub trait Measure { + fn measure(&mut self, qubit: usize) -> Option; + + fn measure_many(&mut self, targets: &[usize]) -> Vec> { + targets.iter().map(|&q| self.measure(q)).collect() + } +} +``` + +`Some(false)` and `Some(true)` represent computational-basis outcomes; `None` +represents a lost qubit. This is the existing core representation used by +`GeneralizedTableau`. The Python binding may continue mapping it to +`MeasurementResult::{ZERO, ONE, LOST}`. The old public split between +`Measure -> bool` and `LossyMeasure -> Option` is removed. The bare +boolean Clifford measurement routine becomes a private helper called only +after the public implementation has established that the target is present. + +The common trait and result type do not imply a common measurement algorithm: + +- `Tableau::measure` checks the loss plane and then uses the pure Clifford + stabilizer measurement procedure. It does not decompose against a sparse + generalized-state coefficient basis. +- `GeneralizedTableau::measure` checks the inner tableau's loss plane, then + decomposes the measured Pauli into stabilizers and destabilizers and updates + the sparse coefficients. This path is fundamentally \(O(n^2)\). + +The physical tableau must make both stabilizer and destabilizer generators +available to the generalized decomposition, but that requirement must not be +promoted into the `Measure` trait or force the generalized algorithm onto the +pure Clifford implementation. Pure and generalized measurement performance +must be benchmarked separately; Stim's inverse-tableau measurement +optimizations are not complexity promises for the generalized algorithm. + ## Gate access patterns The implementation should categorize mutations by physical access and hash @@ -227,30 +306,31 @@ hasher and cache representations; neither is inherited from `PauliWord`. The structural hash is composed from independent logical components: ```text -tableau hash = combine(xz hash, phase hash[, loss hash]) +tableau hash = combine(xz hash, phase hash, loss hash) ``` The X and Z planes share an `xz_hash` cache because most Clifford mutations update them together. The phase plane has a separate cache so Pauli -conjugations and sign changes do not force a matrix rehash. Loss should use a -third cache if it is independently mutable and part of key identity. +conjugations and sign changes do not force a matrix rehash. The independently +mutable loss plane uses a third cache. ```rust pub struct Tableau { data: TableauData, - xz_hash: XzHashCache, - phase_hash: PhaseHashCache, - loss_hash: Option, + lost_count: usize, + xz_hash: OnceLock, + phase_hash: OnceLock, + loss_hash: OnceLock, rng: SmallRng, } ``` -The cache types above are concrete choices made by the tableau author. The -shared `Indexable::HashCache` associated type may name a small aggregate or -another representation chosen by the implementation. +The cache fields are private representation choices made by the tableau +author. `Indexable` exposes only the associated build hasher; it does not name +cache types or expose invalidation. Equality and hashing include logical qubit count, generator order, all logical -X/Z bits, phases, and loss state when present. They exclude: +X/Z bits, phases, and loss state. They exclude: - RNG state; - allocation capacity; @@ -266,14 +346,13 @@ The component invalidation rules are: | Direct phase change | preserve | invalidate | preserve | | `H`, `S`, `CNOT`, `CZ` | invalidate | invalidate if changed | preserve | | Row multiplication | invalidate | invalidate if changed | preserve | -| Mark or clear loss | preserve | preserve | invalidate | +| Toggle a loss bit | preserve | preserve | invalidate | | Physical transpose | preserve | preserve | preserve | | RNG update | preserve | preserve | preserve | The current `ppvm-tableau-sum` split between `word_fingerprint` and `phase_loss_hash` is evidence that component hashing matters. The new tableau -owns these components directly and separates loss as well when the logical -model permits it. +owns X/Z, phase, and loss components directly. ## Cloning and mixture use @@ -307,12 +386,18 @@ not belong in the PPVM trait system. The prototype should include: - property tests comparing gates and measurements with the existing tableau; +- differential tests for the pure Clifford and generalized measurement + algorithms, including lost targets; +- tests that gates skip lost targets and retain the `lost_count == 0` fast + path; - round-trip tests for column-major -> row-major -> column-major transpose; - equality and hash tests across physical orientations; - tests proving phase-only changes preserve the X/Z hash cache; - tests proving padding never affects equality or hashing; - benchmarks for one- and two-qubit Clifford gates; -- benchmarks for deterministic and random measurement paths; +- separate benchmarks for pure Clifford and generalized deterministic and + random measurement paths; +- benchmarks for lossless gates, sparse loss, and Clifford-plus-loss sampling; - benchmarks for clone-and-mutate mixture branching; and - benchmarks comparing permanent row-major, permanent column-major, and temporary-transpose variants on representative circuits. @@ -321,9 +406,8 @@ The prototype should include: 1. What block width and alignment should the first implementation use? 2. Should phases occupy one or two bits per generator in the tableau model? -3. Should loss be stored per qubit, per generator, or outside the core tableau? -4. Which operations should receive a transposition guard versus performing +3. Which operations should receive a transposition guard versus performing column-strided work directly? -5. Does a dual-orientation representation outperform temporary transposition +4. Does a dual-orientation representation outperform temporary transposition for PPVM's measurement-heavy workloads? -6. Should PPVM ultimately store a forward or inverse tableau? +5. Should PPVM ultimately store a forward or inverse tableau? diff --git a/docs/design/traits-2-configuration-and-hashing.md b/docs/design/traits-2-configuration-and-hashing.md index 5d242a2a..2709ea04 100644 --- a/docs/design/traits-2-configuration-and-hashing.md +++ b/docs/design/traits-2-configuration-and-hashing.md @@ -98,20 +98,23 @@ pub struct FermionSite { } ``` -Thus an ordinary packed Pauli word implements `Word`, a lossy -wrapper implements `Word>`, and a future ordered -fermionic product can implement `Word`. A fermionic word's -index denotes factor order; `FermionSite` carries the physical mode. For a -dense Pauli word, the index is the qubit and `n_sites()` is its width. +Thus an ordinary packed Pauli word implements `Word`, a +concrete packed lossy word implements `Word>`, and a +future ordered fermionic product can implement `Word`. A +fermionic word's index denotes factor order; `FermionSite` carries the physical +mode. For a dense Pauli word, the index is the qubit and `n_sites()` is its +width. `weight()` is the number of non-identity factors according to the concrete site alphabet; an ordered representation that stores no explicit identities may therefore have `weight() == n_sites()`. Implementations of `set()` preserve their representation invariants and invalidate the affected hash components. -The concrete `WithLoss` family provides loss mutation and -`loss_weight()` as inherent methods. Loss channels and loss-specific -truncation specialize directly on `WithLoss`; there is no one-implementation -`LossyPauliWord` capability trait. +The concrete `LossyPauliWord` stores packed X, Z, and loss planes directly and +provides loss mutation and `loss_weight()` as inherent methods. Loss channels +and loss-specific truncation specialize directly on that concrete type; there +is no one-implementation `LossyPauliWord` capability trait. A generic loss +wrapper should be reconsidered only after a second real word representation +needs the same composition. Concrete packed Pauli, lossy, phased, and hash-cache layouts are described in [`word-data-structures.md`](word-data-structures.md). None of those layouts is @@ -144,12 +147,31 @@ pub trait RotationOne { pub trait PauliError { fn pauli_error(&mut self, qubit: usize, probabilities: [C; 3]); } + +pub trait Measure { + fn measure(&mut self, qubit: usize) -> Option; + + fn measure_many(&mut self, targets: &[usize]) -> Vec> { + targets.iter().map(|&q| self.measure(q)).collect() + } +} ``` The same concrete tableau may implement a numeric trait for every supported coefficient type without storing that coefficient type. Measurement and reset traits likewise expose behavior and result types without exposing tableau -rows, packed matrix blocks, or matrix orientation. +rows, packed matrix blocks, or matrix orientation. `Measure` is loss-aware for +both `Tableau` and `GeneralizedTableau`: `Some(false)` and `Some(true)` denote +computational-basis outcomes and `None` denotes a lost qubit. The former +`Measure -> bool` and `LossyMeasure -> Option` split is removed. +Python may continue translating this Rust representation to its +`MeasurementResult` enum. + +Sharing the result type does not share the measurement algorithm. `Tableau` +uses the pure Clifford measurement procedure. `GeneralizedTableau` performs +its coefficient-aware stabilizer/destabilizer decomposition and update, which +is always \(O(n^2)\). The shared behavioral trait must not force the latter +algorithm or its scratch state into the concrete tableau. There is no shared `TableauStorage` trait in the first design. If multiple tableau implementations are later useful, each concrete type can implement @@ -259,6 +281,20 @@ implementation detail for the prototype. The important boundary is that the trait preserves the current semantic operation names without exposing physical auxiliary maps or scratch buffers. +`SumStorage` owns the semantic whole-map operations and its reusable workspace. +It delegates to the lower-level `ACMap` batch kernels without restoring the +removed map-to-map `ACMapInsert::map_insert` method: + +```text +SumStorage::map_insert -> ACMapInsert::map_insert_vec +SumStorage::map_insert_multiple -> ACMapInsert::map_insert_multiple +SumStorage::map_add -> ACMapAddAssign::map_add_assign +``` + +This boundary is compatible with +`refactor/shrink-internal-trait-surface`: the higher-level sparse-sum operation +remains, while the dead low-level primitive stays removed. + The generalized engine may be named `OperatorSum`, but the Pauli specialization retains the existing domain-facing `PauliSum` name. This is a new internal generalization, not a requirement to rename Pauli call sites. @@ -300,9 +336,9 @@ changed according to whether their underlying responsibility changes: | `PauliWordTrait` | `Word` plus `Indexable` where used as a key | Word operations are generalized through `Word::Site`; hashing becomes a separate capability. | | `n_qubits`, `get`, `set`, `weight` | `n_sites`, `get`, `set`, `weight` | Only the Pauli-specific extent name changes; the other operation names stay. | | concrete `PauliWord` | `PauliWord` | The packed X/Z word is the same domain concept. | -| concrete `LossyPauliWord` | `LossyPauliWord` alias over `WithLoss` | The public concept stays; only the implementation becomes a generic wrapper. | -| `PhasedPauliWord` | `PhasedPauliWord` alias over `Phased` | The public Pauli name stays; the implementation becomes a word-generic wrapper. | -| `rehash` | `invalidate_hash` for lazy modes | Recalculation changes from eager mutation-time work to lazy demand-time work. | +| concrete `LossyPauliWord` | `LossyPauliWord` | The packed X/Z/loss representation remains concrete and flattened. | +| `PhasedPauliWord` | `PhasedPauliWord` alias over non-indexable `Phased` | The wrapper is generic over ordinary and lossy words but is not a production map key. | +| `rehash` | private cache invalidation | Recalculation changes from eager mutation-time work to lazy demand-time work without exposing cache mechanics through `Indexable`. | | `Strategy` | `Policy` | Intentional terminology change requested for this redesign. | | `ACMap` | `ACMap` | The associative coefficient map has the same role. | | `PauliSum::data`, `map_insert`, `map_add` | same method names on `SumStorage` | These semantic operations already match the proposed boundary. | @@ -310,11 +346,12 @@ changed according to whether their underlying responsibility changes: | `PauliSum` | `PauliSum` over generalized `OperatorSum` machinery | Pauli-facing code keeps its established name; `OperatorSum` names the new cross-algebra engine. | | `GeneralizedTableauSum` | `GeneralizedTableauSum` | The classical mixture algorithm remains the same concept. | | `EntryStore`, `VecStorage`, `MapStorage` | unchanged | The proposal uses the existing storage boundary and implementations. | -| `BuildHasher`, `HashFinalize` | unchanged | The hasher concepts remain, but ownership moves from `Config` to each indexable key type. | +| `BuildHasher` | associated with `Indexable` | Hasher ownership moves from `Config` to each indexable key type. | +| `HashFinalize` | removed from the shared contract | Concrete keys may finalize or compose hashes privately. | | `PauliStorage` | removed | Packed backing storage becomes private to the concrete word representation. | -Names such as `Word`, `Indexable`, `SumStorage`, `WithLoss`, and -`OperatorSum` are therefore new because they denote abstractions that do not +Names such as `Word`, `Indexable`, `SumStorage`, and `OperatorSum` are +therefore new because they denote abstractions that do not exist in the current implementation, not because the existing API is being renamed wholesale. @@ -339,7 +376,7 @@ This rule keeps: It rejects the removed global `Config`, `PauliSumAlgorithm`, `TableauMixtureAlgorithm`, and `TableauStorage` traits, as well as word subtraits named `PauliWord`, `FermionWord`, or `LossyPauliWord`. Their -distinctions are expressed by `Word::Site` or by concrete wrapper types instead +distinctions are expressed by `Word::Site` or by concrete types instead of one-alphabet subtraits; the concrete `PauliWord` and `LossyPauliWord` type names remain available. @@ -386,14 +423,11 @@ a benchmark decision, not a type-system requirement. Hash-enabled `Word` values and `Tableau` can both be expensive, mostly-stable map keys. Their hashing contract should be expressed independently of any map. -The tentative common capability is: +The common capability is intentionally minimal: ```rust pub trait Indexable: Clone + Eq + Hash { - type BuildHasher: std::hash::BuildHasher + Clone + Default + HashFinalize; - type HashCache; - - fn invalidate_hash(&mut self); + type BuildHasher: std::hash::BuildHasher + Clone + Default; } ``` @@ -403,34 +437,24 @@ The important points are: `Config` to the key type; - the build hasher is associated with the key type, not with a configuration bundle; -- the hash cache is an associated type, not a framework-provided concrete - `HashCache` structure; -- the author of the concrete data type owns the cache layout and invalidation - implementation; and +- cache layout and invalidation are private representation invariants of the + concrete type; and - equality and hashing cover only structural key identity, never cache fields or incidental runtime state. -For example, a word implementation may choose `u64`, `Option`, an atomic -representation, or another type. `ppvm-traits-2` does not prescribe the -fields that store it. +No generic consumer needs to name a cache type or request invalidation. +Structural mutation already occurs through `&mut self`, so each mutator can +clear the affected private cache as part of maintaining its concrete +invariants. ### Lazy hashing and interior mutability -Rust's `Hash::hash` receives `&self`. A cache that is first populated from -inside `Hash::hash` therefore requires interior mutability. Possible concrete -representations include: - -- `Cell` plus `Cell` for single-threaded keys; -- atomics for keys that must remain `Sync`; or -- a plain `u64` plus `bool` when cache preparation is eager or explicitly - performed through `&mut self` before hashing. - -This is deliberately a representation-level decision. A cache using interior -mutability will generally prevent the containing value from being `Copy`. - -Multiple concurrent readers may compute the same invalid hash more than once; -that is acceptable as long as they publish the same structural hash safely. -Structural mutation still requires `&mut self`. +Rust's `Hash::hash` receives `&self`, so shipped indexable words and tableaus +use private component `OnceLock` caches. `Hash::hash` may populate a cache +through shared access, while structural mutators clear affected cells through +their exclusive `&mut self` access. This preserves `Send + Sync` for the +shipped representations. `Indexable` itself does not require either +concurrency bound. ### Key mutation invariant @@ -444,8 +468,8 @@ through mutation guards that invalidate on completion. ## Concrete word hashing -Every concrete `Word` owns its `BuildHasher`, cache representation, structural -hash algorithm, and invalidation logic through `Indexable`. Pauli words hash +Every concrete `Word` owns its `BuildHasher`, private cache representation, +structural hash algorithm, and invalidation logic. Pauli words hash their X/Z content, lossy words compose Pauli and loss components, and future fermion words hash their ordered factors. Factor order is part of fermionic identity. @@ -463,7 +487,7 @@ hasher and cache representation. This does not imply that a tableau is a `Word`; they only share the `Indexable` key capability. The tableau's structural hash is composed from its logical X/Z matrix, phase -plane, and optional loss plane. It excludes RNG, padding, cache state, and +plane, and per-qubit loss plane. It excludes RNG, padding, cache state, and physical matrix orientation. Separate component caches allow phase-only changes to avoid rehashing the X/Z matrix. Physical transposition is a layout change, not a logical mutation, and does not invalidate the structural hash. @@ -493,16 +517,11 @@ Domain-specific aliases or wrappers can preserve `PauliSum` and introduce - Defining one collection interface shared by all algorithms. - Requiring every sparse-sum storage backend to physically contain both an auxiliary map and a scratch buffer. -- Selecting a single cache representation for every key type. +- Exposing cache representation or invalidation through `Indexable`. - Preserving `Copy` at the expense of correct lazy caching. - Adding runtime dispatch for storage, hashing, or algorithm policies. ## Open design questions -1. What is the minimal method surface of `Indexable` beyond the associated - `BuildHasher` and `HashCache` types? Cache mechanics should remain owned by - the concrete type, but algorithms may need a common invalidation operation. -2. Which cache representations must support `Send` and `Sync` in the first - prototype? -3. Do benchmarks justify retaining both the auxiliary-map and vector-staging +1. Do benchmarks justify retaining both the auxiliary-map and vector-staging fast paths in the default sparse-sum storage backend? diff --git a/docs/design/word-data-structures.md b/docs/design/word-data-structures.md index 5ece8e5c..30d87f22 100644 --- a/docs/design/word-data-structures.md +++ b/docs/design/word-data-structures.md @@ -28,9 +28,9 @@ Generic propagation and collection code uses behavioral traits and never names the backing memory. `Word` does not extend `Indexable`. A word used as an `ACMap` key implements -both traits, while a mutable intermediate such as `Phased<_, NoHash>` can still -implement `Word` without pretending to be a valid map key. This separation -replaces the current `REHASH = false` use case with an explicit non-key mode. +both traits, while a mutable intermediate such as `Phased` can implement +`Word` without pretending to be a valid map key. This separation replaces the +current `REHASH = false` use case with an explicitly non-indexable type. This document initially focuses on: @@ -81,7 +81,7 @@ pub enum LossySite { } ``` -An ordinary word implements `Word`. `WithLoss` implements +An ordinary word implements `Word`. `LossyPauliWord` implements `Word>` and returns `Lost` for marked sites. This keeps the word interface independent of the chosen operator alphabet. @@ -90,18 +90,19 @@ the word interface independent of the chosen operator alphabet. The initial packed implementation stores parallel fixed-size X and Z arrays: ```rust -pub struct PauliWord { +pub struct PauliWord { xbits: BitArray, zbits: BitArray, nqubits: usize, - hash_cache: M, - _phantom: PhantomData, + hash_cache: OnceLock, + _hasher: PhantomData H>, } ``` -`A` is an implementation parameter such as `[u8; N]` or `[usize; N]`. It is -not exposed through `Word`. The implementation validates that `nqubits` fits -the arrays and ignores or canonicalizes unused high bits. +`A` is an implementation parameter such as `[u8; N]` or `[usize; N]`, and `H` +is the build hasher associated through `Indexable`. Neither is exposed through +`Word`. The implementation validates that `nqubits` fits the arrays and +ignores or canonicalizes unused high bits. The structural identity is: @@ -117,41 +118,36 @@ invariants and invalidate the hash when a logical Pauli site changes. ## Lossy Pauli word -Loss is best modeled as an orthogonal wrapper around a Pauli word: +The first prototype keeps the established lossy Pauli word as a flattened +packed representation: ```rust -pub struct WithLoss -where - W: Word, -{ - word: W, +pub struct LossyPauliWord { + xbits: BitArray, + zbits: BitArray, lbits: BitArray, - loss_hash: M, + nqubits: usize, + xz_hash_cache: OnceLock, + loss_hash_cache: OnceLock, + _hasher: PhantomData H>, } ``` -`A` is the packed bit-array backing selected by the concrete type. The public -alias retains the current `LossyPauliWord` name: - -```rust -pub type LossyPauliWord = WithLoss, A>; -``` - -`WithLoss` is a justified new implementation name because the proposal changes -the current standalone lossy representation into a generic wrapper. The -established domain-facing alias does not need to change. - -Neither `A` nor the underlying word's array type appears in `Word`. +Inlining all three planes avoids wrapper nesting and keeps the lossy hot path +and component hashes direct. A generic loss wrapper is not introduced until a +second real word representation demonstrates that it needs the same +composition. `A` and `H` remain private implementation parameters from the +perspective of `Word`. ### Canonical loss invariant -A lost site must contain identity in the underlying Pauli word: +A lost site must contain identity in its X/Z planes: ```text -lost[q] = 1 => word[q] = I +lost[q] = 1 => xbits[q] = 0 and zbits[q] = 0 ``` -`set_lost(q)` first sets the underlying word to `I`, then sets the loss bit. +`set_lost(q)` first clears the X/Z bits, then sets the loss bit. `set(q, LossySite::Present(p))` clears the loss bit and then writes `p`. This prevents multiple physical encodings from representing the same logical lossy word. @@ -159,7 +155,7 @@ word. The structural identity is: ```text -(underlying Pauli word, logical loss bits) +(nqubits, logical X bits, logical Z bits, logical loss bits) ``` `weight()` counts `X`, `Y`, `Z`, and `Lost`; `loss_weight()` counts only lost @@ -169,13 +165,10 @@ sites. Generic lossy Pauli propagation sees `LossySite::Lost` through `Word` and preserves or skips the site according to the operation's semantics. Operations -that create, clear, or count loss use inherent `WithLoss` methods: +that create, clear, or count loss use inherent `LossyPauliWord` methods: ```rust -impl WithLoss -where - W: Word, -{ +impl LossyPauliWord { pub fn is_lost(&self, qubit: usize) -> bool; pub fn set_lost(&mut self, qubit: usize); pub fn clear_loss(&mut self, qubit: usize); @@ -184,13 +177,11 @@ where ``` Loss channels and maximum-loss-weight truncation specialize directly on the -generic wrapper family: +concrete lossy word: ```rust -impl LossChannel - for OperatorSum, S, P> -where - W: Word, +impl LossChannel + for OperatorSum, S, P> { // ... } @@ -199,21 +190,19 @@ where There are no traits named `PauliWord`, `LossyPauliWord`, or `FermionWord`. `PauliWord` and `LossyPauliWord` remain concrete domain type names. Algorithms select the algebra through `Word::Site`; loss-only operations remain inherent -to `WithLoss`. +to `LossyPauliWord`. ## Phased words Phase is another orthogonal wrapper: ```rust -pub struct Phased +pub struct Phased where W: Word, - M: HashMode, { word: W, phase: Phase, - hash_state: M::State, } ``` @@ -221,43 +210,34 @@ where the established name through an alias: ```rust -pub type PhasedPauliWord = Phased; +pub type PhasedPauliWord = Phased; ``` For Pauli use, `Phase` represents `+1`, `+i`, `-1`, and `-i`. The wrapper may -also be useful for other word algebras, but algebra-specific multiplication is -implemented only under the appropriate specialized word bound. +wrap both ordinary and lossy Pauli words. It may also be useful for other word +algebras, but algebra-specific multiplication is implemented only under the +appropriate specialized word bound. Loss and phase compose without a new combined representation: ```rust -PhasedPauliWord, A>, Mode> +PhasedPauliWord> ``` -The wrapper fields are private so phase mutation cannot bypass component-cache -invalidation. +No phased word is a production map key in the first prototype, so `Phased` +does not implement `Hash` or `Indexable` and stores no hash mode or cache. ## Hash ownership -Every hash-enabled word implements `Indexable` and selects: +Every hash-enabled word implements `Indexable`, associates its build hasher, +and privately owns the fields and algorithm used to cache its structural +hash. Cache representation and invalidation are not exposed through +`Indexable`. -- a build hasher; -- an `Indexable::HashCache` associated type; -- concrete cache fields; and -- the algorithm that hashes its private structural data. - -The associated cache type does not require every implementation to use the -same representation. Examples include: - -- a plain unsigned integer with a separate validity flag for eager or explicit - recomputation; -- `Cell` plus `Cell` for lazy single-threaded hashing; -- atomics for a `Sync` key; and -- `()` for a mode that deliberately provides no cache. - -Because `Hash::hash` receives `&self`, a cache populated lazily from that method -requires interior mutability. Such a representation will generally prevent -the containing word from being `Copy`. +The shipped indexable words use component `OnceLock` caches. `Hash::hash` +can populate them through `&self`; structural mutators clear affected cells +through `&mut self`. This preserves `Send + Sync` without imposing either +bound on the `Indexable` trait. ## Component hashes @@ -266,59 +246,27 @@ Hash composition follows the logical wrappers: ```text packed Pauli hash = hash(nqubits, X bits, Z bits) lossy hash = combine(Pauli hash, loss hash) -phased hash = combine(inner-word hash, phase hash) -phased lossy hash = combine(Pauli hash, loss hash, phase hash) ``` `combine` must be ordered and domain-separated. It must not be an unqualified XOR of arbitrary component digests. -The phase has only four values, so `CompositeHash` can normally compute its -contribution from a small table or mixer without another cache. `CachedHash` -is justified only if profiling identifies a caller that benefits from caching -the combined phased value. - -Loss masks may be large, so `WithLoss` can cache the loss component separately -from the Pauli component. A loss-only mutation then avoids rehashing X/Z. +Loss masks may be large, so `LossyPauliWord` caches the loss component +separately from the X/Z component. A loss-only mutation then avoids rehashing +X/Z. `Phased` is absent from this composition because it is not indexable. ## Invalidation rules -| Mutation | Pauli component | Loss component | Phase component | -| --- | --- | --- | --- | -| Change ordinary Pauli site | invalidate | preserve | preserve | -| Mark identity site lost | preserve | invalidate | preserve | -| Mark nonidentity site lost | invalidate | invalidate | preserve | -| Clear loss to identity | preserve | invalidate | preserve | -| Replace loss with Pauli | invalidate if nonidentity | invalidate | preserve | -| Change phase | preserve | preserve | recompute or invalidate | - -Constructors compute caches eagerly or mark them invalid according to the -selected cache mode. Cloning copies a valid cache because the clone initially -has identical structural contents. - -## Hash modes - -Wrappers that are sometimes used only as mutable intermediate values should be -generic over hash behavior: - -```rust -pub trait HashMode { - type State: Clone; -} - -pub struct NoHash; -pub struct CompositeHash; -pub struct CachedHash(PhantomData); -``` - -- `NoHash` stores no wrapper cache and does not implement `Hash` or - `Indexable` for the wrapper. -- `CompositeHash` implements `Hash` by combining cached inner components on - demand, with no combined cache. -- `CachedHash` stores a lazily or eagerly maintained combined cache. +| Mutation | X/Z component | Loss component | +| --- | --- | --- | +| Change ordinary Pauli site | invalidate | preserve | +| Mark identity site lost | preserve | invalidate | +| Mark nonidentity site lost | invalidate | invalidate | +| Clear loss to identity | preserve | invalidate | +| Replace loss with Pauli | invalidate if nonidentity | invalidate | -The concrete policy protocol should be finalized while implementing the first -two modes. It should not expose cache state to propagation algorithms. +Constructors leave caches empty. Cloning may copy a valid cached value because +the clone initially has identical structural contents. ## Ordering and serialization @@ -328,7 +276,8 @@ identity: - Pauli sites compare in a documented order. - Loss participates after the underlying Pauli content or through an explicit `LossySite` ordering. -- Phase participates only when the phased wrapper's identity includes it. +- Phase participates in equality and serialization for `Phased`, but not in + map-key hashing because the wrapper is not indexable. - Unused bits and cache state never participate. Serialization uses logical symbols and lengths, not raw native-word memory, so @@ -340,18 +289,16 @@ The prototype should include: - round-trip parsing tests for ordinary and lossy symbols; - property tests comparing packed operations with a simple symbol vector; -- tests enforcing `lost => underlying I` after every mutator; -- equality/hash agreement tests for all wrappers and modes; -- tests showing loss-only changes preserve the Pauli hash component; -- tests showing phase-only changes preserve Pauli and loss components; +- tests enforcing `lost => X/Z identity` after every mutator; +- equality/hash agreement tests for ordinary and lossy indexable words; +- tests showing loss-only changes preserve the X/Z hash component; +- equality and serialization tests for ordinary and lossy `Phased` values; - tests proving unused high bits do not affect identity; -- `Send`/`Sync` assertions for cache modes intended for concurrent maps; and -- benchmarks comparing eager, lazy, composite, and uncached hashing. +- `Send`/`Sync` assertions for shipped indexable words; and +- benchmarks comparing uncached structural hashing with the private lazy + component caches. ## Open questions -1. Should `WithLoss` permit a loss-mask storage width different from its inner - word's packed width? -2. Does any real phased-word caller benefit from a combined `CachedHash` mode? -3. Does `Indexable::HashCache` have a generic consumer, or should the cache - type also remain entirely private like word storage? +1. Should the loss plane use the same packed array width as the X/Z planes, or + a separately selected private width?