Multipathogen

[Julian-Patzner]

Multipathogen: Concurrent Multi-Disease Simulation with a Customizable Transmission, Immunity & Infectiousness Architecture

Overview

This PR generalizes GEMS from a single-disease simulator into a true multipathogen framework, allowing an arbitrary number of distinct pathogens to circulate, interact, and co-infect the same population concurrently within a single simulation run.

The core change is a move away from per-individual scalar disease fields toward a per-(individual, pathogen) infection model backed by a cache + sharded-overflow registry architecture. On top of this, the PR introduces a fully composable transmission system (base rates × stackable modifiers), customizable infectiousness and immunity profiles, cross-pathogen interactions (viral interference and cross-immunity), and a type-stable pathogen parameterization that keeps the multi-pathogen hot loops dispatch-free. The PR also makes the entire logging, post-processing, reporting, and plotting pipeline pathogen-aware, and performs a large-scale repository restructure to organize the codebase around its actual domains (pathogen, population, settings, registries, simulation, analysis, logging, contacts).

---

Key Changes & Features

1. Per-(Individual, Pathogen) Infection Model

The simulation no longer tracks a single infection per agent. Each individual can now carry multiple concurrent infections, one per distinct pathogen.

• Bits-Type InfectionState: A new immutable, isbits InfectionState record encapsulates the entire disease timeline for a single infection (exposure, infectiousness_onset, symptom_onset … recovery, death), plus its pathogen_id, a precomputed per-tick infectiousness::Int8, and an active flag. The matching ImmunityState mirrors this for immunity records, tagged with a source flag (IMMUNITY_SOURCE_NATURAL / IMMUNITY_SOURCE_VACCINE).
• Cache + Overflow Storage: Each Individual holds a small inline infection_cache::NTuple{INFECTIONS_CACHE_SIZE, InfectionState} (and a parallel immunity_cache) for its hot infection, plus an infection_head::Int32 / immunity_head::Int32 pointer into a registry-backed linked list that holds any additional concurrent infections. This keeps the common single-infection case allocation-free and cache-local while supporting unbounded concurrency through overflow.
• Pathogen Bitmasks: Replaced the scalar infected::Bool semantics with primitive active_pathogens_mask::UInt32 and detected_mask::UInt32 fields on the Individual struct. Bit id - 1 flags active infection / detection for a given pathogen, giving $O(1)$ "is this agent infected with pathogen X" queries without touching the registry. All mask mutations are routed through infected!(ind, pid, val) / detected!(ind, pid, val) setters to keep the invariant single-sourced.
• Single-Active-Infection Invariant: infect! and set_progression! now guard against concurrent re-infection with the same pathogen (@warn + skip), enforcing the ≤1 active infection per (individual, pathogen) rule.

2. Sharded Registries & Lock-Free Concurrency

• Three Registry Types: Introduced InfectionRegistry, ImmunityRegistry, and TestRegistry (with a per-pathogen TestState / get_test_state), holding the overflow records and test bookkeeping that no longer fit on the agent struct.
• Thread-Sharded Ownership: Each registry type is stored on the Simulation as a Vector sharded by _owner_shard(id) = mod(id - 1, maxthreadid()) + 1, so each agent deterministically belongs to exactly one shard. Element-wise reads/writes during the threaded update and spread phases are therefore contention-free.
• Deferred Structural Mutation: Linked-list insertions and removals (which mutate registry structure) are staged and applied in a per-shard threaded flush (flush_pending_infections!, flush_ended_infections!) between phases, keeping the parallel hot loops purely element-wise. Overflow unlinks are ordered before cache promotions to prevent an ended overflow record from being resurrected into a freed cache slot.
• Lazy Iterators: registry_iterators.jl provides type-constrained iterators (InfectionIterator/ImmunityIterator) that lazily resolve shards and walk an individual's cache + overflow chain, presenting all active infections through a single uniform interface (each_infection / each_immunity).

3. Composable Transmission System

Transmission was decomposed into a base rate × stackable modifiers architecture, cleanly separating TransmissionFunction (produces a base probability) from TransmissionModifier (adjusts it via a transmission_factor contract).

• CompositeTransmissionRate: Wraps a base TransmissionFunction together with a statically-typed tuple of TransmissionModifiers, multiplying in each modifier's transmission_factor without re-multiplying base rates.
• SinusoidalSeasonalTransmissionRate / SinusoidalSeasonalModifier: Adds seasonal forcing of the transmission rate.
• CrossImmunityTransmissionRate / CrossImmunityModifier: Models cross-protection, where prior exposure/immunity to one pathogen reduces susceptibility to another. Pathogens are referenced by pathogen_names (resolved to ids at construction) to avoid id-ordering bugs.
• ViralInterferenceTransmissionRate / ViralInterferenceModifier: Models interference, where an active infection with one pathogen suppresses transmission of another. Reads interfering state from the registry (not same-tick staged state), so it is order-independent of intra-tick spread.
• AgeDependentTransmissionRate / ConstantTransmissionRate: Retained and adapted to the new interface, with concrete typing and allocation-free evaluation.

4. Customizable Infectiousness & Immunity Profiles

• InfectiousnessProfile: A new abstraction (calculate_infectiousness) computing an individual's shedding level over the course of an infection from its InfectionState snapshot, cached per-tick so the spread phase reads it directly off the agent. Two concrete profiles ship: ConstantInfectiousness (a fixed level across the whole infectious window) and StagedInfectiousness (a per-stage mapping — asymptomatic / presymptomatic / symptomatic / severe / critical).
• ImmunityProfile: A new abstraction (calculate_immunity) defining how immunity is acquired and decays — FullImmunity, NoImmunity, and the waning models ExponentialWaning / SigmoidalWaning (now with a configurable vaccine_buildup_duration). An immunity_is_stable hook lets the engine clear needs_immunity_update once an individual's immunity has settled, skipping per-tick recomputation while immunity is constant.
• Pathogen as Composition Root: The Pathogen struct composes progressions, a progression-assignment function, a transmission function, an infectiousness profile, and an immunity profile — making each axis independently swappable per pathogen.

5. Pathogen Parameterization & Type-Stable Dispatch

To keep the multi-pathogen hot loops allocation- and dispatch-free, the pathogen set is encoded in the type system rather than queried dynamically.

• Fully Parameterized Pathogen: Pathogen{PRG, PA, TF, IP, IM} carries the concrete types of its progressions tuple, progression-assignment function, transmission function, and infectiousness/immunity profiles, so every dispatch on a pathogen resolves to a concrete method.
• Simulation{P<:Tuple}: Pathogens are stored as a statically-typed heterogeneous tuple, making the entire pathogen configuration part of the Simulation's type.
• @generated Lookup: get_pathogen(sim, pid) is a @generated function that unrolls the tuple search at compile time, recovering the concrete Pathogen{…} type even when indexed by a runtime pid::Int8. The spread and progression loops can therefore specialize on each pathogen's concrete transmission/infectiousness/immunity types instead of paying dynamic-dispatch overhead per contact.

6. Multi-Pathogen Seeding, Configuration & Start Conditions

• MultiStartCondition: A composite start condition that applies a vector of per-pathogen StartConditions, enabling each pathogen to be seeded independently (e.g. different InfectedFraction / PatientZero / RegionalSeeds per disease). determine_start_condition constructs one automatically when multiple pathogens are supplied.
• TOML Configuration: The config schema now defines named [Pathogens.<Name>] tables, each with its own transmission_function, infectiousness_profile, and immunity_profile blocks, and start conditions can target a specific pathogen by name.
• Id Finalization & Validation: _finalize_pathogen_ids! assigns and validates contiguous pathogen ids (1..N), bounded by MAX_PATHOGENS, regardless of whether pathogens arrive via config or as a user-supplied tuple.

7. Pathogen-Aware Analysis, Reporting & Logging

• Pathogen-Tagged Loggers: Event loggers (infection, test, vaccination, …) now carry a pathogen_id, and the logging layer was reorganized into logging/{logger, event_loggers, infection_logger, tick_loggers}.jl.
• Multipathogen Post-Processing: The full pp_* post-processing suite (incidence, attack rates, effective R, R0, serial/generation intervals, tick cases/deaths/tests, detection) was made pathogen-aware, splitting and grouping results per pathogen.
• Per-Pathogen Aggregation: Added aggregate_by_pathogen and updated ResultData / Batch aggregation (and constructors, now that Simulation is parameterized) to render and store per-pathogen results across runs.

8. Repository Restructure

A large structural cleanup with no behavioral intent, reorganizing the source tree around its domains:

• structs/ + methods/ were dissolved into domain folders: pathogen/ (pathogens, progressions, transmission functions, profiles, vaccines), population/ (individuals + methods, populations, age groups), settings/ (settings, containers, ags), registries/, simulation/ (simulation, infections, calibration, batch, initialization, termination), contacts/, analysis/ (formerly model_analysis/), and logging/ (formerly logger/).
• agents.jl was split into population/individuals.jl (struct) and population/individual_methods.jl (behavior). Setting membership/lookup moved onto the individual.
• Tightened the public API surface: removed exports of non-public helpers and added _ prefixes to internal symbols (e.g. _basefolder).

Unit-Testing

• Added a dedicated test/registriestest.jl covering cache/overflow promotion, sharded ownership, deferred flush ordering, and iterator traversal.
• Substantially expanded test/pathogentest.jl for the new transmission functions and modifiers (composite, seasonal, cross-immunity, viral interference), both infectiousness profiles, immunity/waning profiles, and the single-active-infection guard.
• Added test/settingstest.jl and updated agentstest, infectionstest, interventionstest, simulationtest, and utilstest for the per-(individual, pathogen) model, bitmask getters/setters, and MultiStartCondition seeding.
• Updated logger, postprocessor, reporting, and resultdata tests to handle pathogen-tagged, per-pathogen output.