Reliability-Based Genetic-Algorithm Baseline for Bridge Life-Cycle Optimization

Reliability-based optimization of element-level bridge maintenance policies with a genetic algorithm (GA). Four reliability-index (β) thresholds are treated as decision variables and optimized with PyGAD to minimize the expected discounted life-cycle cost (LCC) of a deteriorating steel-girder element (National Bridge Element 107).

This repository is the reliability-based GA benchmark from the paper:

Interpretable Deep Reinforcement Learning for Element-level Bridge Life-cycle Optimization Seyyed Amirhossein Moayyedi and David Y. Yang, Portland State University. arXiv:2604.02528  ·  https://arxiv.org/abs/2604.02528

In the paper, the GA policy is compared against a dynamic-programming (DP) condition-based policy and the paper's main contribution — an interpretable soft-tree / oblique-decision-tree reinforcement-learning actor. All methods are evaluated on the same Gymnasium/TorchRL environment, so their life-cycle costs are directly comparable.

---

Method in brief

Each year, the element's condition is an array of condition-state (CS) proportions s = [s₁, s₂, s₃, s₄]. This is mapped to a single reliability index

β(s) = Φ⁻¹(1 − pf(s)),   pf(s) = pf_arrayᵀ · s

and a maintenance action is chosen by comparing β(s) against four optimized thresholds β₁ ≥ β₂ ≥ β₃ ≥ β₄ (five actions: do-nothing, maintenance, repair, rehabilitation, replacement):

action = 0  (do nothing)       if  β(s) >  β₁
         1  (maintenance)       if  β₂ ≤ β(s) < β₁
         2  (repair)            if  β₃ ≤ β(s) < β₂
         3  (rehabilitation)    if  β₄ ≤ β(s) < β₃
         4  (replacement)       if  β(s) < β₄

The GA searches for the thresholds that minimize the expected discounted LCC over a finite horizon, propagating the CS distribution through action-dependent Markov transition matrices and accumulating direct maintenance cost plus discounted failure risk.

Results

Optimized β-thresholds reproduced by this repository (pygad_beta_dp_report.json):

Threshold	β₁	β₂	β₃	β₄
Value	4.200	3.558	3.367	3.191

Because β₁ ≈ 4.200 sits at the upper bound of attainable reliability, the do-nothing option is effectively never selected — matching Eq. (18) of the paper.

Life-cycle cost comparison (paper, Table 8; 1,000 validation episodes):

Policy	Mean LCC	Std
Oblique decision tree (RL)	1590.86	740.31
Oblique decision tree (RL + ad hoc rule)	1560.96	672.09
Condition-based policy (DP)	2133.42	1178.30
Reliability-based policy (GA — this repo)	1758.91	918.04

Reproducing this repository yields a GA mean LCC of ≈ 1782 (over the fixed initial-state panel) / ≈ 1774 (1,000 evaluation episodes), within ~1% of the published 1758.91; the small difference comes from library and RNG versions. The optimized β-thresholds reproduce the paper exactly.

GA convergence: best expected discounted cost per generation.

Life-cycle cost of the optimized GA policy as a function of the element's initial reliability index, over 1,000 evaluation episodes.

Repository structure

.
├── pygad_reliability.py                # Main script: GA β-threshold optimization + evaluation + plots
├── test_constants.py                   # Central GA / evaluation hyperparameters
├── bridge_gym/
│   ├── debug_example_nbe107.py         # Standalone environment demo
│   └── example_nbe107/
│       ├── settings.py                 # NBE-107 deterioration matrices, unit costs, failure probs
│       ├── rl_env.py                   # SingleElement Gymnasium environment
│       └── cost_util.py                # Cost-normalization helpers
├── figures/                            # Result figures
├── initial_beta_vs_LCC_GA_policy.csv   # Evaluation data (initial β vs LCC)
├── pygad_beta_dp_report.json           # Best policy, thresholds, and sample trajectory
├── requirements.txt
└── LICENSE

Installation

Tested with Python 3.11.

python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate
pip install -r requirements.txt

Usage

python pygad_reliability.py

The script is organized into # %% cells, so it can also be run interactively in VS Code or Jupyter. It (1) optimizes the β-thresholds with PyGAD, (2) evaluates the resulting policy in the shared Gymnasium/TorchRL environment, and (3) writes:

• pygad_beta_dp_report.json — optimized thresholds and a representative trajectory,
• initial_beta_vs_LCC_GA_policy.csv — per-episode initial reliability vs. LCC.

Optimization settings (population size, generations, horizon, discount, initial-state distribution, etc.) are collected in test_constants.py. The full configuration in this repository (500 generations × 512 population, horizon 200, 1,000 evaluation episodes) is compute-intensive; reduce these values for a quick run.

Citation

If you use this code or its results, please cite the accompanying paper:

@article{moayyedi2026interpretable,
  title   = {Interpretable Deep Reinforcement Learning for Element-level Bridge Life-cycle Optimization},
  author  = {Moayyedi, Seyyed Amirhossein and Yang, David Y.},
  journal = {arXiv preprint arXiv:2604.02528},
  year    = {2026},
  url     = {https://arxiv.org/abs/2604.02528}
}

License

Released under the MIT License.

Acknowledgements

Developed at Portland State University. The NBE-107 deterioration model follows Thompson et al. (1998) and the AASHTO Bridge Element Inspection Guide Manual; the GA formulation follows Yang and Frangopol (2020), implemented with PyGAD.