Public reproducibility package for:
MicroForge: Reliability Benchmarking of Foundation Models for Concrete SEM Damage Segmentation
This repository supports the JMCE/ASCE R1 revision. MicroForge is framed here as a diagnostic audit toolkit for concrete SEM segmentation, not as a validated fully automated measurement system.
- R1 GPU scripts for SAM-1, SAM 2.1, calibration, LoRA replication, U-Net 106-FOV
supervised checks, ASRC circularity sensitivity, and figure generation:
src/revision_gpu/ - Deterministic ASHG rule scaffold:
src/ashg_rule_pipeline.py - R1 per-image and per-mask records:
data/r1_revision_results/ - R1 summary tables:
data/r1_tables/ - Final R1 figure exports:
figures/jmce_r1/ - Legacy original-submission audit trail:
src/analysis/,data/source_truth/, anddata/verification_tables/
The legacy files are retained to make the correction history auditable. Use the
r1_* directories for the revised manuscript numbers.
| Item | Revised result | Source |
|---|---|---|
| SAM-1 AMG union IoU | 0.126 in the rerun; manuscript reports 0.129 at the original/revision precision | exp1_union_standard.json |
| SAM-1 precision / recall | 0.136 / 0.634 | exp1_union_standard.json |
| SAM-1 deployment-level ECE | 0.697 | revision_local_analyses.json |
| SAM 2.1 base-plus union IoU | 0.067 | exp5_sam21.json |
| SAM 2.1 large union IoU | 0.083 | exp5b_sam21_large.json |
| Corrected context-expert vs auto damage IoU | 0.463 | corrected_manual_auto_agreement.json |
| Assisted expert vs auto damage IoU | 0.894 | expert106_agreement.json |
| U-Net 106-FOV split, benchmark cross-protocol IoU | 0.698 | exp3_unet106.json |
| LoRA V0/V1 AMG union IoU | 0.116 / 0.103 | exp2_lora_variants.json |
See data/r1_tables/R1_key_metrics.csv and
supplementary/SINGLE_SOURCE_OF_TRUTH.md for the compact number map.
Raw third-party data and model checkpoints are not included.
- NIST Concrete Damage SEM data: https://isg.nist.gov/deepzoomweb/concreteScoring/index.html
- NIST raw/annotation files: https://isg.nist.gov/deepzoomweb/data/concreteScoring
- SAM-1 checkpoint:
sam_vit_h_4b8939.pthfrom Meta's Segment Anything repository - SAM 2.1 checkpoints:
sam2.1_hiera_base_plus.ptandsam2.1_hiera_large.pt - Raw ASRC SEM/XRD files are not released here; the repository contains derived ASRC case-study records and figure exports.
python -m venv .venv
. .venv/Scripts/activate # Windows PowerShell users can use .venv\Scripts\Activate.ps1
pip install -r requirements.txtSet local asset paths when rerunning GPU experiments:
set MICROFORGE_NIST_DIR=C:\path\to\nist_concrete_scoring
set MICROFORGE_SAM_CKPT=C:\path\to\sam_vit_h_4b8939.pth
set MICROFORGE_SAM21_BASE_PLUS_CKPT=C:\path\to\sam2.1_hiera_base_plus.pt
set MICROFORGE_SAM21_LARGE_CKPT=C:\path\to\sam2.1_hiera_large.pt
set MICROFORGE_ASRC_SEM_DIR=C:\path\to\asrc_semTo regenerate the deterministic ASHG output from released tables:
python src/ashg_rule_pipeline.pyFull reproduction notes are in REPRODUCIBLE_RUNBOOK.md.
The manuscript cites the versioned GitHub release v2.0-jmce-r1 and its commit
hash. If a repository DOI is minted later through an external archive, cite the
verified DOI in addition to the GitHub release.
Code is released under the MIT License. See LICENSE.