Summary
The new-era experiment: / data: surface (ADR-0028, #423) makes the simulation output at exactly the data's independent-variable points via BNGL sample_times (BNG2.pl + bngsim) / RoadRunner simulate(times=…). This works for the common methods on every backend — except NFsim and RuleMonkey on the bngsim backend, where sample_times is dropped and the simulation falls back to a uniform grid. This issue tracks re-enabling those two methods on the new-era surface once upstream bngsim honors explicit output times.
Verified compatibility matrix (bngsim 0.9.40, BNG2.pl 2.9.3, RoadRunner)
| method |
BNG2.pl (bngl_backend = bionetgen/auto) |
bngsim (bngl_backend = bngsim) |
RoadRunner (SBML) |
| ode |
✓ sample_times |
✓ sample_times |
✓ simulate(times=) (cvode) |
| ssa |
✓ |
✓ |
✓ (gillespie) |
| psa |
✓ |
✓ (verified) |
n/a |
| pla |
✓ |
n/a (bngsim doesn't support pla — out of scope) |
n/a |
| nf (NFsim) |
✓ (simulate_nf) |
✗ drops sample_times → uniform grid |
n/a |
| RuleMonkey |
✓ |
✗ drops sample_times → uniform grid |
n/a |
Verified directly: NFsim and RuleMonkey under bngsim ignore sample_times=[0,2,5,9] (warn "sample_times is not supported for bngsim network-free simulation; ignoring") and emit a 0..100 / 101-point grid instead.
Impact
A new-era experiment with method: nf (or RuleMonkey) under bngl_backend = bngsim will not output at the data's points, so the objective's by-independent-variable match fails (or, if the data times happen to coincide with the fallback grid, mis-scores). The same job under BNG2.pl works correctly. ODE/SSA/PSA — the overwhelming common case — are fully compatible everywhere.
The fix is two-sided
The drop lives in PyBNF's bngsim bridge (pybnf/bngsim_model/nf_model.py:176,360), anticipating that bngsim's network-free simulator API can't yet emit at arbitrary explicit times.
- Upstream (bngsim): add explicit-output-time support to the vendored NFsim and RuleMonkey backends (tracked upstream).
- PyBNF (this repo), once available: stop dropping
sample_times in the NF bridge and pass them through; re-verify NFsim/RuleMonkey new-era experiments output at the data's points; add recovery/e2e coverage.
Interim options (decide separately)
- Guard: make
_load_experiments reject a network-free method (nf / RuleMonkey) under bngl_backend = bngsim with a clear "use a BNG2.pl backend for NFsim/RuleMonkey with the new experiment: surface" error, instead of the current silent fallback to a 0..100 grid.
- Bridge-side workaround: none clean — NFsim/RuleMonkey are stochastic, so selecting rows from a fine grid isn't equivalent to native explicit-time output. Native upstream support is the right fix.
Don't lose track
This is the "remember to update PyBNF when bngsim is fixed" tracker. Closing it means: bngsim honors explicit output times for NFsim + RuleMonkey, the bridge passes sample_times through, and the new-era surface is verified for both on the bngsim backend.
Refs: #423 (ADR-0028, the new-era surface), #426 (parameter_scan endpoint-time, a sibling deferral). Shipped in ADR-0028 Chunk 3.
Summary
The new-era
experiment:/data:surface (ADR-0028, #423) makes the simulation output at exactly the data's independent-variable points via BNGLsample_times(BNG2.pl + bngsim) / RoadRunnersimulate(times=…). This works for the common methods on every backend — except NFsim and RuleMonkey on the bngsim backend, wheresample_timesis dropped and the simulation falls back to a uniform grid. This issue tracks re-enabling those two methods on the new-era surface once upstream bngsim honors explicit output times.Verified compatibility matrix (bngsim 0.9.40, BNG2.pl 2.9.3, RoadRunner)
bngl_backend = bionetgen/auto)bngl_backend = bngsim)sample_timessample_timessimulate(times=)(cvode)pla— out of scope)simulate_nf)sample_times→ uniform gridsample_times→ uniform gridVerified directly: NFsim and RuleMonkey under bngsim ignore
sample_times=[0,2,5,9](warn "sample_times is not supported for bngsim network-free simulation; ignoring") and emit a 0..100 / 101-point grid instead.Impact
A new-era experiment with
method: nf(or RuleMonkey) underbngl_backend = bngsimwill not output at the data's points, so the objective's by-independent-variable match fails (or, if the data times happen to coincide with the fallback grid, mis-scores). The same job under BNG2.pl works correctly. ODE/SSA/PSA — the overwhelming common case — are fully compatible everywhere.The fix is two-sided
The drop lives in PyBNF's bngsim bridge (
pybnf/bngsim_model/nf_model.py:176,360), anticipating that bngsim's network-free simulator API can't yet emit at arbitrary explicit times.sample_timesin the NF bridge and pass them through; re-verify NFsim/RuleMonkey new-era experiments output at the data's points; add recovery/e2e coverage.Interim options (decide separately)
_load_experimentsreject a network-free method (nf/ RuleMonkey) underbngl_backend = bngsimwith a clear "use a BNG2.pl backend for NFsim/RuleMonkey with the new experiment: surface" error, instead of the current silent fallback to a 0..100 grid.Don't lose track
This is the "remember to update PyBNF when bngsim is fixed" tracker. Closing it means: bngsim honors explicit output times for NFsim + RuleMonkey, the bridge passes
sample_timesthrough, and the new-era surface is verified for both on the bngsim backend.Refs: #423 (ADR-0028, the new-era surface), #426 (parameter_scan endpoint-time, a sibling deferral). Shipped in ADR-0028 Chunk 3.