Add finite-replica confidence floor and within-source block reference for multi-replica edge confidence - #112
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
MDPath already reports a per-edge multi-replica confidence (
-confidence). Two thingswere missing to make that number interpretable:
Rnon-negative replica values the coefficient of variation is bounded, so
confidence = 1/(1+CV)cannot fall belowc_min = 1/(1+CV_max). Without that bounda "0.6" from 3 replicas and a "0.6" from 20 replicas look the same but aren't.
produces on its own, with no genuine replica-to-replica agreement behind it.
This PR adds both, plus the visualization and tests to go with them. Everything is
opt-in: without the new flags, behaviour is identical to current
master.What's new
Finite-replica floor (
mdpath/src/confidence.py)edge_confidence.csvgainsc_min(the attainable lower bound),c_norm(the scorerescaled from
[c_min, 1]onto[0, 1]) andddof.Within-source block reference (
mdpath/src/null_model.py, new)Each source trajectory is cut into contiguous, non-overlapping blocks of length
L;random groups of
Rblocks are resampled to estimate how reproducible an edge looksfrom within-trajectory sampling alone. Observed replicas are subsampled to the same
(L, R)so the comparison is matched. The headline output isi.e. does the between-replica dispersion exceed the within-source reference?
Null-aware visualization (
mdpath/src/path_confidence_viz.py)The existing ChimeraX viewer's hover gains the reference metrics; colouring, tube
widths, spline geometry and structure embedding are unchanged. An opt-in second viewer
(
-experimental-c-corr) colours by the experimental corrected score.Terminology (deliberate)
confidence/c_normare dispersion-based stability scores, not probabilities,significance levels, or effect sizes. An edge with consistently negligible MI can
score highly.
external "no-coupling" null — the blocks are real trajectory windows and still contain
genuine dynamics. It is named as such throughout.
c_corr_experimental(quadrature CV subtraction) is an unvalidated heuristic. It isopt-in, labelled experimental in the CLI/metadata/legend, and finite only when
corr_status == "above_reference"; undefined and at-or-below-reference edges stayNaNso they are never rendered as perfect confidence.
rhois the recommended output.New CLI flags
-null {none,block}none-confidence).-null-block-sweep25,50,100,250-null-block-ns100-null-sourcesall-null-draws100-null-seed0-ddof {0,1}0-null-outdirnull_analysis-experimental-c-corrc_corrviewer.mdpath -top system.pdb -traj rep1.dcd rep2.dcd rep3.dcd rep4.dcd \ -confidence -null block -null-block-sweep 25,50,100,250 -null-block-ns 100