feat(summary): detect binned quality scores + run-level summary-merge#65
Merged
Conversation
Add per-sample binning detection to `summary` and a `summary-merge`
subcommand that unions per-sample quality-value distributions into a
run-level report.
`summary` gains an always-on `quality_metrics` block (distinct Q values +
counts, min/max, is_binned, binned_scores), derived from the existing
per-position histogram with no extra pass. New flags: --binned-threshold
(default 8) and --report (human-readable metrics to stderr; stdout stays
clean JSON).
`summary-merge` reads N `summary` JSONs and unions their observed Q values
so rare bins missed by any single low-count sample are recovered at the
run level. It flags per-sample bins missing from the run union, and
--expected-bins validates the run against a user-declared scheme: observed
values outside the declared set are a violation (catches silent vendor
scheme changes, e.g. NovaSeq v1.2 {2,12,24,40} -> v1.3 {2,9,24,40});
declared-but-unobserved bins are reported as informational. Vendor schemes
are never hardcoded.
A shared judge_binned() keeps the per-sample and run-level rule identical.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds tooling to quickly determine whether a dataset's quality scores are binned (NovaSeq/NextSeq/MiSeq i100 collapse Phred to a handful of levels) and to what levels — the starting point for analyzing binned-quality datasets.
summary: per-sample binning detectionquality_metricsblock in the JSON: distinct Q values + per-value base counts, min/max Q,is_binned, andbinned_scores. Derived from the per-position histogram already computed — no extra pass over the reads.--binned-threshold(default 8; a file with 2–N distinct Q levels is judged binned) and--report(human-readable metrics to stderr; stdout stays clean JSON for piping).summary-merge: run-level union reportA single low-count sample can miss rare bins (the true MiSeq i100 scheme is
{2,12,24,38}, but Q2 is so rare it's often absent). The bin set is a property of the run, not a sample.summaryJSONs and unions their observed Q values so rare bins are recovered at the run level.missing_from_run) — spot odd samples.--expected-bins 2,12,24,40validates the run against a user-declared scheme: observed values outside the declared set are a violation (catches silent vendor scheme changes, e.g. NovaSeq v1.2{2,12,24,40}→ v1.3{2,9,24,40}); declared-but-unobserved bins are informational.A shared
judge_binned()keeps the per-sample and run-level rule identical.Validation
data/dada2/samPB) → not binned, 90 distinct values.{12,24,38};--expected-bins 2,12,24,38reports Q2 as declared-but-unobserved (CONSISTENT); a wrong scheme yields VIOLATION.{12,24,38}+ B{2,12,24,38}→ run recovers{2,12,24,38}, A flaggedmissing Q2.judge_binned, binned/continuous detection, and count sums. Full suite (70 tests) passes.🤖 Generated with Claude Code