Skip to content

feat(summary): detect binned quality scores + run-level summary-merge#65

Merged
cjfields merged 1 commit into
mainfrom
feat/binned-quality-detection
Jul 7, 2026
Merged

feat(summary): detect binned quality scores + run-level summary-merge#65
cjfields merged 1 commit into
mainfrom
feat/binned-quality-detection

Conversation

@cjfields

@cjfields cjfields commented Jul 7, 2026

Copy link
Copy Markdown
Member

Summary

Adds tooling to quickly determine whether a dataset's quality scores are binned (NovaSeq/NextSeq/MiSeq i100 collapse Phred to a handful of levels) and to what levels — the starting point for analyzing binned-quality datasets.

summary: per-sample binning detection

  • New always-on quality_metrics block in the JSON: distinct Q values + per-value base counts, min/max Q, is_binned, and binned_scores. Derived from the per-position histogram already computed — no extra pass over the reads.
  • New flags: --binned-threshold (default 8; a file with 2–N distinct Q levels is judged binned) and --report (human-readable metrics to stderr; stdout stays clean JSON for piping).

summary-merge: run-level union report

A single low-count sample can miss rare bins (the true MiSeq i100 scheme is {2,12,24,38}, but Q2 is so rare it's often absent). The bin set is a property of the run, not a sample.

  • Reads N summary JSONs and unions their observed Q values so rare bins are recovered at the run level.
  • Flags per-sample bins missing from the run union (missing_from_run) — spot odd samples.
  • --expected-bins 2,12,24,40 validates the run against a user-declared scheme: observed values outside the declared set are a violation (catches silent vendor scheme changes, e.g. NovaSeq v1.2 {2,12,24,40} → v1.3 {2,9,24,40}); declared-but-unobserved bins are informational.
  • Vendor schemes are never hardcoded — they change without notice.

A shared judge_binned() keeps the per-sample and run-level rule identical.

Validation

  • PacBio HiFi (data/dada2/samPB) → not binned, 90 distinct values.
  • MiSeq i100 (11 samples) → binned, {12,24,38}; --expected-bins 2,12,24,38 reports Q2 as declared-but-unobserved (CONSISTENT); a wrong scheme yields VIOLATION.
  • Synthetic A {12,24,38} + B {2,12,24,38} → run recovers {2,12,24,38}, A flagged missing Q2.
  • New unit tests for judge_binned, binned/continuous detection, and count sums. Full suite (70 tests) passes.

🤖 Generated with Claude Code

Add per-sample binning detection to `summary` and a `summary-merge`
subcommand that unions per-sample quality-value distributions into a
run-level report.

`summary` gains an always-on `quality_metrics` block (distinct Q values +
counts, min/max, is_binned, binned_scores), derived from the existing
per-position histogram with no extra pass. New flags: --binned-threshold
(default 8) and --report (human-readable metrics to stderr; stdout stays
clean JSON).

`summary-merge` reads N `summary` JSONs and unions their observed Q values
so rare bins missed by any single low-count sample are recovered at the
run level. It flags per-sample bins missing from the run union, and
--expected-bins validates the run against a user-declared scheme: observed
values outside the declared set are a violation (catches silent vendor
scheme changes, e.g. NovaSeq v1.2 {2,12,24,40} -> v1.3 {2,9,24,40});
declared-but-unobserved bins are reported as informational. Vendor schemes
are never hardcoded.

A shared judge_binned() keeps the per-sample and run-level rule identical.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@cjfields cjfields merged commit 69cc991 into main Jul 7, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant