Skip to content

Flow verification module: automated reversed-section detection #198

Description

@jameshgrn

Summary

Built an automated flow verification system that detects and validates reversed flow directions at the section level (junction-to-junction segments). The module is functional but yields are low — parking for now to refocus on higher-impact POM fixes.

What exists

  • src/sword_v17c_pipeline/flow_verification.py — full module with:
    • Multi-signal scoring: DEM WSE slope, SWOT WSE slope (Theil-Sen at node level), facc monotonicity, junction slope validation
    • Confidence tiers: HIGH / MEDIUM / LOW / SKIP
    • Quality filters: ghost junctions (type=5/6), all-lake sections, single-reach sections, all-ghost sections
    • Pre-flip verification: DAG check, connectivity preservation, outlet reachability, facc improvement
    • Batch DuckDB writes with provenance + rollback support
  • scripts/topology/fix_reversed_sections.py — CLI with --dry-run, --apply, --rollback
  • tests/sword_duckdb/test_flow_verification.py — 30 passing tests
  • Figures in figures/flow_verification_NA/ and figures/flow_verification_all/

Results (dry-run, all regions)

Region Approved Rejected Low Skip
NA 3 15 71 731
SA 2 8 204 171
EU 4 2 62 127
AF 1 3 47 108
AS 9 13 331 369
OC 1 1 18 160
Total 20 42 733 1,666

Why parking

  1. Low yield: 20 approved flips out of 2,032 invalid sections (1%)
  2. Small global impact: fixes 46 of 4,814 WSE inversions (~1%), essentially zero facc improvement
  3. DEM signal dominance: MERIT Hydro WSE always has data, SWOT is sparse. DEM controls the vote but is lower quality than SWOT on flat rivers
  4. Diminishing returns: Most remaining invalid sections are ambiguous — flat rivers where noise dominates signal

To pick back up

  1. Drop DEM as a signal — SWOT-only scoring would be higher confidence per flip but even fewer approvals
  2. Tighten slopes_plus_supporting — AS section 18081 was approved despite SWOT actively contradicting DEM (DEM=+0.39, SWOT=-0.40). Require no signal actively contradicts.
  3. Or ship the ~17 clean ones — drop 3 sketchy sections (AS 18081 etc.), apply the rest, move on

Related

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions