Skip to content

Exclude specific sensors/channels from all analysis (bad-data drop) #56

Description

@matt-pharr

Summary

We need a way to exclude specific sensors/channels from all analysis. If a single
probe misbehaves on a given shot — dead channel, railed/clipped signal, wrong gain,
disconnected integrator — its samples currently flow straight into both core analyses
and silently corrupt the fits. There is no way to say "ignore MPID66M on shot X"
short of editing code.

A bad sensor is not a rare edge case for real DIII-D data; the operator needs a first-class
"drop this channel" control, and it must apply everywhere — not per-view.

Why this bites both analyses

A single bad channel is not isolated to the plot it appears on:

  • Rotating (MODESPEC): every toroidal-array node stacks all present channels via
    nodes._array_channels(shot, families)_stack. A railed channel biases the SVD /
    phase-vs-φ fit and the n-spectrum for the whole array. There is no exclusion step between
    h5source.channel_names(shot) and the stack.
  • Quasi-stationary (SLCONTOUR): the fit selects channels by channel_filter (a regex or
    a friendly subset name, _slcontour/io_data.valid_channels). valid_channels already drops
    all-NaN channels, but a channel that carries plausible-looking but wrong signal passes
    the filter and skews K / χ² / the mode amplitudes. There's no per-shot manual exclude.

So "exclude sensor" has to live below both paths, at channel selection, not in the GUI
per-view.

Proposed shape (for discussion — this is a contract change, not a hack)

The natural seam is a single exclusion set threaded through channel selection:

  1. Data/selection layer: an excluded: set[str] (channel names) consulted in
    nodes._array_channels (rotating) and passed into the QS channel_filter resolution
    (subtract excluded names from the resolved regex/list). One source of truth so both
    analyses honor the same exclusions.
  2. Seam / params: carry the exclusion list as a /api/node query param (e.g.
    exclude=MPID66M,MPID132M) so it rides the existing params dict
    (app.pynodes.build_node) — and add it to the kind-node contract on both sides
    (core/contracts.pygui/web/src/lib/contract.ts) rather than faking it client-side.
  3. Scope decision (needs a call):
    • Per-shot (bad channel on one shot) vs persistent (a probe known-dead for a date
      range) — persistent could live shot-indexed alongside geometry in
      data/device/diiid.json, like the availability segments.
    • Whether exclusions are session-only (GUI state) or saved.
  4. GUI: a channel toggle in the Sensors view and/or a small "excluded sensors" control —
    clicking a bad sensor dot to drop it, with the analyses re-fitting live. Keep the user
    informed (which sensors are active vs excluded, and the resulting channel count).

Open questions

  • Per-shot vs persistent/date-ranged exclusions (or both)?
  • Should an excluded channel still render in the Sensors map (greyed out) so the operator
    can see what was dropped, while being pulled from every fit?
  • Auto-flagging: do we want a heuristic that suggests bad channels (railed / flatlined /
    σ-outlier), separate from the manual exclude? (Could be a follow-up.)

Touch points

  • src/magnetics/service/nodes.py_array_channels, _stack (rotating channel selection)
  • src/magnetics/core/qs_bridge.py + src/magnetics/_slcontour/io_data.py::valid_channels
    (QS channel selection)
  • src/magnetics/service/app.py — query-param passthrough
  • src/magnetics/core/contracts.pygui/web/src/lib/contract.ts — contract
  • gui/web/.../SensorsTab — the exclude UI

Cross-team (Data Streamers own selection, Interfacers own the seam/GUI, both analysis teams
consume it) — worth a quick Slack/PR coordination before implementing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions