Skip to content

Write continuous-raster symbology sidecars from to_geotiff (#3537)#3538

Merged
brendancol merged 5 commits into
mainfrom
issue-3537
Jun 26, 2026
Merged

Write continuous-raster symbology sidecars from to_geotiff (#3537)#3538
brendancol merged 5 commits into
mainfrom
issue-3537

Conversation

@brendancol

Copy link
Copy Markdown
Contributor

Closes #3537.

Adds an opt-in color_ramp parameter to to_geotiff. For a continuous single-band raster it writes two sidecars next to the file so QGIS opens it with a color ramp instead of a flat grayscale stretch:

  • <base>.qml — a QGIS singlebandpseudocolor style (extension-replacing name, which is what QGIS auto-loads).
  • STATISTICS_MINIMUM/MAXIMUM/MEAN/STDDEV in the PAM <file>.aux.xml, so GDAL tools and QGIS get a min/max stretch even if the .qml is ignored.

This is the continuous-data counterpart to the categorical RAT sidecar (#3482 / #3483). Categorical rasters keep the RAT sidecar; the two are mutually exclusive and never collide.

Usage

dem.xrs.to_geotiff('dem.tif', color_ramp='viridis')
# writes dem.tif, dem.qml, dem.tif.aux.xml

Ramps ship as hardcoded control points (viridis default, plus plasma, magma, inferno, cividis, greys, spectral, terrain), so there is no matplotlib runtime dependency. color_ramp_range=(min, max) sets the stretch bounds and skips the statistics pass for large dask graphs.

Backend coverage

Statistics come from a single backend-aware reduction pass; the dask path fuses the reductions into one dask.compute so the graph is read once. Covered across numpy, cupy, dask+numpy, dask+cupy.

Gating

  • Opt-in only (no color_ramp → no sidecars, behaviour unchanged).
  • Continuous only (skips a raster carrying category_names).
  • Single-band only; file-like destinations skipped; no-op when there is no finite data.
  • Constant raster (min == max): writes the stats sidecar, skips the degenerate .qml.
  • Unknown ramp name raises ValueError before any bytes are written.

Test plan

  • pytest xrspatial/geotiff/tests/write/test_symbology_sidecar_3537.py (19 passed, GPU + dask+cupy included locally)
  • Categorical regression: test_category_sidecar_backends_3483.py, test_rasterize_categorical_3482.py (21 passed)
  • flake8 clean on changed files
  • Manual: wrote a DEM, confirmed dem.tif / dem.qml / dem.tif.aux.xml; .qml parses with correct classificationMin/Max; stats exclude the NaN hole

Add an opt-in color_ramp parameter to to_geotiff. For a continuous
single-band raster it writes a QGIS .qml singlebandpseudocolor style
plus STATISTICS_* in the PAM .aux.xml, so the file opens in QGIS with a
color ramp instead of a flat grayscale stretch -- the continuous
counterpart to the existing categorical RAT sidecar.

- new _symbology.py: hardcoded ramps (viridis default), QML builder,
  cross-backend single-pass _finite_stats, sidecar orchestration
- _pam.py: build/write stats PAM sidecar helpers
- eager.py: color_ramp / color_ramp_range params, folded into the
  shared sidecar closure across all write paths (numpy, dask, GPU, VRT)
- covers numpy / cupy / dask+numpy / dask+cupy with parity tests

@brendancol brendancol left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review: Write continuous-raster symbology sidecars from to_geotiff (#3537)

Reviewed the four changed files in full on the PR branch. The feature is well-scoped and mirrors the categorical sidecar pattern cleanly.

Blockers (must fix before merge)

None.

Suggestions (should fix, not blocking)

  • xrspatial/geotiff/_writers/eager.py:481_sym_data = data captures the original DataArray, but _pack rewrites data later at line 544. When pack=True and color_ramp are combined, the statistics and the .qml classificationMin/Max are computed from the logical (pre-pack) values, while the TIFF stores the packed integers. QGIS renders against the raw stored pixels, so the ramp would be stretched over the wrong range. pack is experimental and the combination is unusual, but it produces a silently wrong-looking render. Worth either skipping symbology when pack=True (with a one-line note) or documenting the interaction in the color_ramp docstring.

Nits (optional improvements)

  • xrspatial/geotiff/_symbology.py — the greys ramp follows matplotlib's orientation (0 -> white, 1 -> black), so low elevations render light and high render dark. That is the matplotlib convention, but it is the opposite of what many DEM users expect. Not wrong, just worth a mention in the docstring if it is easy.
  • No end-to-end test exercises the .vrt write path with color_ramp (the closure is shared, so the unit + numpy/dask/gpu coverage already protects it, but a one-liner would close the gap).

What looks good

  • _finite_stats fuses the dask reductions into a single dask.compute, so the graph is read once rather than four times. Backend parity is verified by an actual test across numpy/dask/cupy/dask+cupy.
  • Gating is thorough and tested: categorical wins, multiband skipped, file-like ignored, unknown ramp raises before any bytes, constant raster writes stats but skips the degenerate QML.
  • The qml_path extension-replace vs PAM append distinction is called out with a comment explaining why a wrong name is silently ignored by QGIS — the exact trap to flag.
  • Ramps are static constants, no matplotlib runtime dependency.

Checklist

  • Algorithm/format matches QGIS singlebandpseudocolor + GDAL STATISTICS_*
  • All four backends produce consistent statistics
  • NaN handling correct (finite mask excludes NaN and the nodata sentinel)
  • Edge cases covered (all-NaN, constant, nodata exclusion, multiband, file-like)
  • Dask path reads the graph once, no premature materialization
  • Benchmark not needed (opt-in sidecar I/O)
  • README matrix not applicable (parameter on existing function)
  • Docstrings present for the new params and helpers

…3537)

- skip symbology when pack=True: the on-disk packed values would not
  match a ramp/stats built from the logical values
- docstring notes for the pack interaction and greys orientation
- tests for the pack-skip gate and the .vrt write path

@brendancol brendancol left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up review (after 7ceb4b3)

Re-reviewed against the follow-up commit. All findings from the first pass are resolved.

Disposition of prior findings

  • Suggestion — pack/color_ramp range mismatch — fixed. _writers/eager.py now gates symbology on not pack (with a comment), so a packed write never emits sidecars whose range disagrees with the stored pixels. Documented in the color_ramp docstring. Covered by test_pack_skips_symbology.
  • Nit — greys orientation — fixed. The color_ramp docstring now notes greys follows matplotlib's light-to-dark orientation.
  • Nit — no VRT end-to-end test — fixed. Added test_vrt_write_emits_sidecars, which writes a dask-backed array to a .vrt path and asserts both mosaic.qml and mosaic.vrt.aux.xml.

Verification

  • pytest xrspatial/geotiff/tests/write/test_symbology_sidecar_3537.py — 21 passed (numpy, dask, GPU, dask+cupy parity).
  • Categorical regression suites still green.
  • flake8 clean on all four changed files.

No remaining blockers, suggestions, or nits.

…3537)

The #1922 signature-parity test requires _write_geotiff_gpu to declare
the same kwargs as to_geotiff (minus gpu). Add color_ramp /
color_ramp_range as wrapper-only no-ops (del'd like pack): the symbology
sidecars are written by to_geotiff's sidecar closure, not the GPU writer.
@brendancol brendancol merged commit 647371e into main Jun 26, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

to_geotiff: write best-practice symbology sidecars for continuous rasters (QGIS color ramp)

1 participant