diff --git a/.claude/sweep-test-coverage-state.csv b/.claude/sweep-test-coverage-state.csv index aa9b0d49e..8f62b40da 100644 --- a/.claude/sweep-test-coverage-state.csv +++ b/.claude/sweep-test-coverage-state.csv @@ -8,7 +8,7 @@ dasymetric,2026-06-20,3407;3406,HIGH,2;3;4;5,"deep-sweep test-coverage on a CUDA diffusion,2026-06-20,3422,HIGH,1;2;3;4,"Pass 1 (2026-06-20, deep-sweep test-coverage, CUDA host). diffuse() dispatch table registers all 4 backends but test_diffusion.py only exercised numpy + dask+numpy. Cat 1 HIGH: cupy (_diffuse_cupy/_diffuse_step_gpu) and dask+cupy (_diffuse_dask_cupy/_diffuse_chunk_cupy) registered but never invoked -- no test ran them. Cat 4 HIGH: boundary accepts nan/nearest/reflect/wrap; only nearest+wrap tested, reflect had none. Cat 3 HIGH: 1x1 single-pixel and Nx1/1xN strip rasters never tested. Cat 2 MEDIUM: NaN tested numpy-only; Inf and all-NaN inputs untested. Filed #3422, added 14 tests (PR #3424, test-only, source untouched): cupy/dask+cupy parity vs numpy (incl. spatially-varying alpha + NaN propagation), reflect boundary across all 4 backends, 1x1 + Nx1 + 1xN (numpy + chunked dask strip), all-NaN stays NaN, Inf contamination smoke test. All 14 RAN+PASSED on a CUDA host; the 4 cupy/dask+cupy tests genuinely executed (not skipped); full file 39 passed. All paths verified correct before the tests were added -- coverage gap, not a bug. LOW (documented, not fixed): non-square cellsize (res[0]!=res[1]) never exercised -- diffuse uses res[0] as dx and assumes square cells; empty 0-row/0-col raster untested; asv benchmark absent; 'nan' boundary-mode edge=NaN behaviour not directly asserted on diffuse (covered indirectly via wrap/nearest)." fire,2026-06-19,,HIGH,1;5,"Cat1 HIGH: dask+cupy dispatch registered but untested for rdnbr/burn_severity_class/fireline_intensity/flame_length/rate_of_spread/kbdi (only dnbr had it). Cat5 MEDIUM: only dnbr asserted attrs/coords/dims preservation. Added 6 test_numpy_equals_dask_cupy + 6 test_output_preserves_metadata; all 66 fire tests pass on a CUDA host. Cat2 LOW (not fixed): Inf inputs untested (pure per-cell math, low risk)." focal,2026-06-10,3220;3219;3225,HIGH,1;2;3;4,"Deep-sweep 2026-06-10 on CUDA host, all 4 backends executed. Filed #3220 (coverage) and added 36 tests in PR branch: Inf inputs for mean/focal_stats (HIGH Cat2 - no Inf test existed anywhere), mean NaN input (HIGH Cat2 - default excludes=[nan] semantics never asserted), 1x1 + 1xN/Nx1 strips (HIGH Cat3), empty 0-row raster numpy-only (MEDIUM Cat3), mean passes=2 == mean(mean) and excludes sentinel -9999 behavioral tests (MEDIUM Cat4), dask+cupy non-default boundary modes for mean/apply/focal_stats (MEDIUM Cat1/4). Bugs surfaced, filed separately (NOT fixed here): #3219 hotspots silently returns all zeros on Inf input (nan global std passes the std==0 guard, all 4 backends); #3225 empty raster works on numpy but crashes cupy (raw CudaAPIError) and dask (map_overlap depth ValueError). hotspots+Inf and non-numpy empty behavior left unpinned until those are fixed. Backend matrix for the 4 public funcs was already solid (all 4 backends + parity); boundary modes covered except dask+cupy. Siblings filed #3214-3217 same day (dtype/docstring/apply-default-func) - no overlap." -geotiff,2026-06-12,3266,MEDIUM,1,"Pass 22 (2026-06-12, deep-sweep test-coverage): delta audit of the ~20 commits since pass 21 (06-09..06-12, mostly pack/unpack fixes + #3241 GPU streaming writer + coregister #3254/#3248). Filed #3266 (tests). Cat 1 MEDIUM: pack=True gained working gpu/dask+gpu support in #3240, but three pack features were tested numpy+dask only: float32 width preservation (#3080, test_pack_float_width_3080.py), nodata kwarg fill (#3168, test_pack_nodata_kwarg_3168.py), band-subset per-band SCALE/OFFSET rewrite (#3161, test_pack_band_subset_3161.py). Live probe on this CUDA host: all six gpu/dask+gpu legs pass today (no source bug, pure coverage gap). Added one gpu/dask-gpu parametrized round-trip test per file (6 tests, requires_gpu, RUN+passing locally) and fixed two stale docstrings claiming unpack/pack is CPU-only (wrong since #3075/#3240). Verified NOT gaps this pass: #3128 int64 sentinel tests cover eager+dask+gpu; #3241 streaming writer landed with byte-identical band-first/band-last/BytesIO/small-buffer tests; #3104 scale-zero rejection has gpu legs; #3169 revived the dead compression-corpus oracle gate. Out of scope: coregister=True lives in accessor.py (excluded module); its multi-band + polar gaps are documented as experimental caveats in docs/source/reference/geotiff.rst (#3248). LOW (carried, documented not fixed): Inf as the declared nodata sentinel never tested. || PREVIOUS: Pass 21 (2026-06-09, deep-sweep test-coverage): filed #3114 (tests) + #3112 (source bug). Cat 1 HIGH: to_geotiff(pack=True) round-trip was tested only on numpy and dask+numpy (write/test_pack_3064.py); #3075 made unpack=True work on gpu and dask+gpu reads, but no test packed a GPU-read array back. Live probe on this CUDA host: BOTH GPU legs crash today -- eager gpu raises AttributeError (cupy has no astype, the known cupy 13.6/xarray 2025.12 where/astype incompat) and dask+gpu raises TypeError (numpy fill value inside cupy.where) -- both from _pack's out.fillna(nodata) in _attrs.py; _writers/gpu.py says the pre-dispatch re-pack is supposed to make every write path work. Source bug filed as #3112; test-only PR adds test_pack_round_trip_gpu (gpu + dask-gpu params, requires_gpu, xfail(strict=True) on #3112 so the fix flips them loudly) and fixes the stale module docstring claiming GPU rejects mask_and_scale. Ran on CUDA host: 13 passed, 2 xfailed. Verified NOT gaps this pass (probed before flagging): empty/zero-band writer guard is covered (test_basic.py 2075/2095 blocks incl. gpu + dask + streaming entry points); degenerate shapes covered on all 4 read backends (read/test_degenerate_shapes.py); overview_resampling all 7 modes parametrized; missing_sources raise/warn + invalid, band_nodata first/invalid, unpack on all 4 backends, masked/parse_coordinates/lock/cache/default_name/name-deprecation all exercised; attrs contract per-backend (attrs/test_contract.py). LOW (documented, not fixed): Inf as the declared nodata sentinel is never tested (only one nan+inf data round-trip in test_edge_cases.py). || PREVIOUS: Pass 20 (2026-06-06, deep-sweep test-coverage): filed #2984 and added test_writer.py degenerate-shape GPU write coverage (Cat 1 backend + Cat 3 geometric edge). Read side already covers 1x1/1xN/Nx1 on all 4 backends (read/test_degenerate_shapes.py) and the dask streaming writer covers them (integration/test_dask_pipeline.py); the GPU write path was the gap (smallest shape in gpu/test_writer.py was 2x2). Added test_write_geotiff_gpu_degenerate_round_trip (1x1/1xN/Nx1 x none/deflate) + test_to_geotiff_dask_gpu_degenerate_round_trip (dask+cupy via gpu=True). 9 new tests RUN+passing on a CUDA host. Verified paths work first (not a source bug); transform supplied explicitly via attrs. Wider tree audit (~92k test LOC vs ~33k source): rioxarray-compat (#2961), bbox NaN/Inf/rotated, 8-backend parity matrix, codec round-trips already covered -- no other real gaps. | Pass (2026-06-05 test-coverage sweep): mature module (~31k src / ~124k test LOC, 9 test dirs). Exhaustive existing coverage -- parity/test_backend_matrix.py runs all 4 backends + VRT + HTTP + fsspec; golden_corpus full-manifest parity; read_rioxarray_compat_2961 covers masked/mask_and_scale/parse_coordinates/default_name on eager+dask. Cat1+Cat3 gap found (MEDIUM): degenerate-shape READS (1x1/1xN/Nx1) were tested only on the eager numpy reader (test_edge_cases.py) and the dask streaming WRITE path (integration/test_dask_pipeline.py); the windowed dask READ (chunks=) and GPU READ (gpu=True) on a single-pixel dimension were never exercised (smallest dask-read source in read/test_tiling is 8x8/2x32, parity fixtures 32x32/64x64). Probed: paths work today, no source bug -- pure coverage gap. Added read/test_degenerate_shapes.py (18 tests): dask read x{chunks 1,3,4} x{1x1,1xN,Nx1} + coord/transform/crs parity + GPU read + dask+gpu read. GPU cells RAN and PASSED on this CUDA host (grid-size-1 launch validated). Fixture supplies explicit attrs['transform'] (writer cannot infer pixel size from a 1-element coord axis). Branch deep-sweep-test-coverage-geotiff-degenerate-read-01. NOTE: pre-existing union-merge CRLF/duplicate-record corruption in this CSV left untouched -- appended one clean record; DictReader last-write-wins picks this one." +geotiff,2026-06-25,3518,MEDIUM,1,"Pass 23 (2026-06-25, deep-sweep test-coverage, CUDA host): delta audit of the ~15 geotiff commits since pass 22 (06-12..06-24): #3483 categorical PAM sidecar, #3375/#3376/#3380 xarray backend engine, #3373/#3374 chunked GPU read-once, #3371/#3372 reject predictor+lossy codec, #3331/#3332 reject zero/non-finite ModelPixelScale, #3327 gate dict gdal_metadata behind rich-tag opt-in, #3323/#3325 masked_nodata dtype-cast, #3277 pack nodata native width. Filed #3518 (tests). Cat 1 MEDIUM: the #3483 categorical PAM sidecar (_pam.py + _write_category_sidecar in _writers/eager.py) is round-trip tested only on the eager numpy write path (xrspatial/tests/test_rasterize_categorical_3482.py); the dask streaming (eager.py:1063) and GPU/nvCOMP (eager.py:880) write branches each have their own sidecar emit call that no test touches. Live probe on this CUDA host: dask + GPU categorical round-trips both emit the sidecar and read back names+RGBA colors today (no source bug, pure coverage gap). Added geotiff/tests/write/test_category_sidecar_backends_3483.py: dask write round-trip + GPU write round-trip (requires_gpu, RAN+passing locally) + names-only (category_colors=None build branch + names-only read path, never hit by the existing colors-always suite). 3 tests pass. Verified NOT gaps this pass: #3327 dict gdal_metadata gate has 43-line test_contract.py coverage; #3331/#3332 zero pixel scale covered by unit/test_degenerate_pixel_size_3331.py; #3371/#3372 predictor+lossy-codec reject has 2 test files; #3375/#3376/#3380 xarray engine covered by test_xarray_backend_3365.py + coregister 3376/3379; #3277 pack native width has 3 test files. LOW (carried, documented not fixed): Inf as the declared nodata sentinel still never tested. || Pass 22 (2026-06-12, deep-sweep test-coverage): delta audit of the ~20 commits since pass 21 (06-09..06-12, mostly pack/unpack fixes + #3241 GPU streaming writer + coregister #3254/#3248). Filed #3266 (tests). Cat 1 MEDIUM: pack=True gained working gpu/dask+gpu support in #3240, but three pack features were tested numpy+dask only: float32 width preservation (#3080, test_pack_float_width_3080.py), nodata kwarg fill (#3168, test_pack_nodata_kwarg_3168.py), band-subset per-band SCALE/OFFSET rewrite (#3161, test_pack_band_subset_3161.py). Live probe on this CUDA host: all six gpu/dask+gpu legs pass today (no source bug, pure coverage gap). Added one gpu/dask-gpu parametrized round-trip test per file (6 tests, requires_gpu, RUN+passing locally) and fixed two stale docstrings claiming unpack/pack is CPU-only (wrong since #3075/#3240). Verified NOT gaps this pass: #3128 int64 sentinel tests cover eager+dask+gpu; #3241 streaming writer landed with byte-identical band-first/band-last/BytesIO/small-buffer tests; #3104 scale-zero rejection has gpu legs; #3169 revived the dead compression-corpus oracle gate. Out of scope: coregister=True lives in accessor.py (excluded module); its multi-band + polar gaps are documented as experimental caveats in docs/source/reference/geotiff.rst (#3248). LOW (carried, documented not fixed): Inf as the declared nodata sentinel never tested. || PREVIOUS: Pass 21 (2026-06-09, deep-sweep test-coverage): filed #3114 (tests) + #3112 (source bug). Cat 1 HIGH: to_geotiff(pack=True) round-trip was tested only on numpy and dask+numpy (write/test_pack_3064.py); #3075 made unpack=True work on gpu and dask+gpu reads, but no test packed a GPU-read array back. Live probe on this CUDA host: BOTH GPU legs crash today -- eager gpu raises AttributeError (cupy has no astype, the known cupy 13.6/xarray 2025.12 where/astype incompat) and dask+gpu raises TypeError (numpy fill value inside cupy.where) -- both from _pack's out.fillna(nodata) in _attrs.py; _writers/gpu.py says the pre-dispatch re-pack is supposed to make every write path work. Source bug filed as #3112; test-only PR adds test_pack_round_trip_gpu (gpu + dask-gpu params, requires_gpu, xfail(strict=True) on #3112 so the fix flips them loudly) and fixes the stale module docstring claiming GPU rejects mask_and_scale. Ran on CUDA host: 13 passed, 2 xfailed. Verified NOT gaps this pass (probed before flagging): empty/zero-band writer guard is covered (test_basic.py 2075/2095 blocks incl. gpu + dask + streaming entry points); degenerate shapes covered on all 4 read backends (read/test_degenerate_shapes.py); overview_resampling all 7 modes parametrized; missing_sources raise/warn + invalid, band_nodata first/invalid, unpack on all 4 backends, masked/parse_coordinates/lock/cache/default_name/name-deprecation all exercised; attrs contract per-backend (attrs/test_contract.py). LOW (documented, not fixed): Inf as the declared nodata sentinel is never tested (only one nan+inf data round-trip in test_edge_cases.py). || PREVIOUS: Pass 20 (2026-06-06, deep-sweep test-coverage): filed #2984 and added test_writer.py degenerate-shape GPU write coverage (Cat 1 backend + Cat 3 geometric edge). Read side already covers 1x1/1xN/Nx1 on all 4 backends (read/test_degenerate_shapes.py) and the dask streaming writer covers them (integration/test_dask_pipeline.py); the GPU write path was the gap (smallest shape in gpu/test_writer.py was 2x2). Added test_write_geotiff_gpu_degenerate_round_trip (1x1/1xN/Nx1 x none/deflate) + test_to_geotiff_dask_gpu_degenerate_round_trip (dask+cupy via gpu=True). 9 new tests RUN+passing on a CUDA host. Verified paths work first (not a source bug); transform supplied explicitly via attrs. Wider tree audit (~92k test LOC vs ~33k source): rioxarray-compat (#2961), bbox NaN/Inf/rotated, 8-backend parity matrix, codec round-trips already covered -- no other real gaps. | Pass (2026-06-05 test-coverage sweep): mature module (~31k src / ~124k test LOC, 9 test dirs). Exhaustive existing coverage -- parity/test_backend_matrix.py runs all 4 backends + VRT + HTTP + fsspec; golden_corpus full-manifest parity; read_rioxarray_compat_2961 covers masked/mask_and_scale/parse_coordinates/default_name on eager+dask. Cat1+Cat3 gap found (MEDIUM): degenerate-shape READS (1x1/1xN/Nx1) were tested only on the eager numpy reader (test_edge_cases.py) and the dask streaming WRITE path (integration/test_dask_pipeline.py); the windowed dask READ (chunks=) and GPU READ (gpu=True) on a single-pixel dimension were never exercised (smallest dask-read source in read/test_tiling is 8x8/2x32, parity fixtures 32x32/64x64). Probed: paths work today, no source bug -- pure coverage gap. Added read/test_degenerate_shapes.py (18 tests): dask read x{chunks 1,3,4} x{1x1,1xN,Nx1} + coord/transform/crs parity + GPU read + dask+gpu read. GPU cells RAN and PASSED on this CUDA host (grid-size-1 launch validated). Fixture supplies explicit attrs['transform'] (writer cannot infer pixel size from a 1-element coord axis). Branch deep-sweep-test-coverage-geotiff-degenerate-read-01. NOTE: pre-existing union-merge CRLF/duplicate-record corruption in this CSV left untouched -- appended one clean record; DictReader last-write-wins picks this one." idw,2026-06-04,2919,HIGH,1;4,"cupy/dask+cupy backends untested (Cat1 HIGH); GPU k-reject error path untested (Cat4 MED). Added 6 GPU tests, validated on CUDA host. Inf-in-points (Cat2) and attrs-preservation (Cat5) are LOW, documented not fixed." interpolate,2026-06-12,3290,MEDIUM,2;3;4;5,"Deep-sweep 2026-06-12 on CUDA host. Backend coverage already complete: all 4 backends exercised for idw/kriging/spline incl. cross-backend equivalence and variance paths; no Cat 1 gaps. Filed #3290 for MEDIUM gaps, all verified correct-by-probe before filing (test-only fix): idw fill_value zero-weight branch (deterministic via 1e200 distance weight underflow; added numpy+dask+cupy, cupy RAN+PASSED), idw power only tested at default (exact oracle 10/(2^p+1)), spline collinear lstsq fallback, kriging duplicate points + all-equal-z (zero-variance variogram) + exactly-singular K regularisation retry (unit test on _build_kriging_matrix with all-zero variogram), spline/kriging 1x1 template, Inf/-Inf point filtering (only NaN was tested), lat/lon dim-name propagation (parametrized all 3 funcs), idw attrs preservation, 0-column template. Remaining minor untested: _build_kriging_matrix warn-then-NaN branch (needs mocked LinAlgError on retry). LOW documented not fixed: no asv benchmarks, non-uniform cell spacing unasserted. Full file 82 passed 0 skipped locally." interpolate-kriging,2026-06-04,2920;2921,HIGH,1;2;3;4;5,"Single public fn kriging(); all 4 backends already had cross-backend parity tests (numpy/cupy/dask+numpy/dask+cupy) incl. cupy & dask+cupy variance -- ran green on CUDA host. Gaps closed (test-only, #2921): Cat1 dask+numpy return_variance branch (_chunk_var) was untested -> added test_dask_return_variance_matches_numpy (atol=1e-12, var ~1e-14). Cat4 nlags only default(15) tested -> added non-default nlags=5 + invalid paths (nlags=0/-1 ValueError, nlags=2.5 TypeError). Cat2/3 two-point <3-lag-bins UserWarning branch -> test_two_point_warns_few_lag_bins. Cat2 all-NaN kriging input -> test_kriging_all_nan_points (only idw covered before). Cat5 output metadata (coords/dims/attrs/name) untested -> added test_output_metadata. Single-point kriging CRASHES (zero-size array reduction in _experimental_variogram, N=1) -- real source bug filed #2920; added xfail(strict, raises=ValueError) test_single_point documenting expected graceful behavior; source fix left to #2920 (test-only PR). LOW/not filed: singular-matrix K_inv-is-None all-NaN branch is defensive and unreachable via public API. GPU-validated." diff --git a/xrspatial/geotiff/tests/write/test_category_sidecar_backends_3483.py b/xrspatial/geotiff/tests/write/test_category_sidecar_backends_3483.py new file mode 100644 index 000000000..976910de5 --- /dev/null +++ b/xrspatial/geotiff/tests/write/test_category_sidecar_backends_3483.py @@ -0,0 +1,97 @@ +"""Categorical PAM sidecar round-trips across write backends (#3483, #3518). + +``to_geotiff`` emits a PAM ``.aux.xml`` sidecar from +``attrs['category_names']`` / ``attrs['category_colors']`` so GDAL/QGIS show +class labels. The emit is wired into every write path -- the eager numpy +writer, the dask streaming writer, and the GPU (nvCOMP) writer each call +``_write_category_sidecar()`` after writing pixels. The original feature +tests (``xrspatial/tests/test_rasterize_categorical_3482.py``) only round-trip +the eager numpy path, so a refactor that dropped the sidecar emit on the dask +or GPU branch would lose category labels with no failing test. + +These tests close that backend-coverage gap: a categorical int32 raster is +written through the dask and GPU backends and read back, asserting both the +labels and the RGBA colors survive. A colors-less variant also covers the +``category_colors=None`` build branch and the names-only read path, which the +existing suite always exercises with colors present. +""" +import os + +import numpy as np +import pytest +import xarray as xr + +from xrspatial.geotiff import open_geotiff, to_geotiff + +from .._helpers.markers import requires_gpu + +pytest.importorskip("tifffile") + +_NAMES = ["water", "forest", "urban"] +_COLORS = [(0, 0, 255, 255), (0, 255, 0, 255), (128, 128, 128, 255)] +# -1 is the categorical nodata sentinel rasterize uses for untouched cells. +_CODES = np.array([[-1, 0, 1], [1, 2, 0], [2, -1, 1]], dtype=np.int32) + + +def _categorical_da(values, *, with_colors=True): + """An int32 categorical DataArray carrying category attrs + georef.""" + attrs = { + "category_names": list(_NAMES), + "nodata": -1, + "transform": (1.0, 0.0, 0.0, 0.0, -1.0, 3.0), + "crs": 4326, + } + if with_colors: + attrs["category_colors"] = list(_COLORS) + return xr.DataArray( + values, + dims=("y", "x"), + coords={"y": [2.5, 1.5, 0.5], "x": [0.5, 1.5, 2.5]}, + attrs=attrs, + ) + + +def test_dask_write_round_trips_categories(tmp_path): + """The dask streaming writer emits and round-trips the sidecar.""" + import dask.array as dsa + + da = _categorical_da(dsa.from_array(_CODES, chunks=(2, 3))) + path = str(tmp_path / "dask_cat_3483.tif") + to_geotiff(da, path) + + assert os.path.exists(path + ".aux.xml") + back = open_geotiff(path) + assert back.attrs["category_names"] == _NAMES + assert list(back.attrs["category_colors"]) == _COLORS + + +@requires_gpu +def test_gpu_write_round_trips_categories(tmp_path): + """The GPU (nvCOMP) writer emits and round-trips the sidecar.""" + import cupy + + da = _categorical_da(cupy.asarray(_CODES)) + path = str(tmp_path / "gpu_cat_3483.tif") + to_geotiff(da, path, gpu=True) + + assert os.path.exists(path + ".aux.xml") + back = open_geotiff(path) + assert back.attrs["category_names"] == _NAMES + assert list(back.attrs["category_colors"]) == _COLORS + + +def test_round_trips_names_without_colors(tmp_path): + """A names-only categorical array round-trips labels with no colors. + + Exercises the ``category_colors=None`` branch of ``build_pam_xml`` (no + Red/Green/Blue/Alpha RAT columns) and the matching names-only read path, + which the existing suite never hits because it always supplies colors. + """ + da = _categorical_da(_CODES, with_colors=False) + path = str(tmp_path / "names_only_3483.tif") + to_geotiff(da, path) + + assert os.path.exists(path + ".aux.xml") + back = open_geotiff(path) + assert back.attrs["category_names"] == _NAMES + assert "category_colors" not in back.attrs