Remove the previous file's PAM sidecar when to_geotiff overwrites a path (#3595)#3599
Merged
Merged
Conversation
brendancol
commented
Jul 2, 2026
brendancol
left a comment
Contributor
Author
There was a problem hiding this comment.
PR Review: Remove the previous file's PAM sidecar when to_geotiff overwrites a path (#3595)
Blockers (must fix before merge)
None.
Suggestions (should fix, not blocking)
-
xrspatial/geotiff/_writers/eager.py:515-531: the removal only runs at the four success return points, so a failed write leaves the old file and its sidecar consistent with each other. That ordering matters and nothing pins it. A small test (patch the encoder to raise mid-write, assert the old.aux.xmlsurvives) would keep a future refactor from moving the removal before the pixel write.
Nits (optional improvements)
-
xrspatial/geotiff/_writers/eager.py:421-427: the refresh contract is documented under thecolor_rampparameter, but it applies to every string-path write, including plain ones with no ramp and no categories. A reader checking why their hand-placed.aux.xmlvanished after a plain write won't look there. Consider a Notes section or a sentence near the top of the docstring. -
xrspatial/geotiff/_writers/eager.py:529:except OSError: passalso swallowsPermissionError, so on a locked sidecar (Windows) the write succeeds and the stale sidecar silently stays. That matches GDAL's QuietDelete, so it's a documented-behavior question rather than a bug; a comment word about it would do.
What looks good
- The removal lives in the shared
_write_sidecarsclosure, so all four write paths (eager, dask streaming, GPU dispatch, VRT) get it from one place, and theisinstance(path, str)guard keeps file-like destinations untouched. - Keeping the
.qmlis the right call: QGIS users expect styles to survive data updates, and the test pins that choice explicitly. - Test coverage is thorough for a sidecar fix: both sidecar kinds in both directions, the multiband symbology no-op, foreign sidecars at fresh paths, bare ndarray input, and dask/VRT/GPU paths. The VRT test correctly documents why it can't use a same-path overwrite (tiles-dir guard).
Checklist
- Algorithm matches reference (GDAL
GDALDriver::QuietDeletesemantics) - All write paths produce consistent sidecar state
- NaN handling: n/a (no pixel math)
- Edge cases covered by tests (foreign sidecar, multiband no-op, ndarray, file-like unaffected by guard)
- Dask path exercised
- No premature materialization (one
os.remove, no data reads) - Benchmark not needed (metadata-only path)
- README feature matrix: no new function, no change needed
- Docstring updated (see nit on placement)
…comment on OSError swallow (#3595)
brendancol
commented
Jul 2, 2026
brendancol
left a comment
Contributor
Author
There was a problem hiding this comment.
Follow-up review: commit 5fed444
All three findings from the first pass are addressed:
- Failed-write ordering is now pinned by
test_failed_write_keeps_old_sidecar, which monkeypatches the module-levelwriteseam to raise and asserts the old sidecar (and itscategory_nameson re-read) survives. - The refresh contract now also appears in the top-level
to_geotiffdocstring, ahead of the Parameters section, so it's visible for plain writes with no ramp or categories. - The
except OSErrorblock explains the PermissionError/QuietDelete trade-off inline.
Test file is at 11 passing tests. No new findings; nothing blocking.
Contributor
Author
|
@copilot resolve the merge conflicts in this pull request |
…e eager.py docstring conflict)
Contributor
Done. Merged |
…eotiff-2026-07-01 # Conflicts: # xrspatial/geotiff/_writers/eager.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3595.
_write_sidecarsnow deletes a pre-existing<path>.aux.xmlon every successful string-path write before re-creating it, so an overwrite can't hand the old file'scategory_names/STATISTICS_*to the new pixels. This matches GDAL'sGDALDriver::QuietDeleteon dataset creation..qmlstyle sidecar is left alone unless a newcolor_ramp=write replaces it: QGIS treats.qmlas user styling that persists across data updates.Backend coverage: numpy, dask, GPU (nvCOMP), and VRT writes are each exercised in the new tests; the GPU test ran live on a CUDA host.
Test plan:
xrspatial/geotiff/tests/write/test_stale_sidecar_overwrite_3595.py(categorical->plain, ramp->plain keeps.qml, ramp->categorical, categorical->ramp, multiband symbology no-op still removes, foreign sidecar at a fresh path, bare ndarray input, dask, VRT, GPU)Found by the metadata propagation sweep (state CSV update included on this branch).