Match cKDTree max_distance check to the brute-force float32 convention (#3392)#3555
Merged
Conversation
#3392) The cKDTree backends (eager numpy PROXIMITY and the dask paths for every mode) compared the float64 distance against max_distance, while the brute-force and CUDA kernels round the distance to float32 first. A max_distance landing inside the float32 ulp gap just below a target's distance dropped that target to NaN on the cKDTree paths but kept it on the brute-force backends. _inclusive_upper_bound now widens the bound to the midpoint between the largest float32 <= max_distance and the next float32 up, matching the kernels' float32 keep test, while preserving the absolute floor that keeps the p=2 boundary inclusive at small max_distance.
brendancol
commented
Jun 27, 2026
brendancol
left a comment
Contributor
Author
There was a problem hiding this comment.
PR Review: Match cKDTree max_distance check to the brute-force float32 convention (#3392)
Reviewed the full _inclusive_upper_bound change and the new tests against the
surrounding proximity code. The root-cause analysis holds up: I traced the
reproducer and it does reach _process_dask_kdtree (not the map_overlap halo
path the issue text guessed) because max_possible_distance is computed through
_distance, which returns float32. Putting the fix in the shared bound helper is
the right call since it covers every cKDTree consumer at once.
Blockers (must fix before merge)
None.
Suggestions (should fix, not blocking)
- The new tests pin numpy against dask+numpy only. The same boundary is worth
asserting on cupy / dask+cupy so the four backends can't drift apart again
(the older #3389*_gpu_matches_cputests cover GPU allocation, and the #3442
block parametrizes all four backends viacreate_test_raster). Not blocking
because the fix is unreachable from the brute-force GPU paths and the helper
is backend-independent, but a 4-backend parametrization would close the gap.
Nits (optional improvements)
xrspatial/tests/test_proximity.py:2670uses
np.linalg.norm([lon[j] - lon[0]], ord=p)to get the distance. For a
one-element vector that is justabs(lon[j] - lon[0])for both p=1 and p=2;
a plainabs(...)reads more directly. The current form is correct, only
slightly indirect.
Out of scope (mention only)
max_possible_distanceis still float32 (via_distanceat
xrspatial/proximity.py:1388), so a sub-ulp bounded request is routed to the
global-tree path instead ofmap_overlap. With this fix the result is correct
either way, so it is a routing/perf nuance rather than a bug. Fine to leave;
flagging so it is on record.- Pre-existing flake8 E127 at
xrspatial/tests/test_proximity.py:1134and
:1160, unrelated to this PR.
What looks good
- The midpoint bound is the exact supremum of float64 distances that round to a
float32 value within range, so it matches the kernels without admitting
anything they exclude. Verified the widen is half a float32 ulp across
magnitudes and the absolute floor still keeps p=2 inclusive at small
max_distance (md=0 -> 8.88e-16, squaring-safe). - Regression tests fail without the change (6 of them) and the mirror
"just below" tests confirm a genuinely out-of-range target still NaNs out. - Manually confirmed all four backends return the target at the boundary.
Checklist
- Algorithm matches the established float32 keep convention (#3389)
- All implemented backends produce consistent results
- NaN handling is correct (inclusion widened, exclusion preserved)
- Edge cases covered (md=0 floor, inf, just-below boundary)
- Dask paths handled (shared helper covers global/tiled and chunk fns)
- No premature materialization or unnecessary copies
- Benchmark not needed (scalar helper, no perf-critical change)
- README feature matrix not applicable (no new function)
- Docstring updated to explain the float32 reasoning
…3392) Parametrize the new keep/exclude tests across numpy, dask+numpy, cupy, and dask+cupy so the brute-force GPU backends are pinned to the cKDTree backends at the boundary and cannot drift apart. Replace the one-element np.linalg.norm with a plain longitude difference.
brendancol
commented
Jun 27, 2026
brendancol
left a comment
Contributor
Author
There was a problem hiding this comment.
Follow-up review (after 917dc90)
Re-reviewed the diff after the follow-up commit.
Disposition of prior findings
- Suggestion (4-backend coverage): fixed. The keep/exclude tests now
parametrize over numpy, dask+numpy, cupy, and dask+cupy via the_ulp_raster
helper, so the brute-force GPU backends are pinned to the cKDTree backends at
the boundary. 36 boundary cases pass locally (CUDA backends included). - Nit (
np.linalg.normon a one-element vector): fixed. Replaced with the
plain longitude difference, which is the distance for both p=1 and p=2 on the
constant-latitude row. max_possible_distancefloat32 routing and the pre-existing flake8 E127s:
left as-is, both noted in the first review as out of scope (correct either
way / not introduced here).
Status
No blockers, no remaining actionable findings. Full test_proximity.py passes
(579 tests across all four backends).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3392
Bounded EUCLIDEAN/MANHATTAN allocation (and proximity/direction) on a dask
array could return NaN for a target whose distance sits one float32 ulp above
max_distance, while the eager numpy/cupy backends returned the target.Root cause
The cKDTree backends compared the float64 distance against
max_distance, butthe brute-force and CUDA kernels round the distance to float32 before the
dist <= max_distancecheck (the convention #3389 established). Whenmax_distancelands in the float32 ulp gap just below a target's distance, thetwo conventions disagree.
The dask reproducer reaches the cKDTree path because
max_possible_distanceisitself float32-rounded, so a sub-ulp bounded request gets routed to the
global-tree path rather than the halo'd
map_overlappath. The samefloat64/float32 mismatch affects every cKDTree consumer, including eager numpy
proximity, so the fix lives in the shared bound helper instead of the routing.
Fix
_inclusive_upper_boundnow widens the query bound to the midpoint between thelargest float32
<= max_distanceand the next float32 up. That midpoint is thesupremum of float64 distances that still round to a float32 value within range,
so the cKDTree decision matches the kernels and nothing the kernels exclude is
admitted. The absolute floor that keeps the p=2 boundary inclusive at small
max_distance(#3442) is preserved.Backend coverage
Test plan
eager numpy at a float32-ulp
max_distance, for EUCLIDEAN and MANHATTAN(they fail without the fix).
max_distanceone float32 ulp below the target'sdistance still excludes it on both backends.
xrspatial/tests/test_proximity.pypasses (552 tests).