Fix missing return in _retryer that propagated None tiles#282
Open
madhavcodez wants to merge 1 commit into
Open
Fix missing return in _retryer that propagated None tiles#282madhavcodez wants to merge 1 commit into
madhavcodez wants to merge 1 commit into
Conversation
_retryer retries on non-404 failures, but the retry branch ran
request = _retryer(...)
without returning the result, so the function fell through and implicitly
returned None. That None was collected as a tile array and later crashed
_merge_tiles at "h, w, d = arrays[0].shape" (AttributeError) or at the array
assignment (TypeError), which is the intermittent failure reported in geopandas#252.
Return the recursive call. As defense-in-depth, _merge_tiles now raises a clear
ValueError if any tile array is None instead of failing with an obscure
AttributeError/TypeError further down.
Add network-free tests: _merge_tiles raises the clear ValueError for a missing
tile in first and later positions; _retryer returns the array after a non-404
failure then a successful retry; and _retryer raises (not returns None) when
retries are exhausted.
Closes geopandas#252
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause
_retryerretries on non-404 failures, but the retry branch never returned the recursive call's result:After kicking off the retry, the function fell through and implicitly returned
None. ThatNonewas collected as a tile "array" and later reached_merge_tiles, whereh, w, d = arrays[0].shaperaisedAttributeError: 'NoneType' object has no attribute 'shape'(or aTypeErroron the array assignment when theNonewas not in first position). This is why #252 was intermittent: it only surfaced when a tile hit a non-404 error and then succeeded on retry. The earliernp.asarray(None)theory in the thread was not the cause.Fix
_retryer(request = _retryer(...)becomesreturn _retryer(...)). This is the actual fix._merge_tilesnow raises a clearValueErrorif any tile array isNone, instead of failing with an obscureAttributeError/TypeErrorfurther down.Tests
Three network-free tests added:
_merge_tilesraises the clearValueErrorfor a missing tile in both first and later positions;_retryerreturns the array (notNone) after a non-404 failure followed by a successful retry; and_retryerraises (rather than returningNone) when retries are exhausted. They pass with the fix and fail without it.pytest -m "not network"reports 14 passed.The second change (the
_merge_tilesguard) is defense in depth and not required to fix #252. Happy to drop it if you would prefer the root-cause change only.Closes #252