Skip retries in retryOperation for non-retriable storage errors#786
Draft
elirangoshen wants to merge 2 commits into
Draft
Conversation
Introduce a NON_RETRIABLE_ERRORS classification alongside the existing STORAGE_ERRORS list, and short-circuit retryOperation for errors where retrying is provably futile (the underlying IDB connection/store is broken at the filesystem level). The healing path is responsible for recovery; retryOperation should defer to it, not compete with it. Scoped to "Internal error opening backing store" — the LevelDB backing-store class analyzed in Expensify/App#87862, which accounts for ~26% of storage failures and produces ~6× log volume due to the 5-attempt retry loop. Fixes Expensify/App#90633 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
52 tasks
CI's "Verify API Docs Are Up To Date" step caught that the auto-generated docs missed the new "Non-retriable errors" branch added to retryOperation's JSDoc. Regenerated via `npm run build:docs`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Details
Introduces a
NON_RETRIABLE_ERRORSclassification alongside the existingSTORAGE_ERRORSlist and short-circuitsretryOperationfor errors where retrying is provably futile — specificallyInternal error opening backing store for indexedDB.open, where the LevelDB backing store is broken at the filesystem level and cannot recover from immediate retries.Today, every non-capacity storage error in
retryOperationretries 5 times with no delay, then silently resolves viaPromise.resolve(). For theUnknownError: Internal error opening backing store…class analyzed in Expensify/App#87862, this produces ~6× the log/Sentry volume of the underlying event rate — 884,955 occurrences (26.3% of all storage failures) inflated to ~5M log lines, all noise because no retry can succeed.The healing path (separate action item Expensify/App#90636) is responsible for recovering the IDB connection for this error class —
retryOperationshould defer to it, not compete with it.Why this is safe
retryOperationis reached, so callers cannot observe retry success vs. failure.Promise.resolve(), so the only observable change is log volume.STORAGE_ERRORS(includesagainst lowercasednameandmessage).Scope
Scoped to
Internal error opening backing store. The new classification structure (IDB_NON_RETRIABLE_ERRORS→NON_RETRIABLE_ERRORS, mirroringIDB_STORAGE_ERRORS→STORAGE_ERRORS) makes adding other connection-state errors trivial later.Related Issues
Expensify/App#90633
Automated Tests
Added two unit tests to
tests/unit/onyxUtilsTest.tsin the existingdescribe('retryOperation', ...)block:should not retry for non-retriable IndexedDB backing-store errors— mocksStorage.setItemto reject withnew Error('Internal error opening backing store for indexedDB.open.')(name:UnknownError), assertsretryOperationis called exactly once (not 6×).should log a single skip alert for non-retriable errors— asserts the newStorage operation skipped retry for non-retriable error...alert fires exactly once, and the existing "5 retries exhausted" alert does not fire.All existing
retryOperationregression tests continue to pass:Local results:
npx jest— 451/451 pass across 16 suites.npx tsc --noEmitclean. ESLint clean on changed files.Manual Tests
End-to-end manual verification against
Expensify/App(link to the App-side companion PR will be added once opened):Expensify/App'spackage.jsonreact-native-onyxdependency to this PR's head SHA, runnpm install.Storage.setItemand reject it withObject.assign(new Error('Internal error opening backing store for indexedDB.open.'), {name: 'UnknownError'}).Onyx.set()call (e.g., navigate to a screen that writes a transient key).Failed to save to storage. Error: ... retryAttempt: 0/5log line (not six).Storage operation skipped retry for non-retriable error. Error: ... onyxMethod: setWithRetry.alert.Storage operation failed after 5 retries...alert.Post-deploy production signal (per Expensify/App#90633 test plan):
Failed to save to storage. Error: ...Internal error opening backing store...drops ~5–6×.Storage operation skipped retry for non-retriable error...appears at roughly 1× the underlying event rate.Author Checklist
### Related Issuessection aboveTestssectiontoggleReportand notonIconClick)myBool && <MyComponent />.STYLE.md) were followedAvatar, I verified the components usingAvatarare working as expected)/** comment above it */thisproperly so there are no scoping issues (i.e. foronClick={this.submit}the methodthis.submitshould be bound tothisin the constructor)thisare necessary to be bound (i.e. avoidthis.submit = this.submit.bind(this);ifthis.submitis never passed to a component event handler likeonClick)Avataris modified, I verified thatAvataris working as expected in all cases)mainbranch was merged into this PR after a review, I tested again and verified the outcome was still expected according to theTeststeps.Screenshots/Videos
Android: Native
Android: mWeb Chrome
iOS: Native
iOS: mWeb Safari
MacOS: Chrome / Safari