fix: propagate panics from TableStrategy parallel column writers (upstream #8672)#73
Merged
1 commit merged intoJul 20, 2026
Merged
Conversation
…ks. (vortex-data#8672) ## Rationale for this change `TableStrategy` currently hides panics that happen in the individual column streams. ## What changes are included in this PR? Instead of detaching the fanout work, we await it and make sure everything finishes successfully. ## What APIs are changed? Are there any user-facing changes? No API change, added test to verify panic propagation. Signed-off-by: Adam Gutglick <adam@spiraldb.com> (cherry picked from commit 4abe3d0)
This was referenced Jul 20, 2026
Closed
There was a problem hiding this comment.
Pull request overview
This PR cherry-picks an upstream fix to vortex-layout’s TableStrategy writer to ensure failures in the “fan-out/transposition” stage (including panics) cannot be silently converted into channel EOF, which previously allowed columns to finalize at different lengths without failing the write.
Changes:
- Replace the detached fan-out task with a
fanout_futthat is joined with the column-writer futures viafutures::future::try_join, so producer errors/panics surface to the caller. - Add an explicit error when a column writer finishes early (send fails), preventing silent truncation.
- Add a regression test (
table_fanout_panic_propagates) that validates a panic in the transposition stream propagates (i.e., fails loudly).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This was referenced Jul 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry-pick of upstream vortex-data#8672 (
4abe3d04f0) ontospiceai-54.Upstream yanked 0.76.0 (the release this branch tracks) over this bug:
TableStrategyruns its column-chunk fan-out task detached and discards channel-send results, so a panic/death in the parallel write pipeline lets surviving column writers see a clean EOF and finalize columns at different lengths silently — the written file still reports full row count. With the fix, the fan-out future is joined with the column writers and any producer panic/error fails the write loudly.Context and evidence chain: spiceai/spiceai#11942 (likely root cause of the SF1000
chbench_q10c_citydivergence behind spiceai/spiceai#11910). Consumed by spiceai/spiceai#11943, which currently pins this branch's tip68badd689— after this merges, that pin moves to thespiceai-54merge SHA.Upstream's regression test
table_fanout_panic_propagatespasses on this branch (cargo test -p vortex-layout table_fanout).