compute: columnar arrange input (C1)#37745
Closed
antiguru wants to merge 1 commit into
Closed
Conversation
Generalize `ensure_collections` and `arrange_collection` to accept a `CollectionEdge` instead of a `VecCollection`, so the arrange input path no longer decodes a columnar edge through `ColumnarToVec`. The columnar arm forms the arrangement key by iterating the borrowed column with `into_index_iter`, mirroring the proven `flat_map_datums` pattern, and never materializes an owned `Row`. The Vec arm is unchanged. The passthrough output preserves the input variant. Temporal bucketing (node C8) still consumes and produces a `Vec` edge, so the decode moves into that branch as a sanctioned leaf seam. Producers still emit `Vec`, so the columnar arm is dead in production and this change carries no runtime behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
antiguru
force-pushed
the
moritz/columnar-c1
branch
from
July 20, 2026 12:32
76a0d9b to
d04828f
Compare
Member
Author
|
Superseded by a same-repo stacked-PR chain on upstream (columnar edge migration C1 -> C3). Reopening the cross-fork PR is unnecessary; the same-repo stack replaces it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root of the C1 stack for the columnar dataflow edge migration (design doc in #37744).
Generalize
ensure_collectionsandarrange_collectionto accept aCollectionEdgeinstead of aVecCollection. The columnar arm forms the arrangement key directly off the borrowed column viainto_index_iter, mirroring theflat_map_datumspattern, and never materializes an ownedRow. The Vec arm is byte-identical to before, and the passthrough output preserves the input variant.Temporal bucketing (node C8) still consumes and produces a
Vecedge, so theinto_vecdecode moves into that branch as the sanctioned leaf seam until C8 makes bucketing columnar-native.Producers still emit
Vecin Wave 1, so the columnar arm is dead in production: this makes the consumer ready without any runtime behavior change. No producer is flipped here.Adds a cross-arm agreement unit test that arranges the same input as both a
Vecand a columnar edge, asserts identical arranged results, and asserts the columnar arm keeps the columnar passthrough variant (proving noColumnarToVecdecode on the path).