Skip to content

feat(compute): support casts between binary/string and binary_view/string_view #802

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
zeroshade wants to merge 23 commits into
base: main
Choose a base branch
from
Open

feat(compute): support casts between binary/string and binary_view/string_view #802

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
65aa957
feat(compute): support casts between binary/string and binary_view/st…
zeroshade May 5, 2026
d5f2473
fix(compute): coalesce view payload into a single data buffer in cast
zeroshade May 5, 2026
486c612
linting
zeroshade May 5, 2026
a741fd8
fix(compute): handle multi-buffer views and size view outputs for cast
zeroshade May 5, 2026
703758f
fix(compute): extend view coalescing to chunked inputs and guard over…
zeroshade May 5, 2026
f787135
fix(compute): respect destination builder limits in formatted-data guard
zeroshade May 5, 2026
27e6af8
fix(compute): clamp formatted-data limit to platform int on 32-bit bu…
zeroshade May 5, 2026
fee033f
test(compute): make large_utf8 limit assertion platform-aware
zeroshade May 5, 2026
41e5abf
fix(compute): unpack view-typed dictionaries without TakeArray
zeroshade May 5, 2026
8dbf269
fix(compute): preserve value-level nulls when unpacking view dictiona…
zeroshade May 5, 2026
a5e93fa
fix(compute): keep view outputs in a single buffer for dict and bool …
zeroshade May 5, 2026
9ad9cae
fix(compute): use exact formatted payload for bool casts; add binary_…
zeroshade May 5, 2026
b8d9b96
test(compute): cover binary_view dict value-level null preservation
zeroshade May 5, 2026
b54a657
fix(compute): skip view overflow-buffer reservation for inline-bound …
zeroshade May 5, 2026
f15582b
fix(compute): treat 12-byte values as inline to match ViewHeader.IsIn…
zeroshade May 5, 2026
45760d1
fix(arrow): align IsViewInline with ViewHeader.IsInline and docs (<=12)
zeroshade May 5, 2026
3100f32
fix(compute): coalesce multi-buffer view values inside dictionary inputs
zeroshade May 5, 2026
02afee2
fix(compute): recurse into nested children when coalescing view payloads
zeroshade May 5, 2026
76b3ff5
fix(compute): treat extension inputs as their storage type when coale…
zeroshade May 5, 2026
09a8574
fix(compute): preserve Dictionary() across extension unwrap/rewrap
zeroshade May 5, 2026
d05c46b
refactor(compute): consolidate branch-introduced duplication
zeroshade May 6, 2026
e1414c4
refactor(compute): share structure between view cast paths
zeroshade May 6, 2026
074dc5e
refactor(compute): box unpackViewDictionaryIntoBuilder closures in an…
zeroshade May 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions arrow/array/util.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -447,6 +447,8 @@ func getMaxBufferLen(dt arrow.DataType, length int) int {
return bufferLen
case arrow.OffsetsDataType:
return maxOf(dt.OffsetTypeTraits().BytesRequired(length + 1))
case arrow.BinaryViewDataType:
return maxOf(arrow.ViewHeaderSizeBytes * length)
case *arrow.FixedSizeListType:
return maxOf(getMaxBufferLen(dt.Elem(), int(dt.Len())*length))
case arrow.ExtensionType:
Expand Down Expand Up @@ -496,6 +498,8 @@ func (n *nullArrayFactory) create() *Data {
dictData = arr.Data()
case arrow.FixedWidthDataType:
bufs = append(bufs, n.buf)
case arrow.BinaryViewDataType:
bufs = append(bufs, n.buf)
case arrow.BinaryDataType:
bufs = append(bufs, n.buf, n.buf)
case arrow.OffsetsDataType:
Expand Down
Loading