[Go][Compute] Panic in isIn kernel for FixedSizeBinary: slice bounds out of range in bufferbuilder.go

[Patzifist]

Describe the bug, including details regarding any error messages, version, and platform.

Description:
I am encountering a runtime panic when executing a SetLookup (IsIn) operation on a FixedSizeBinary column using the Go implementation of Apache Arrow (v18).
The stack trace indicates that the panic occurs within visitFSB (Fixed Size Binary visitor) inside the isInKernelExec. Specifically, it fails when the BinaryMemoTable attempts to look up values, leading to an invalid slice operation in the underlying bufferBuilder.

Steps to Reproduce:
Data: TPC-DS dataset SF1

The issue appears when using a filter with an IN clause on a column with a fixed-size type.
Example logic:

-- i_category is a FixedSizeBinary type
SELECT i_item_id, i_item_desc, i_category, i_current_price
FROM item
WHERE i_category IN ('Sports', 'Books', 'Home', 'Test');

Observed Behavior:
The application panics with:

panic: runtime error: slice bounds out of range [:16] with capacity 0

goroutine 3835099 [running]:
://github.com
arrow/array/bufferbuilder.go:85
...
://github.com
arrow/compute/internal/kernels/scalar_set_lookup.go:75
://github.com
arrow/compute/internal/kernels/scalar_set_lookup.go:73

Component(s)

Parquet, Other

[Patzifist]

Works only if you trim the column.
SELECT
i_item_id,
i_category,
i_current_price
FROM item
WHERE trim(i_category) IN ('Sports', 'Books', 'Home', 'Test');

[zeroshade]

Can you provide code that reproduces the issue?

[Patzifist]

package main

import (
	"context"
	"fmt"
	"sync"
	"testing"

	"github.com/apache/arrow-go/v18/arrow"
	"github.com/apache/arrow-go/v18/arrow/array"
	"github.com/apache/arrow-go/v18/arrow/compute"
	"github.com/apache/arrow-go/v18/arrow/memory"
)

func TestReproduceArrowPanicAndScalarError(t *testing.T) {
	mem := memory.NewGoAllocator()
	ctx := context.Background()

	catType := &arrow.FixedSizeBinaryType{ByteWidth: 50}
	schema := arrow.NewSchema([]arrow.Field{{Name: "i_category", Type: catType, Nullable: true}}, nil)

	numRows := 1000
	b := array.NewFixedSizeBinaryBuilder(mem, catType)
	for i := 0; i < numRows; i++ {
		buf := make([]byte, 50)
		copy(buf, "SPORTS")
		b.Append(buf)
	}
	dataArr := b.NewArray()
	defer dataArr.Release()

	batch := array.NewRecordBatch(schema, []arrow.Array{dataArr}, int64(numRows))
	defer batch.Release()

	vBuilder := array.NewFixedSizeBinaryBuilder(mem, catType)
	buf := make([]byte, 50)
	copy(buf, "SPORTS")
	vBuilder.Append(buf)
	valueSet := vBuilder.NewArray()
	defer valueSet.Release()

	opts := compute.SetOptions{
		ValueSet:     compute.NewDatumWithoutOwning(valueSet),
		NullBehavior: compute.NullMatchingMatch,
	}

	var wg sync.WaitGroup
	numWorkers := 40
	errChan := make(chan error, numWorkers*100)

	for w := 0; w < numWorkers; w++ {
		wg.Add(1)
		go func() {
			defer wg.Done()
			for i := 0; i < 50; i++ {

				col := batch.Column(0)
				col.Retain()
				input := compute.NewDatum(col)

				mask, err := compute.IsIn(ctx, opts, input)
				if err != nil {
					errChan <- err
					continue
				}

				batchInput := compute.NewDatumWithoutOwning(batch)
				_, err = compute.Filter(ctx, batchInput, mask, *compute.DefaultFilterOptions())
				if err != nil {
					errChan <- fmt.Errorf("filter err: %w", err)
				}

				mask.Release()
			}
		}()
	}

	wg.Wait()
	close(errChan)

	for e := range errChan {
		fmt.Printf("Caught: %v\n", e)
	}
}