Background
Gluten vendors its Substrait .proto files (gluten-substrait/src/main/resources/substrait/proto/substrait/) pinned near Substrait v0.23.0 with local patches, while upstream Substrait is now at 0.98.0. The JVM plan producer and both native consumers (Velox, ClickHouse) regenerate from this single proto source and ship together, and Gluten's Substrait plans are transient (built per query, serialized across JNI, discarded) — so there is no wire-compatibility constraint and the rebase is a pure code migration.
Goal
Bring the vendored proto to official 0.98.0 core (verbatim) plus Gluten's genuinely engine-specific additions grafted at non-colliding numbers, migrating the JVM producer and both native backends in lockstep.
Approach
Land as small, independently-green PRs, one per message family (each keeps protoc + JVM + both native builds compiling). Minimal rebase first — relocate only colliding fork fields and drop dead/decided-drop items; extracting engine-specific forks to formal Substrait extensions is deferred to a follow-up.
Increments
Related: upstream consolidation work is tracked under the apache-gluten label in substrait-io/substrait.
Background
Gluten vendors its Substrait
.protofiles (gluten-substrait/src/main/resources/substrait/proto/substrait/) pinned near Substrait v0.23.0 with local patches, while upstream Substrait is now at 0.98.0. The JVM plan producer and both native consumers (Velox, ClickHouse) regenerate from this single proto source and ship together, and Gluten's Substrait plans are transient (built per query, serialized across JNI, discarded) — so there is no wire-compatibility constraint and the rebase is a pure code migration.Goal
Bring the vendored proto to official 0.98.0 core (verbatim) plus Gluten's genuinely engine-specific additions grafted at non-colliding numbers, migrating the JVM producer and both native backends in lockstep.
Approach
Land as small, independently-green PRs, one per message family (each keeps
protoc+ JVM + both native builds compiling). Minimal rebase first — relocate only colliding fork fields and drop dead/decided-drop items; extracting engine-specific forks to formal Substrait extensions is deferred to a follow-up.Increments
Expression.Enumplan.proto/extensions.proto→ 0.98 (URI→URN,AdvancedExtension.optimizationrepeated)PrecisionTimestamp*; dropType.NothingCrossRel→NestedLoopJoinRel(+ join-type enum remap)WindowRel→ConsistentPartitionWindowRel;WindowType→BoundsTypeFetchRel/AggregateRel.Grouping/ExpandRelTopNRel/WriteRelReadRel(relocatestream_kafka; text-options redesign;VirtualTable)Reloneof reconciliation to exact 0.98 layoutRelated: upstream consolidation work is tracked under the
apache-glutenlabel insubstrait-io/substrait.