What
The one remaining lookup-join item — perf, not coverage. Coverage is done: the native operators drive Flink's own generated lookup runners over each Arrow probe batch (sync + async, INNER + LEFT, field-ref and constant keys, pre-filter, dim-side calc, residual condition), byte-identical to the host by construction; the only matcher gate left is the upsert-materialized lookup, which needs a changelog probe the island doesn't admit anyway.
The item
Preload a bounded dim into a native hash table once (RisingWave/Arroyo cache the dim side) and probe fully columnar — zero per-batch JVM crossing in steady state. Plus Arroyo's within-batch key dedup: one connector call for the batch's distinct missing keys (lookup_join.rs).
Deliberately deprioritized (2026-07-05)
After the collect-time columnar assembly shipped (the collector writes looked-up rows straight into the Arrow builders; q13 +22%), no lookup leaf registers above the 2% floor in the q13 profile — the query is bound by the rowwise perimeter and the generator itself. Nexmark's dim is an in-memory test connector, so the preload's real win (a slow external dim: JDBC/HBase-class latency) is invisible there by construction.
Pick this up when a real connector-backed workload profiles lookup-bound — not to chase a benchmark number.
Also parked here: a cross-batch async overlap (the AsyncWaitOperator port) would only matter if per-lookup latency were so high that blocking on a single batch stalls checkpoints unacceptably; not the case today (see the async-UDF issue for the within-batch model rationale).
What
The one remaining lookup-join item — perf, not coverage. Coverage is done: the native operators drive Flink's own generated lookup runners over each Arrow probe batch (sync + async, INNER + LEFT, field-ref and constant keys, pre-filter, dim-side calc, residual condition), byte-identical to the host by construction; the only matcher gate left is the upsert-materialized lookup, which needs a changelog probe the island doesn't admit anyway.
The item
Preload a bounded dim into a native hash table once (RisingWave/Arroyo cache the dim side) and probe fully columnar — zero per-batch JVM crossing in steady state. Plus Arroyo's within-batch key dedup: one connector call for the batch's distinct missing keys (
lookup_join.rs).Deliberately deprioritized (2026-07-05)
After the collect-time columnar assembly shipped (the collector writes looked-up rows straight into the Arrow builders; q13 +22%), no lookup leaf registers above the 2% floor in the q13 profile — the query is bound by the rowwise perimeter and the generator itself. Nexmark's dim is an in-memory test connector, so the preload's real win (a slow external dim: JDBC/HBase-class latency) is invisible there by construction.
Pick this up when a real connector-backed workload profiles lookup-bound — not to chase a benchmark number.
Also parked here: a cross-batch async overlap (the
AsyncWaitOperatorport) would only matter if per-lookup latency were so high that blocking on a single batch stalls checkpoints unacceptably; not the case today (see the async-UDF issue for the within-batch model rationale).