Skip to content

Perf backlog: native timing hook, last scalar-keyed maps, Fluss aggregate levers #14

Description

@jordepic

What

The remaining tail of the standing profiling/perf backlog (the Criterion micro-bench harness, the end-to-end vs-Flink throughput bench, and most of the hot-path fixes they surfaced have shipped — see docs/benchmarks.md and docs/optimizations.md).

Remaining items

  • A lightweight native timing/counter hook behind a feature flag — per-operator batch count, row count, and wall time, dumpable on close — so a real job can be profiled without a full tracing dependency. Zero-cost when the flag is off.
  • Two-phase local/global micro-benches (the Criterion suite covers the single-phase operators).
  • The last scalar-keyed maps, under the standing rule swap to arrow-row keys only with a bench showing it pays: the window Top-N ranker (WindowRanker), the changelog normalizer, the temporal join, and the mini-batch local aggregate (hot only under tuned mini-batch mode — revisit when the tuned column becomes a standing benchmark). Everything else already uses arrow-row keys; the wins that motivated the rule are in docs/optimizations.md §4.
  • Fluss-rung aggregate levers, re-scoped by the 2026-07-06 source profile (.claude/research/fluss-source-profile-2026-07.md): the coalescing hypothesis is refuted (fluss-rs delivers ~83K-row batches; the source is zstd-bound and lean) — the q15/q16/q17 cells are decided by the changelog aggregate's allocation churn (Vec growth, mimalloc purge, ScalarValue emit). Ranked levers + acceptance benchmarks in the research note. (The other lever the profile ranked — per-row DATE_FORMAT re-parsing its pattern — shipped: the pattern is now compiled once and rendered into a reused buffer, commit 2c7f694.)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions