[core] Optimize exact and TDigest percentile aggregation, including StarTree group-by

[xiangfu0]

Summary

• replace exact-percentile full sorting with an introspective three-way rank selector
• batch raw TDigest values and linearly merge them with sorted centroids
• decode and linearly merge serialized TDigest centroids on non-grouped and group-by star-tree query paths
• keep per-group raw-value and merge buffers lazy, including one-digest high-cardinality groups
• stop star-tree construction from compressing a TDigest after every input value while retaining safe writer sizing
• add deterministic JMH benchmarks, correctness coverage, a user guide, sample queries, and a sample star-tree config

Why

Exact percentile previously sorted every retained value even though the query needs only one rank. TDigest 3.2
profiling found three additional hot spots:

1. raw aggregation repeatedly sorted old centroids whenever its input buffer filled;
2. star-tree construction called byteSize() after every value, and byteSize() compresses the digest;
3. star-tree queries deserialized, materialized, shuffled, and sorted every stored digest before merging it.

The exact path now uses three-way introselect with median/ninther pivoting and a bounded sort fallback. Raw and
serialized TDigest paths merge sorted primitive arrays directly. Star-tree construction maintains a non-compressing
serialized-size bound and compresses at serialization time. Oversized legacy compact digests retain compact encoding
so clone and forward-index readback remain valid with t-digest 3.2.

Intermediate results and distributed merging still use the standard MergingDigest representation.

User guide and scope

The query optimizations are automatic. Exact percentile remains the choice for an exact order statistic:

SELECT PERCENTILE(latencyMs, 75) AS p75Exact
FROM myTable;

Use TDigest for bounded, mergeable approximation:

SELECT PERCENTILETDIGEST(latencyMs, 75) AS p75Approx
FROM myTable;

The same stored metric is used for grouped queries:

SELECT region, PERCENTILETDIGEST(latencyMs, 75) AS p75Approx
FROM myTable
GROUP BY region;

A star-tree can pre-aggregate TDigest with the default compression used by that query:

{
  "tableIndexConfig": {
    "starTreeIndexConfigs": [
      {
        "dimensionsSplitOrder": ["region", "deviceType"],
        "skipStarNodeCreationForDimensions": [],
        "functionColumnPairs": ["percentileTDigest__latencyMs"],
        "maxLeafRecords": 10000
      }
    ]
  }
}

An explicit query compression must match the index compression. Exact PERCENTILE cannot use a stored star-tree
percentile. The serialized query optimization covers non-grouped and group-by aggregation. Group states retain their
first serialized digest and allocate primitive merge buffers only when another entry reaches the group; the raw-value
buffer is allocated only if a raw value is added.

The complete user guide, workload boundaries, and commands are in
pinot-perf/benchmark-results/percentile-aggregation-100m-results.md.

Benchmark results

Single-thread JMH 1.37 on an Apple M4 Pro with 24 GiB RAM and OpenJDK 21.0.10. Matched comparisons used two forks and
five measurements per fork.

Workload	Baseline	Optimized	Latency reduction	Speedup
Exact P75, fresh holder, 100M raw rows	7,710.804 ms/op	764.626 ms/op	90.084%	10.084x
Exact P75, reused holder, 100M raw rows	7,499.571 ms/op	707.555 ms/op	90.565%	10.599x
Exact P75 extraction, 100M retained rows	7,448.796 ms/op	618.253 ms/op	91.700%	12.048x
TDigest P75, 100M raw rows	5,677.240 ms/op	2,555.127 ms/op	54.994%	2.222x
Star-tree query: merge 10K digests representing 100M rows	85.658 ms/op	7.100 ms/op	91.711%	12.065x
Star-tree construction kernel: 1M rows, 1K rows/group	1,840.391 ms/op	53.770 ms/op	97.078%	34.227x

The optimized star-tree query reduced allocation from 198,223,964 to 26,361 bytes/op and measured GC events from ten
to zero. It retained an exact digest size of 100 million and produced P75 0.7498860479, versus the sorted oracle
0.7499315464 (absolute error 0.0000454985).

The group-by extension uses 4a950bde9455—the preceding PR version with only the non-grouped optimization—as its
baseline. Every workload merges 10,000 stored entries representing 100 million source rows and extracts every group:

Groups	Digests/group	Baseline	Optimized	Latency reduction	Speedup	Baseline allocation	Optimized allocation
100	100	119.244 ms/op	9.275 ms/op	92.222%	12.856x	197,636,620 B/op	2,894,952 B/op
1,000	10	96.039 ms/op	11.371 ms/op	88.159%	8.446x	192,278,646 B/op	28,821,662 B/op
10,000	1	12.107 ms/op	9.078 ms/op	Not distinguishable	Parity	136,520,147 B/op	137,480,126 B/op

The merge-heavy 100- and 1,000-group cases reduce allocation by 98.535% and 85.010%. At 10,000 groups each group has
only one digest, so there is no merge to optimize; confidence intervals overlap and allocation is effectively parity
(+0.703%).
That case keeps the first digest pending and avoids allocating unused raw/merge buffers.

The final optimized construction kernel also completed an actual 100-million-row run in 5,455.170 ms
(18.331 million rows/s). Its matched baseline comparison uses 1 million rows; no unmeasured 100-million-row baseline
is extrapolated. The construction benchmark includes per-group serialization but excludes dimension sorting and
forward-index I/O. The query benchmark excludes planner, traversal, forward-index I/O, and distributed latency.

Validation

• PercentileTDigestAggregationFunctionTest: 41 tests passed, including serialized SV/MV group-by, lazy allocation,
  malformed input, and mixed serialized/raw growth coverage
• real StarTree PERCENTILETDIGEST ... GROUP BY execution matched useStarTree=false and scanned fewer documents;
  PercentileTDigestQueriesTest and its MV suite passed 12 tests
• PercentileTDigestValueAggregatorTest: 7 tests passed, including oversized compact clone/readback
• raw, multi-value, and pre-aggregated star-tree TDigest suites: 6 tests passed
• broader exact/TDigest aggregation and serialized-query suites passed
• spotless:apply: 0 files changed for the newly affected pinot-core and pinot-perf modules
• checkstyle:check: 0 violations in the newly affected modules
• license:format and license:check: passed in the newly affected modules
• full 64-module test-compile with -Xlint:all: passed with no warnings on changed lines
• final clean pinot-segment-local -Xlint:all compile: passed with no warnings on changed lines

[codecov-commenter]

Codecov Report

❌ Patch coverage is 83.09677% with 131 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.33%. Comparing base (fd772ee) to head (912280c).
⚠️ Report is 8 commits behind head on master.

Files with missing lines	Patch %	Lines
...egation/function/PercentileTDigestAccumulator.java	79.29%	102 Missing and 28 partials ⚠️
...ata/table/DeterministicConcurrentIndexedTable.java	83.33%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #18996      +/-   ##
============================================
+ Coverage     65.21%   65.33%   +0.11%     
  Complexity     1405     1405              
============================================
  Files          3418     3420       +2     
  Lines        214651   215425     +774     
  Branches      33922    34039     +117     
============================================
+ Hits         139989   140747     +758     
+ Misses        63356    63351       -5     
- Partials      11306    11327      +21     

Flag	Coverage Δ
custom-integration1	100.00% <ø> (ø)
integration	100.00% <ø> (ø)
integration1	100.00% <ø> (ø)
integration2	0.00% <ø> (ø)
java-21	65.33% <83.09%> (+0.11%)	⬆️
temurin	65.33% <83.09%> (+0.11%)	⬆️
unittests	65.33% <83.09%> (+0.11%)	⬆️
unittests1	57.04% <82.45%> (+0.19%)	⬆️
unittests2	37.63% <2.58%> (-0.11%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Copilot]

Pull request overview

This PR optimizes Pinot percentile aggregations by replacing full sorts for exact percentiles with an in-place rank selection algorithm, and by substantially reducing TDigest aggregation overhead (raw ingestion, serialized merging, and star-tree query/group-by paths) via batched + linear merge strategies.

Changes:

• Replace exact-percentile full sorting with an introspective three-way rank selector (with bounded sort fallback).
• Introduce a PercentileTDigestAccumulator to efficiently merge raw values and serialized TDigest centroids (including star-tree and group-by paths) without repeated centroid materialization/sorting.
• Add new deterministic JMH benchmarks and a percentile aggregation benchmark/user guide under pinot-perf.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/aggregator/PercentileTDigestValueAggregatorTest.java	Adds coverage for TDigest value-aggregator sizing/serialization contracts and deferred compression behavior.
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/aggregator/PercentileTDigestValueAggregator.java	Updates star-tree TDigest value aggregation to avoid per-update compression and to track safe serialized size bounds.
pinot-perf/src/main/java/org/apache/pinot/perf/aggregation/BenchmarkPercentileTDigestValueAggregator.java	Adds JMH benchmark for the star-tree construction TDigest raw-value aggregation + serialization kernel.
pinot-perf/src/main/java/org/apache/pinot/perf/aggregation/BenchmarkPercentileTDigestStarTreeAggregation.java	Adds JMH benchmark for merging serialized star-tree TDigest entries (non-grouped + group-by).
pinot-perf/src/main/java/org/apache/pinot/perf/aggregation/BenchmarkPercentileTDigestAggregation.java	Adds end-to-end TDigest percentile aggregation benchmark over 100M rows.
pinot-perf/src/main/java/org/apache/pinot/perf/aggregation/BenchmarkPercentileAggregation.java	Adds end-to-end and extraction-only exact percentile benchmarks over 100M rows.
pinot-perf/README.md	Links to the new percentile benchmark guide/results.
pinot-perf/benchmark-results/percentile-aggregation-100m-results.md	Adds user guide, sample queries/config, results, and reproduction commands for 100M-row percentile benchmarks.
pinot-core/src/test/java/org/apache/pinot/queries/PercentileTDigestQueriesTest.java	Extends query tests to validate star-tree TDigest GROUP BY correctness and fewer docs scanned vs non-star-tree.
pinot-core/src/test/java/org/apache/pinot/queries/PercentileTDigestMVQueriesTest.java	Disables the star-tree group-by test for the MV fixture (single-value star-tree stored metric).
pinot-core/src/test/java/org/apache/pinot/core/query/aggregation/function/PercentileTDigestAggregationFunctionTest.java	Adds extensive correctness tests for raw + serialized TDigest aggregation and group-by behavior (including malformed inputs and lazy allocation).
pinot-core/src/test/java/org/apache/pinot/core/query/aggregation/function/PercentileAggregationFunctionTest.java	Adds correctness tests to validate selection-based exact percentile extraction matches full-sort behavior across adversarial inputs.
pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/PercentileTDigestAggregationFunction.java	Switches TDigest aggregation to use the new accumulator for raw and serialized (including group-by) paths.
pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/PercentileTDigestAccumulator.java	Introduces a new internal accumulator to batch-sort raw values and linearly merge already-sorted serialized centroids.
pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/PercentileAggregationFunction.java	Replaces full sort in exact percentile final extraction with introselect-style selection + depth-limited fallback.

[Copilot]

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated no new comments.

[xiangfu0]

Found one high-signal TDigest serialization issue; see inline comment.

[Copilot]

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 4 comments.

[xiangfu0]

Reducer/group-by follow-up is now pushed in b7b01bd.

Benchmark highlights (TDigest 3.2, 2 forks × 5 measurements):

• Representative full IndexedTable combine, 32-way fan-in: 0.2683 ms pairwise → 0.0340 ms promoted local (7.90×), 0.0201 ms accumulator-local (13.33×), and 0.0233 ms serialized/wire (11.52×).
• Production-shaped 1,440 groups × 7 metrics × 32 segments: 4,574.4 ± 309.3 ms pairwise → 223.0 ± 2.3 ms accumulator-local (20.51×) and 283.0 ± 6.2 ms wire (16.16×). Allocation fell 93.87% and 92.22%.
• The 36 workload shapes × 4 implementations retained exact weight, finite positive centroids, monotonic quantiles, and standard TDigest 3.2 wire bytes. Maximum sampled P75 error was 0.001353 pairwise versus 0.000709 for the primitive candidates.
• The dependency-only TDigest 3.3 experiment was a no-go despite a 4.47× kernel geometric-mean speedup: mean P75 error increased 18.9× and the maximum sampled error reached 0.01138.

The full report and reproduction commands are in pinot-perf/benchmark-results/percentile-aggregation-100m-results.md.

Validation: 318 focused reducer/serde/concurrency tests passed; the 64-module warning-enabled test compilation passed (921 goals); spotless, checkstyle, license format/check, and diff-check all pass. New CI is running on this head.

[xiangfu0]

Found one high-severity TDigest intermediate serialization issue; see inline comment.