Add single-stage parameterized query plan cache #226
Draft
praveenc7 wants to merge 1 commit into
Draft
Conversation
High-QPS single-stage workloads are often dominated by re-parsing the same query shape with only the literal values changing (e.g. point lookups with large IN-lists). Profiling such a query showed ~27% of broker CPU in RequestUtils.parseQuery -> CalciteSqlParser, with no parse/plan cache. Add an opt-in plan cache keyed on a literal-agnostic template: - QueryTemplateNormalizer: a cheap lexer pass (not a full parse) that masks literals with kind-tagged placeholders (?s/?i/?d) and extracts their values in order. Distinct kinds prevent a string from ever binding into a numeric leaf. Correctness is guarded by a per-template self-consistency check (isCacheable re-derives each literal and verifies it reproduces the parsed plan's leaves); positional bind rebinds them on a hit. - ParameterizedPlanCache: caches the un-rewritten PinotQuery per shape; on a hit it deep-copies the template, rebinds literals, and runs the rewriters fresh, producing a result byte-identical to CalciteSqlParser.compileToPinotQuery. Production safety and observability: - Only single-stage DQL is eligible; DML and the per-query usePlanCache=false option bypass the cache. - Optional sampled shadow-verification (verification.sample.rate) compares a hit against a fresh compile and, on any mismatch, returns the fresh plan, evicts the template, and emits QUERY_PLAN_CACHE_VERIFICATION_FAILURE. - Meters QUERY_PLAN_CACHE_HIT/MISS/UNCACHEABLE/VERIFICATION_FAILURE and gauge QUERY_PLAN_CACHE_SIZE. - Gated by pinot.broker.query.plan.cache.enabled (default false), with .size and .verification.sample.rate knobs. JMH (BenchmarkPlanCache, JDK21): on a cache hit, compile drops from ~88us to ~3.7us for the 26-value IN query (~24x), and 32.9us to 0.48us for a simple query -- the hit cost already includes normalize + deepCopy + rebind + rewrite. Tests: QueryTemplateNormalizerTest (round-trip + cross-binding vs direct parse, safety fallbacks) and ParameterizedPlanCacheTest (hit/miss/uncacheable, sampled verification pass + mismatch fail-safe, per-query bypass, metrics). Existing parser/compiler corpus and single-stage handler tests unchanged.
praveenc7
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
High-QPS single-stage workloads are frequently dominated by re-parsing the same query shape with only the literal values changing (e.g. point lookups with large
INlists). Profiling one such broker query showed ~27% of broker CPU inRequestUtils.parseQuery→CalciteSqlParser, with no parse/plan cache on the single-stage path. Because the literals vary per request, a plain text-keyed cache has a ~0% hit rate.https://observe.prod.linkedin.com/profiling-explorer/event/4728628
This adds an opt-in parameterized plan cache keyed on a literal-agnostic template:
QueryTemplateNormalizer— a cheap lexer pass (not a full parse) that masks literals with kind-tagged placeholders (?s/?i/?d) and extracts their values in order. Distinct kinds ensure a string can never bind into a numeric leaf.ParameterizedPlanCache— caches the un-rewrittenPinotQueryper shape; on a hit it deep-copies the template, rebinds the literals, and runs the query rewriters fresh, producing a result byte-identical toCalciteSqlParser.compileToPinotQuery. Query options are excluded from the key (the normal path applies them after rewrite) and bound per request.Anything ineligible transparently falls back to a normal parse. Multi-stage is out of scope (it validates against the schema during planning).
Config (all default-safe)
pinot.broker.query.plan.cache.enabledfalsepinot.broker.query.plan.cache.size1000pinot.broker.query.plan.cache.verification.sample.rate0.0Observability
Meters
QUERY_PLAN_CACHE_HIT/MISS/UNCACHEABLE/VERIFICATION_FAILURE, and gaugeQUERY_PLAN_CACHE_SIZE.Performance (JMH
BenchmarkPlanCache, JDK 21)On a cache hit the compile cost — which already includes normalize + deepCopy + rebind + rewrite — drops:
IN(the profiled query)Rollout
Enable with
enabled=trueand a non-zeroverification.sample.rate(e.g. 0.01–1.0) in EI/canary to catch any normalizer bug at near-zero cost, then lower the sample rate in production.Follow-ups (not in this PR)
Testing Done
QueryTemplateNormalizerTest(round-trip + cross-binding equal to a direct parse, safety fallbacks; 12 cases) andParameterizedPlanCacheTest(hit/miss/uncacheable, sampled verification pass + mismatch fail-safe, per-query bypass, metrics; 6 cases).CalciteSqlCompilerTest(78) andBaseSingleStageBrokerRequestHandlerTest(8) unchanged; checkstyle clean.BenchmarkPlanCache); numbers above."/tmp/plan-cache-pr-body.md" 52L, 3622B
TITLE: Add single-stage parameterized query plan cache (default off)