perf(optimizer): skip O(n^2) connector simplification when no operands can combine#7687
Open
tobymao wants to merge 1 commit into
Open
perf(optimizer): skip O(n^2) connector simplification when no operands can combine#7687tobymao wants to merge 1 commit into
tobymao wants to merge 1 commit into
Conversation
…s can combine _flat_simplify scans operand pairs in O(n^2). For connectors, a pair can only combine when one side is a constant or both are comparisons (see _simplify_connectors / _simplify_comparison), so when no operand is combinable the scan is a guaranteed no-op. Returning early in that case avoids the quadratic blowup on large connectors of inert operands (e.g. a 1000-way OR of ANDs). The check is placed inside _flat_simplify so it reuses the already-flattened operand queue (no extra traversal) and is gated on exp.Connector, leaving the simplify_equality caller unaffected. Output is unchanged. Benchmarks (optimize, best of 5): condition_100: 67.8ms -> 54ms condition_500: 648.7ms -> 297ms (2.2x) condition_1000: 2124ms -> 609ms (3.5x) TPC-H is neutral (verified via interleaved A/B).
Contributor
SQLGlot Integration Test ResultsComparing:
By Dialect
Overallmain: 113231 total, 112181 passed (pass rate: 99.1%) sqlglot:perf/simplify-flat-connector-quadratic: 101037 total, 101037 passed (pass rate: 100.0%) Transitions: Dialect pair changes: 0 previous results not found, 3 current results not found ✅ 64 test(s) passed |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
_flat_simplifyscans connector operand pairs in O(n²). A pair can only combine when one side is a constant or both are comparisons (see_simplify_connectors/_simplify_comparison), so when no operand is combinable the entire scan is a guaranteed no-op. This adds an early return in that case.The check lives inside
_flat_simplifyso it reuses the already-flattened operandqueue(no extra traversal) and is gated onexp.Connector, leaving thesimplify_equalitycaller unaffected.Why
optimizewas ~O(n²) in the size of a flat boolean condition. A 1000-termORofANDs spent ~1M pairwise_simplify_connectorscalls, 0% of them productive (the operands are parenthesized ANDs, which are inert).Impact
Benchmarks (
optimize, best of 5):TPC-H is neutral (verified via interleaved A/B). Output is unchanged — the optimizer and transpile fixtures (which assert exact SQL) pass, and the full unit suite is green.