fix(ol_dbt_cli): sql_parser — don't collapse a macro-as-function-argument into a CTE#2456
Merged
Merged
Conversation
…on argument into a CTE The trailing-comma-CTE rule added in 03787df (#2454) matches any line that is solely `__macro__,`/`__undefined__,` and rewrites it into a synthetic CTE. That shape also occurs when a macro call is simply the first argument of a multi-line function call, e.g.: coalesce( {{ json_query_string('event_object', "'$.thread_id'") }}, regexp_extract(...) ) as thread_id Rewriting that placeholder as a CTE corrupts the argument list; sqlglot's error_level=IGNORE then silently degrades the whole statement to a stray `select 1`, so the model "parses" with no error but yields zero output columns — surfacing as a false-positive "columns in YAML but missing from SQL output" ERROR for every documented column. Distinguish the two shapes by the character immediately preceding the placeholder line: a genuine CTE-position macro always follows the closing `)` of the previous CTE, while a function-argument macro follows the call's opening `(` or a sibling argument's `,`. Found via tfact_chatbot_events.sql, which started failing `ol-dbt validate` only once it entered "changed models" scope on two unrelated open PRs (macro-file and dimensional-model changes that transitively touch models using json_query_string/iso8601_to_time_key) — the model's own SQL and YAML were untouched; the bug was latent on main all along, just never exercised against a model with this exact shape. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
🔎 ol-dbt impact — column-level blast radius✅ No column-level downstream impact detected for the changed models. Posted by |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes a false-positive rewrite in ol_dbt_cli’s Jinja rendering post-processing: a line that looks like a “trailing-comma macro placeholder CTE” is now only rewritten into a synthetic CTE when it is actually in a top-level CTE-list position, avoiding corruption of multi-line function argument lists (and the downstream “zero output columns” parsing degradation).
Changes:
- Refine the “
__macro__,/__undefined__,on its own line” rewrite to only fire when the preceding non-whitespace character indicates a CTE boundary (currently:)). - Add a regression test ensuring a macro used as the first argument of a multi-line function call is not treated as a CTE placeholder.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| src/ol_dbt_cli/ol_dbt_cli/lib/sql_parser.py | Tightens the trailing-comma macro rewrite by inspecting preceding context before rewriting to a synthetic CTE. |
| src/ol_dbt_cli/tests/test_sql_parser.py | Adds a regression test covering the macro-as-function-argument shape to prevent reintroducing the false-positive rewrite. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
__macro__,/__undefined__,line and rewrote it as a synthetic CTE, but that same shape also occurs when a macro call is the first argument of a multi-line function call (e.g.coalesce({{ macro(...) }}, ...)), corrupting the argument list and silently degrading the parse to zero output columns — a false-positive "columns missing from SQL" ERROR.)(end of previous CTE) → genuine CTE position;(or,(inside a call) → leave the placeholder alone.tfact_chatbot_events.sqlunexpectedly failingol-dbt validateon two unrelated open PRs (feat(dbt): add 18 missing starrocks__ cross-db macro variants #2417, fix(dimensional): fix dim_discount grain mismatch causing tfact_order fan-out #2411) once it entered "changed models" scope — the model's own SQL/YAML were untouched; the bug was latent onmainall along.Test plan
cd src/ol_dbt_cli && uv run pytest— 356 passed, no regressionstest_macro_as_first_function_argument_not_treated_as_cte— confirmed it fails without the fix and passes with itol-dbt validate --model tfact_chatbot_eventsclean on fresh checkouts of both affected PR branchesol-dbt validate(no filter) — 0 ERRORs on both affected branches🤖 Generated with Claude Code