feat(dbt): add 18 missing starrocks__ cross-db macro variants by blarghmatey · Pull Request #2417 · mitodl/ol-data-platform

blarghmatey · 2026-07-09T18:02:33Z

What

Adds StarRocks dispatch variants for every cross-db macro that previously had only Trino/DuckDB implementations, so the dbt-starrocks adapter work has full macro coverage. Doc-verified against docs.starrocks.io (version floor 3.3.5).

cross_db_functions.sql: format_datetime, date_format, day_of_week, iso8601_to_date_key, iso8601_to_time_key, last_value_ignore_nulls, unnest_json_map, json_is_object, unnest_json_array, json_extract_varchar_array, is_courserun_current, from_iso8601_timestamp_nanos.

Standalone macro files: cast_date_to_iso8601, cast_timestamp_to_iso8601, date_diff, date_parse, json_extract_scalar, safe_parse_iso8601_date, generate_micromasters_program_readable_id.

Notable divergences (called out inline in the macros)

date_diff swaps argument order like DuckDB (StarRocks computes e1 - e2, not Trino's to - from).
day_of_week must use dayofweek_iso, not dayofweek (which is Sunday-based, not ISO).
generate_micromasters_program_readable_id uses concat() since || is logical OR in StarRocks, not string concatenation.
unnest_json_map: StarRocks' json_each() has fixed key/value output column names it cannot alias, unlike Trino/DuckDB's UNNEST. The new macro raises a compiler error unless callers pass key_col='key', val_col='value' literally. Updated the two call sites (dim_discussion_topic.sql, dim_problem.sql) to use val_col='value' and renamed their downstream t.topic/t.submission_data references to t.value — a no-op rename for Trino/DuckDB since the alias name there is arbitrary; verified via dbt compile that both models still produce identical SQL for the existing targets.

Not done here

Live-cluster validation against an actual StarRocks target is still needed — no starrocks profile target exists yet in profiles.yml, so these are doc-verified, not runtime-verified.

Part of #2375 (2026-07 dbt warehouse audit).

Closes #2396

🤖 Generated with Claude Code

Copilot

Pull request overview

Adds StarRocks adapter.dispatch variants for the project’s cross-database dbt macros so models can compile/run against a StarRocks target, and updates the two existing unnest_json_map call sites to use StarRocks’ fixed json_each() output column names.

Changes:

Added starrocks__* implementations for missing cross-db macros (date/time parsing/formatting, JSON helpers, unnest helpers, etc.).
Updated dim_discussion_topic and dim_problem to use unnest_json_map(..., key_col='key', val_col='value') and reference t.value.
Added StarRocks implementations for several standalone macros (e.g., date_diff, date_parse, ISO8601 casting).

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/ol_dbt/models/dimensional/dim_problem.sql	Updates JSON map unnest column naming (`t.value`) for StarRocks compatibility.
src/ol_dbt/models/dimensional/dim_discussion_topic.sql	Updates JSON map unnest column naming (`t.value`) for StarRocks compatibility.
src/ol_dbt/macros/safe_parse_iso8601_date.sql	Adds StarRocks-safe ISO8601 date parsing behavior.
src/ol_dbt/macros/json_extract_scalar.sql	Adds StarRocks JSON scalar extraction implementation.
src/ol_dbt/macros/generate_program_readable_id.sql	Adds StarRocks program readable-id generator using `concat()` etc.
src/ol_dbt/macros/date_parse.sql	Adds StarRocks `date_parse` variant via `str_to_date`.
src/ol_dbt/macros/date_diff.sql	Adds StarRocks `date_diff` variant with swapped argument order.
src/ol_dbt/macros/cross_db_functions.sql	Adds multiple StarRocks cross-db helpers (formatting, JSON helpers, unnesting, etc.).
src/ol_dbt/macros/cast_timestamp_to_iso8601.sql	Adds StarRocks timestamp→ISO8601 casting behavior.
src/ol_dbt/macros/cast_date_to_iso8601.sql	Adds StarRocks date→ISO8601 casting behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…xtraction Addresses Copilot PR review feedback on #2417. get_json_string() (the starrocks__json_query_string/json_extract_scalar implementation) requires a VARCHAR JSON string argument. Two of its callers can pass a JSON-typed value on StarRocks: - starrocks__json_extract_scalar's json_col can be a JSON-typed element produced by starrocks__unnest_json_array or starrocks__unnest_json_map. - dim_discussion_topic.sql and dim_problem.sql pass t.value from unnest_json_map(), which on StarRocks is json_each()'s JSON-typed value column. Cast to varchar at both points; a no-op for Trino/DuckDB where the value is already text. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

github-actions · 2026-07-11T00:56:42Z

🔎 ol-dbt impact — column-level blast radius

0 breaking, 6 warning, 2 info across 8 changed model(s).

Details

⚠️ macro: cast_date_to_iso8601.sql → 6 downstream model(s)
⚠️ macro: cast_timestamp_to_iso8601.sql → 157 downstream model(s)
⚠️ macro: cross_db_functions.sql → 67 downstream model(s)
⚠️ macro: generate_program_readable_id.sql → 3 downstream model(s)
⚠️ macro: json_extract_scalar.sql → 1 downstream model(s)
⚠️ macro: safe_parse_iso8601_date.sql → 1 downstream model(s)
ℹ️ macro: date_diff.sql → 0 downstream model(s)
ℹ️ macro: date_parse.sql → 0 downstream model(s)

_{Posted by ol-dbt impact (annotate-only — does not block merge).}

…xtraction Addresses Copilot PR review feedback on #2417. get_json_string() (the starrocks__json_query_string/json_extract_scalar implementation) requires a VARCHAR JSON string argument. Two of its callers can pass a JSON-typed value on StarRocks: - starrocks__json_extract_scalar's json_col can be a JSON-typed element produced by starrocks__unnest_json_array or starrocks__unnest_json_map. - dim_discussion_topic.sql and dim_problem.sql pass t.value from unnest_json_map(), which on StarRocks is json_each()'s JSON-typed value column. Cast to varchar at both points; a no-op for Trino/DuckDB where the value is already text. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Implements StarRocks dispatch variants for every dbt macro that previously had only Trino/DuckDB implementations, verified against docs.starrocks.io (version floor 3.3.5): format_datetime, date_format, day_of_week, iso8601_to_date_key, iso8601_to_time_key, last_value_ignore_nulls, unnest_json_map, json_is_object, unnest_json_array, json_extract_varchar_array, is_courserun_current, from_iso8601_timestamp_nanos (cross_db_functions.sql), plus cast_date_to_iso8601, cast_timestamp_to_iso8601, date_diff, date_parse, json_extract_scalar, safe_parse_iso8601_date, and generate_micromasters_program_readable_id in their standalone files. Notable divergences from the Trino/DuckDB bodies, all called out inline: - date_diff swaps argument order like DuckDB (StarRocks computes e1-e2). - day_of_week must use dayofweek_iso, not dayofweek (which is Sunday-based). - generate_micromasters_program_readable_id uses concat() since || is logical OR in StarRocks, not string concatenation. - unnest_json_map: StarRocks' json_each() has fixed 'key'/'value' output column names it cannot alias, unlike Trino/DuckDB's UNNEST. The new macro raises a compiler error unless callers pass key_col='key', val_col='value' literally. Updated the two call sites (dim_discussion_topic.sql, dim_problem.sql) to use val_col='value' and renamed their downstream t.topic/t.submission_data references to t.value accordingly -- a no-op rename for Trino/DuckDB since the alias name there is arbitrary. Live-cluster validation against an actual StarRocks target is still needed (no starrocks profile target exists yet) -- these are doc-verified, not runtime-verified, per the tracking task. Part of #2375 (2026-07 dbt warehouse audit). Witan task: tk-implement-18-missing-starrocks-macro-variants-do-57059b. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…xtraction Addresses Copilot PR review feedback on #2417. get_json_string() (the starrocks__json_query_string/json_extract_scalar implementation) requires a VARCHAR JSON string argument. Two of its callers can pass a JSON-typed value on StarRocks: - starrocks__json_extract_scalar's json_col can be a JSON-typed element produced by starrocks__unnest_json_array or starrocks__unnest_json_map. - dim_discussion_topic.sql and dim_problem.sql pass t.value from unnest_json_map(), which on StarRocks is json_each()'s JSON-typed value column. Cast to varchar at both points; a no-op for Trino/DuckDB where the value is already text. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…alue macro Sentry bug-prediction flagged dim_discussion_topic.sql and dim_problem.sql using raw json_extract() calls, which do not exist as a function on StarRocks (confirmed by starrocks__json_extract_value's own comment: "json_extract does not exist; use json_query on a parsed JSON value") and would fail at StarRocks compile/runtime. Replace all 9 raw json_extract() call sites (1 in dim_discussion_topic.sql feeding unnest_json_map, 8 in dim_problem.sql — 4x the submission column extraction, 4x nested in json_is_object) with the existing json_extract_value dispatch macro. Verified per-engine: - DuckDB: compiles to the identical json_extract(...) call as before (true no-op, confirmed via `dbt compile -t dev_local`). - Trino: json_extract_value dispatches to json_query(...) returning a JSON value (per its own doc comment), matching what json_is_object's Trino path (json_format(json_expr)) expects. - StarRocks: json_extract_value dispatches to json_query(parse_json(...)), which is the fix — the raw json_extract() call this replaces does not exist on StarRocks at all. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Copilot AI review requested due to automatic review settings July 9, 2026 18:02

Copilot started reviewing on behalf of blarghmatey July 9, 2026 18:03 View session

Copilot AI reviewed Jul 9, 2026

View reviewed changes

Comment thread src/ol_dbt/macros/json_extract_scalar.sql

Comment thread src/ol_dbt/models/dimensional/dim_discussion_topic.sql Outdated

Comment thread src/ol_dbt/models/dimensional/dim_problem.sql

Comment thread src/ol_dbt/models/dimensional/dim_problem.sql Outdated

blarghmatey force-pushed the fix/starrocks-18-missing-macro-variants branch from 19e24ee to a8a43c9 Compare July 11, 2026 00:55

blarghmatey force-pushed the fix/starrocks-18-missing-macro-variants branch from a8a43c9 to 73949f4 Compare July 16, 2026 14:06

sentry Bot reviewed Jul 16, 2026

View reviewed changes

Comment thread src/ol_dbt/models/dimensional/dim_discussion_topic.sql Outdated

blarghmatey mentioned this pull request Jul 16, 2026

fix(ol_dbt_cli): sql_parser — don't collapse a macro-as-function-argument into a CTE #2456

Merged

5 tasks

blarghmatey and others added 3 commits July 16, 2026 11:26

blarghmatey force-pushed the fix/starrocks-18-missing-macro-variants branch from 2e79950 to 2b9d0da Compare July 16, 2026 15:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(dbt): add 18 missing starrocks__ cross-db macro variants#2417

feat(dbt): add 18 missing starrocks__ cross-db macro variants#2417
blarghmatey wants to merge 3 commits into
mainfrom
fix/starrocks-18-missing-macro-variants

blarghmatey commented Jul 9, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jul 11, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

blarghmatey commented Jul 9, 2026

What

Notable divergences (called out inline in the macros)

Not done here

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jul 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔎 ol-dbt impact — column-level blast radius

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot commented Jul 11, 2026 •

edited

Loading