feat(b2b_analytics): expose sso_organization_id on the mv_b2b_* views#2478
Open
blarghmatey wants to merge 2 commits into
Open
feat(b2b_analytics): expose sso_organization_id on the mv_b2b_* views#2478blarghmatey wants to merge 2 commits into
blarghmatey wants to merge 2 commits into
Conversation
The ol-analytics-api keys organizations on the Keycloak org UUID (sso_organization_id) -- the one identifier stable across the JWT, MITx Online, and StarRocks -- but the b2b_analytics MVs only carried organization_key (the mitx org_key, e.g. MIT504), so the API's `WHERE sso_organization_id = %s` filter had nothing to match. Add sso_organization_id (from dim_organization) to all six mv_b2b_* views: - The four that already join dim_organization gain one select + group-by column. - mv_b2b_monthly_engagement_trend and mv_b2b_content_engagement_depth read the raw organization_administration_report with no org join, so they get a LEFT JOIN to dim_organization on organization_key (platform='mitxonline'); it's null for free-text organization_keys that don't resolve to a known org, which is fine -- those orgs have no Keycloak identity and aren't reachable via the UUID path anyway. distributed_by stays organization_key (proven layout; changing distribution is a separate optimization). Column documented in the schema yml. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XKVcmKXkJMZciSWS9sfYFB
🔎 ol-dbt impact — column-level blast radius0 breaking, 0 warning, 6 info across 6 changed model(s). Details
Posted by |
Contributor
There was a problem hiding this comment.
Pull request overview
Expose sso_organization_id (Keycloak org UUID) on the b2b_analytics StarRocks materialized views so ol-analytics-api can key/filters organizations using a stable identifier across JWT, MITx Online, and StarRocks.
Changes:
- Added
sso_organization_idto the SELECT + GROUP BY of four existingmv_b2b_*views that already joindim_organization. - Added a
LEFT JOIN dim_organization(mitxonline-only) to twoorganization_administration_report-based views to exposesso_organization_idwhere resolvable. - Documented the new column in
src/ol_dbt/models/b2b_analytics/_b2b_analytics__models.yml.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/ol_dbt/models/b2b_analytics/mv_b2b_program_funnel.sql | Adds sso_organization_id to program funnel MV output and grouping. |
| src/ol_dbt/models/b2b_analytics/mv_b2b_monthly_engagement_trend.sql | Adds dim_organization join to expose sso_organization_id for monthly engagement rollups. |
| src/ol_dbt/models/b2b_analytics/mv_b2b_mit_admin_contract_health.sql | Adds sso_organization_id to MIT admin contract health MV output and grouping. |
| src/ol_dbt/models/b2b_analytics/mv_b2b_enrollment_completion_funnel.sql | Adds sso_organization_id to enrollment/completion funnel MV output and grouping. |
| src/ol_dbt/models/b2b_analytics/mv_b2b_contract_utilization.sql | Adds sso_organization_id to contract utilization MV output and grouping. |
| src/ol_dbt/models/b2b_analytics/mv_b2b_content_engagement_depth.sql | Adds dim_organization join to expose sso_organization_id for course_run engagement depth rollups. |
| src/ol_dbt/models/b2b_analytics/_b2b_analytics__models.yml | Documents sso_organization_id on all six MVs. |
Comments suppressed due to low confidence (1)
src/ol_dbt/models/b2b_analytics/_b2b_analytics__models.yml:101
- The description states this MV is filtered to a confirmed (non-fallback) organization_key, but the SQL only filters
organization_key is not null. Becausereporting.organization_administration_reportpopulatesorganization_keyviacoalesce(..., org_field.organization), fallback/free-text org keys can be present and will result in nullsso_organization_id. Update the description to match the actual behavior.
- name: sso_organization_id
description: >
Keycloak organization UUID (dim_organization.sso_organization_id) -- the
identifier the ol-analytics-api filters organizations on. Null for
organization_keys that don't resolve to a mitxonline org.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…rect docs Review feedback on #2478: - Sentry flagged the raw-OAR MVs' LEFT JOIN on organization_key as a potential fan-out. organization_key is unique per mitxonline org today (source-enforced unique constraint on OrganizationPage.org_key), but nothing in these models guaranteed it. Replace the direct join with a subquery that groups dim_organization to one row per organization_key (min(sso_organization_id)), so the join can never fan out and inflate aggregates even if that invariant regresses. Applies to mv_b2b_monthly_engagement_trend and mv_b2b_content_engagement_depth. - Copilot: the two model descriptions said "confirmed (non-fallback) organization_key" but the query only filters organization_key IS NOT NULL, which still admits free-text fallback values (they resolve to a null sso_organization_id). Corrected the descriptions. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XKVcmKXkJMZciSWS9sfYFB
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
ol-analytics-api is moving to key organizations on the Keycloak org UUID (
sso_organization_id) — the one identifier stable across the JWT, MITx Online, and StarRocks (see mitodl/ol-analytics-api#13 and the identifier analysis on mitodl/hq#11845). Theb2b_analyticsMVs only carriedorganization_key(the mitxorg_key, e.g.MIT504), so the API'sWHERE sso_organization_id = %sfilter had nothing to match.What
Add
sso_organization_id(fromdim_organization) to all sixmv_b2b_*views:dim_organization(org.…) — oneselect+group bycolumn each:mv_b2b_contract_utilization,mv_b2b_enrollment_completion_funnel,mv_b2b_program_funnel,mv_b2b_mit_admin_contract_health.organization_administration_reportwith no org join, so they gain aLEFT JOIN dim_organization ON organization_key AND platform='mitxonline'(columns qualified withoar.per convention):mv_b2b_monthly_engagement_trend,mv_b2b_content_engagement_depth.sso_organization_idis null for free-textorganization_keys that don't resolve to a known org — expected, since those orgs have no Keycloak identity and aren't reachable via the UUID path.distributed_bystaysorganization_key(proven layout; redistributing tosso_organization_idis a separate optimization). New column documented in_b2b_analytics__models.yml.Deploy note
Definition change, so the async MVs are recreated + re-refreshed by the Dagster
refresh_starrocks_analytics_mvsasset. Data-side half of the org-UUID work; ol-analytics-api #13 and the mitxonline manager-endpoint filter are the other two pieces.Validation
sqlflufflint + fix clean on all six models.dbt parse(starrocks target) clean; all sixmv_b2b_*manifest nodes carrysso_organization_id.🤖 Generated with Claude Code