fix(dimensional): platform-scope MicroMasters program_fk lookup in tfact_enrollment#2412
fix(dimensional): platform-scope MicroMasters program_fk lookup in tfact_enrollment#2412blarghmatey wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR fixes a dimensional-layer correctness issue in tfact_enrollment where MicroMasters program_fk enrichment could leak across platforms by joining on courserun_readable_id alone. It aligns the model behavior with the platform-pairing rule described in the issue/PR description and updates the schema docs to reflect the intended enrichment behavior.
Changes:
- Platform-scopes the MicroMasters program lookup by joining on
(courserun_readable_id, platform)rather thancourserun_readable_idalone. - Pre-dedupes the MicroMasters lookup to a single row per
(courserun_readable_id, platform)usingrow_number()(Trino-compatible pattern). - Updates
_fact_tables.ymldocumentation forprogram_fkto include the MicroMasters enrichment behavior for edxorg/mitxonline course enrollments.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| src/ol_dbt/models/dimensional/tfact_enrollment.sql | Adds platform-aware MicroMasters lookup + Trino-friendly pre-dedup and joins it on both courserun and platform to prevent cross-platform leakage/fanout. |
| src/ol_dbt/models/dimensional/_fact_tables.yml | Updates program_fk column documentation to reflect MicroMasters-based population for edxorg/mitxonline course enrollments. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
c2e6da6 to
e3393e5
Compare
🔎 ol-dbt impact — column-level blast radius✅ No column-level downstream impact detected for the changed models. Posted by |
…act_enrollment micromasters_program_lookup joined enrollments_with_fks on courserun_readable_id alone, so any platform's enrollment sharing that readable_id (not just the edxorg/mitxonline enrollments MM courses are actually hosted on) could inherit a MicroMasters program_fk, and a course run reused across two MM programs would fan out the join undetected. Platform-scope the join using the enrollments' own platform values, and dedupe to one row per (courserun_readable_id, platform). _fact_tables.yml documented program_fk as null for course enrollments except the mitxpro program-purchase case, silently omitting the MM enrichment path the code already relied on — document it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
e3393e5 to
1fbb2f3
Compare
rachellougee
left a comment
There was a problem hiding this comment.
LGTM. The changes are mostly preventive/defensive - I verified against production: no enrollment currently has a cross-platform MM program_fk, and no course run currently maps to two MM programs. So no full refresh needed.
What are the relevant tickets?
Closes #2378
Description (What does it do?)
tfact_enrollment.sql'smicromasters_program_lookupCTE enriched course-scope enrollments with a MicroMastersprogram_fkby joining oncourserun_readable_idalone, with no platform constraint. That meant:courserun_readable_id(not just the edxorg/mitxonline enrollments that MM courses actually run on) could inherit a MicroMastersprogram_fk._fact_tables.ymldocumentedprogram_fkas null for course enrollments except the mitxpro program-purchase case, contradicting the MM enrichment behavior the code already relied on.Fix:
platformvalue (mapped from the sourceedX.org/MITx Onlinestrings to theedxorg/mitxonlinecodes used elsewhere in the model) and joining on(courserun_readable_id, platform)instead ofcourserun_readable_idalone.micromasters_program_lookupto one row per(courserun_readable_id, platform)viarow_number()(Trino has noQUALIFY, consistent with the existing dedup pattern later in the same model) so a course run belonging to two MM programs can't fan out the enrollment it's joined against.program_fkcolumn description in_fact_tables.ymlto document the MicroMasters enrichment path.How can this be tested?
dbt parseanddbt compile -s tfact_enrollmentboth succeed against the local duckdb target; the compiled SQL confirms the Jinjavar()calls resolve to the correct source platform strings (edX.org,MITx Online) used in thecasemapping.relationshipstest onprogram_fk→dim_program.program_pkand thedbt_utils.unique_combination_of_columnstest on(enrollment_id, platform, enrollment_type)continue to guard the model's grain; a full run against the QA warehouse is recommended before merge to confirm no unexpectedprogram_fkpopulation changes for edge-case platforms.Additional Context
Found via the dbt warehouse semantic-accuracy audit (dimensional-layer correctness epic). Related identity-collapse issue in the same audit: MicroMasters certificates in
tfact_certificatealso dual-source from edxorg with a separate defensive-dedup gap (not addressed by this PR).