Skip to content

feat: add experiment SRM values as view#1427

Merged
mikewilli merged 3 commits intomainfrom
experiment-srm-view
Apr 8, 2026
Merged

feat: add experiment SRM values as view#1427
mikewilli merged 3 commits intomainfrom
experiment-srm-view

Conversation

@mikewilli
Copy link
Copy Markdown
Contributor

Follow-up to mozilla/bigquery-etl#9167, which added a new table to precompute SRM for all experiments.

We can follow this by updating the SRMs on the experiment enrollments monitoring dashboard to use this view.

Copy link
Copy Markdown
Contributor

@scholtzan scholtzan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add the explore for it as part of this PR too?

@mikewilli
Copy link
Copy Markdown
Contributor Author

Should we add the explore for it as part of this PR too?

Do we need an explore for it? I wasn't sure because none of the other tables have one here, except for experiment_crash_rates.

@scholtzan
Copy link
Copy Markdown
Contributor

I guess it depends how it will be used. If we want to add some graphs to a dashboard based on this table then it might be good to have an explore. But if it's just used for joining a view alone is fine

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 8, 2026

Integration report for "this commit"

lookml.diff

Click to expand!
Only in /tmp/workspace/generated-lookml/experimentation/datagroups: experiment_sample_ratio_mismatch_last_updated.datagroup.lkml
Only in /tmp/workspace/generated-lookml/experimentation/explores: experiment_sample_ratio_mismatch.explore.lkml
Only in /tmp/workspace/generated-lookml/experimentation/views: experiment_sample_ratio_mismatch.view.lkml
Only in /tmp/workspace/looker-hub/firefox_ios: dashboards
Only in /tmp/workspace/looker-hub/marketing: datagroups
Only in /tmp/workspace/looker-hub/search_private/datagroups: search_term_data_validation_reports_last_updated.datagroup.lkml
Only in /tmp/workspace/looker-hub/search_private/datagroups: search_term_sanitization_job_metadata_daily_last_updated.datagroup.lkml
Only in /tmp/workspace/looker-hub/: user_journey
diff -x '.*' -bur --no-dereference --new-file /tmp/workspace/looker-hub/experimentation/datagroups/experiment_sample_ratio_mismatch_last_updated.datagroup.lkml /tmp/workspace/generated-lookml/experimentation/datagroups/experiment_sample_ratio_mismatch_last_updated.datagroup.lkml
--- /tmp/workspace/looker-hub/experimentation/datagroups/experiment_sample_ratio_mismatch_last_updated.datagroup.lkml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-lookml/experimentation/datagroups/experiment_sample_ratio_mismatch_last_updated.datagroup.lkml	2026-04-08 20:52:56.008262432 +0000
@@ -0,0 +1,20 @@
+# *Do not manually modify this file*
+
+# This file has been generated via https://github.com/mozilla/lookml-generator
+
+# Using a datagroup in an Explore: https://cloud.google.com/looker/docs/reference/param-explore-persist-with
+# Using a datagroup in a derived table: https://cloud.google.com/looker/docs/reference/param-view-datagroup-trigger
+
+datagroup: experiment_sample_ratio_mismatch_last_updated {
+  label: "experiment_sample_ratio_mismatch Last Updated"
+  sql_trigger: SELECT MAX(storage_last_modified_time)
+    FROM (
+        
+    SELECT MAX(storage_last_modified_time) AS storage_last_modified_time
+    FROM `moz-fx-data-experiments`.`region-us`.INFORMATION_SCHEMA.TABLE_STORAGE
+    WHERE (table_schema = 'monitoring' AND table_name = 'experiment_sample_ratio_mismatch_v1')
+
+    ) ;;
+  description: "Updates for experiment_sample_ratio_mismatch when referenced tables are modified."
+  max_cache_age: "24 hours"
+}
\ No newline at end of file
diff -x '.*' -bur --no-dereference --new-file /tmp/workspace/looker-hub/experimentation/explores/experiment_sample_ratio_mismatch.explore.lkml /tmp/workspace/generated-lookml/experimentation/explores/experiment_sample_ratio_mismatch.explore.lkml
--- /tmp/workspace/looker-hub/experimentation/explores/experiment_sample_ratio_mismatch.explore.lkml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-lookml/experimentation/explores/experiment_sample_ratio_mismatch.explore.lkml	2026-04-08 20:52:56.009262430 +0000
@@ -0,0 +1,13 @@
+
+# *Do not manually modify this file*
+#
+# This file has been generated via https://github.com/mozilla/lookml-generator
+# You can extend this view in the looker-spoke-default project (https://github.com/mozilla/looker-spoke-default)
+
+include: "/looker-hub/experimentation/views/experiment_sample_ratio_mismatch.view.lkml"
+include: "/looker-hub/experimentation/datagroups/experiment_sample_ratio_mismatch_last_updated.datagroup.lkml"
+
+explore: experiment_sample_ratio_mismatch {
+  view_name: experiment_sample_ratio_mismatch
+  persist_with: experiment_sample_ratio_mismatch_last_updated
+}
\ No newline at end of file
diff -x '.*' -bur --no-dereference --new-file /tmp/workspace/looker-hub/experimentation/views/experiment_sample_ratio_mismatch.view.lkml /tmp/workspace/generated-lookml/experimentation/views/experiment_sample_ratio_mismatch.view.lkml
--- /tmp/workspace/looker-hub/experimentation/views/experiment_sample_ratio_mismatch.view.lkml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-lookml/experimentation/views/experiment_sample_ratio_mismatch.view.lkml	2026-04-08 20:52:56.009262430 +0000
@@ -0,0 +1,79 @@
+
+# *Do not manually modify this file*
+#
+# This file has been generated via https://github.com/mozilla/lookml-generator
+# You can extend this view in the looker-spoke-default project (https://github.com/mozilla/looker-spoke-default)
+
+view: experiment_sample_ratio_mismatch {
+  dimension: dau_p_value {
+    sql: ${TABLE}.dau_p_value ;;
+    type: number
+    suggest_persist_for: "24 hours"
+    description: "P-value of Sample Ratio Mismatch (SRM) for DAU."
+  }
+
+  dimension: dau_probability {
+    sql: ${TABLE}.dau_probability ;;
+    type: number
+    suggest_persist_for: "24 hours"
+    description: "Probability of Sample Ratio Mismatch (SRM) for DAU."
+  }
+
+  dimension: dau_srm_likely {
+    sql: ${TABLE}.dau_srm_likely ;;
+    type: yesno
+    suggest_persist_for: "24 hours"
+    description: "Whether Sample Ratio Mismatch (SRM) is likely (True if p-value < 0.1)."
+  }
+
+  dimension: enrollments_p_value {
+    sql: ${TABLE}.enrollments_p_value ;;
+    type: number
+    suggest_persist_for: "24 hours"
+    description: "P-value of Sample Ratio Mismatch (SRM) for enrollments."
+  }
+
+  dimension: enrollments_probability {
+    sql: ${TABLE}.enrollments_probability ;;
+    type: number
+    suggest_persist_for: "24 hours"
+    description: "Probability of Sample Ratio Mismatch (SRM) for enrollments."
+  }
+
+  dimension: enrollments_srm_likely {
+    sql: ${TABLE}.enrollments_srm_likely ;;
+    type: yesno
+    suggest_persist_for: "24 hours"
+    description: "Whether Sample Ratio Mismatch (SRM) is likely (True if p-value < 0.1)."
+  }
+
+  dimension: experiment {
+    sql: ${TABLE}.experiment ;;
+    type: string
+    suggest_persist_for: "24 hours"
+    description: "Nimbus experiment (or rollout) slug."
+  }
+
+  dimension: unenrollments_p_value {
+    sql: ${TABLE}.unenrollments_p_value ;;
+    type: number
+    suggest_persist_for: "24 hours"
+    description: "P-value of Sample Ratio Mismatch (SRM) for unenrollments."
+  }
+
+  dimension: unenrollments_probability {
+    sql: ${TABLE}.unenrollments_probability ;;
+    type: number
+    suggest_persist_for: "24 hours"
+    description: "Probability of Sample Ratio Mismatch (SRM) for unenrollments."
+  }
+
+  dimension: unenrollments_srm_likely {
+    sql: ${TABLE}.unenrollments_srm_likely ;;
+    type: yesno
+    suggest_persist_for: "24 hours"
+    description: "Whether Sample Ratio Mismatch (SRM) is likely (True if p-value < 0.1)."
+  }
+
+  sql_table_name: `moz-fx-data-experiments.monitoring.experiment_sample_ratio_mismatch_v1` ;;
+}
\ No newline at end of file
diff -x '.*' -bur --no-dereference --new-file /tmp/workspace/looker-hub/marketing/datagroups/firefox_for_families_forecasted_expectations_last_updated.datagroup.lkml /tmp/workspace/generated-lookml/marketing/datagroups/firefox_for_families_forecasted_expectations_last_updated.datagroup.lkml
--- /tmp/workspace/looker-hub/marketing/datagroups/firefox_for_families_forecasted_expectations_last_updated.datagroup.lkml	2026-04-08 20:53:00.988952336 +0000
+++ /tmp/workspace/generated-lookml/marketing/datagroups/firefox_for_families_forecasted_expectations_last_updated.datagroup.lkml	1970-01-01 00:00:00.000000000 +0000
@@ -1,20 +0,0 @@
-# *Do not manually modify this file*
-
-# This file has been generated via https://github.com/mozilla/lookml-generator
-
-# Using a datagroup in an Explore: https://cloud.google.com/looker/docs/reference/param-explore-persist-with
-# Using a datagroup in a derived table: https://cloud.google.com/looker/docs/reference/param-view-datagroup-trigger
-
-datagroup: firefox_for_families_forecasted_expectations_last_updated {
-  label: "firefox_for_families_forecasted_expectations Last Updated"
-  sql_trigger: SELECT MAX(storage_last_modified_time)
-    FROM (
-        
-    SELECT MAX(storage_last_modified_time) AS storage_last_modified_time
-    FROM `mozdata`.`region-us`.INFORMATION_SCHEMA.TABLE_STORAGE
-    WHERE (table_schema = 'analysis' AND table_name = 'marketing_firefox_for_families_forecasted_expectations')
-
-    ) ;;
-  description: "Updates for firefox_for_families_forecasted_expectations when referenced tables are modified."
-  max_cache_age: "24 hours"
-}
\ No newline at end of file
diff -x '.*' -bur --no-dereference --new-file /tmp/workspace/looker-hub/monitoring/views/bigquery_table_storage.view.lkml /tmp/workspace/generated-lookml/monitoring/views/bigquery_table_storage.view.lkml
--- /tmp/workspace/looker-hub/monitoring/views/bigquery_table_storage.view.lkml	2026-04-08 20:53:00.994747457 +0000
+++ /tmp/workspace/generated-lookml/monitoring/views/bigquery_table_storage.view.lkml	2026-04-08 20:52:56.586261396 +0000
@@ -9,84 +9,72 @@
     sql: ${TABLE}.active_logical_bytes ;;
     type: number
     suggest_persist_for: "24 hours"
-    description: "Number of uncompressed bytes less than 90 days old"
   }
 
   dimension: active_physical_bytes {
     sql: ${TABLE}.active_physical_bytes ;;
     type: number
     suggest_persist_for: "24 hours"
-    description: "Number of compressed bytes less than 90 days old"
   }
 
   dimension: dataset_id {
     sql: ${TABLE}.dataset_id ;;
     type: string
     suggest_persist_for: "24 hours"
-    description: "The name of the dataset that contains the table"
   }
 
   dimension: long_term_logical_bytes {
     sql: ${TABLE}.long_term_logical_bytes ;;
     type: number
     suggest_persist_for: "24 hours"
-    description: "Number of uncompressed bytes more than 90 days old"
   }
 
   dimension: long_term_physical_bytes {
     sql: ${TABLE}.long_term_physical_bytes ;;
     type: number
     suggest_persist_for: "24 hours"
-    description: "Number of compressed bytes more than 90 days old"
   }
 
   dimension: project_id {
     sql: ${TABLE}.project_id ;;
     type: string
     suggest_persist_for: "24 hours"
-    description: "The ID of the project that contains the dataset"
   }
 
   dimension: table_id {
     sql: ${TABLE}.table_id ;;
     type: string
     suggest_persist_for: "24 hours"
-    description: "The name of the table"
   }
 
   dimension: time_travel_physical_bytes {
     sql: ${TABLE}.time_travel_physical_bytes ;;
     type: number
     suggest_persist_for: "24 hours"
-    description: "Number of compressed bytes for deleted or changed data"
   }
 
   dimension: total_logical_bytes {
     sql: ${TABLE}.total_logical_bytes ;;
     type: number
     suggest_persist_for: "24 hours"
-    description: "Total number of uncompressed bytes in the table"
   }
 
   dimension: total_partitions {
     sql: ${TABLE}.total_partitions ;;
     type: number
     suggest_persist_for: "24 hours"
-    description: "The total number of partitions present in the table"
   }
 
   dimension: total_physical_bytes {
     sql: ${TABLE}.total_physical_bytes ;;
     type: number
     suggest_persist_for: "24 hours"
-    description: "Total number of compressed bytes used for storage"
   }
 
   dimension: total_rows {
     sql: ${TABLE}.total_rows ;;
     type: number
     suggest_persist_for: "24 hours"
-    description: "The total number of rows in the table"
   }
 
   dimension_group: creation {
@@ -102,7 +90,6 @@
     ]
     convert_tz: no
     datatype: date
-    description: "The table's creation date"
   }
 
   sql_table_name: `moz-fx-data-shared-prod.monitoring.bigquery_table_storage` ;;
diff -x '.*' -bur --no-dereference --new-file /tmp/workspace/looker-hub/namespaces.yaml /tmp/workspace/generated-lookml/namespaces.yaml
--- /tmp/workspace/looker-hub/namespaces.yaml	2026-04-08 20:53:01.433275277 +0000
+++ /tmp/workspace/generated-lookml/namespaces.yaml	2026-04-08 20:53:01.364274395 +0000
@@ -1086,6 +1086,10 @@
       type: table_explore
       views:
         base_view: experiment_crash_rates
+    experiment_sample_ratio_mismatch:
+      type: table_explore
+      views:
+        base_view: experiment_sample_ratio_mismatch
   glean_app: false
   owners:
     - ascholtz@mozilla.com
@@ -1151,6 +1155,11 @@
         - channel: release
           table: mozdata.telemetry.experiment_enrollment_overall_v2
       type: table_view
+    experiment_sample_ratio_mismatch:
+      tables:
+        - channel: release
+          table: moz-fx-data-experiments.monitoring.experiment_sample_ratio_mismatch_v1
+      type: table_view
     experiment_search_aggregates_live:
       tables:
         - channel: release
diff -x '.*' -bur --no-dereference --new-file /tmp/workspace/looker-hub/search_private/datagroups/search_term_data_validation_reports_last_updated.datagroup.lkml /tmp/workspace/generated-lookml/search_private/datagroups/search_term_data_validation_reports_last_updated.datagroup.lkml
--- /tmp/workspace/looker-hub/search_private/datagroups/search_term_data_validation_reports_last_updated.datagroup.lkml	2026-04-08 20:53:01.042811243 +0000
+++ /tmp/workspace/generated-lookml/search_private/datagroups/search_term_data_validation_reports_last_updated.datagroup.lkml	1970-01-01 00:00:00.000000000 +0000
@@ -1,20 +0,0 @@
-# *Do not manually modify this file*
-
-# This file has been generated via https://github.com/mozilla/lookml-generator
-
-# Using a datagroup in an Explore: https://cloud.google.com/looker/docs/reference/param-explore-persist-with
-# Using a datagroup in a derived table: https://cloud.google.com/looker/docs/reference/param-view-datagroup-trigger
-
-datagroup: search_term_data_validation_reports_last_updated {
-  label: "search_term_data_validation_reports Last Updated"
-  sql_trigger: SELECT MAX(storage_last_modified_time)
-    FROM (
-        
-    SELECT MAX(storage_last_modified_time) AS storage_last_modified_time
-    FROM `moz-fx-data-shared-prod`.`region-us`.INFORMATION_SCHEMA.TABLE_STORAGE
-    WHERE (table_schema = 'search_terms_derived' AND table_name = 'search_term_data_validation_reports_v1')
-
-    ) ;;
-  description: "Updates for search_term_data_validation_reports when referenced tables are modified."
-  max_cache_age: "24 hours"
-}
\ No newline at end of file
diff -x '.*' -bur --no-dereference --new-file /tmp/workspace/looker-hub/search_private/datagroups/search_term_sanitization_job_metadata_daily_last_updated.datagroup.lkml /tmp/workspace/generated-lookml/search_private/datagroups/search_term_sanitization_job_metadata_daily_last_updated.datagroup.lkml
--- /tmp/workspace/looker-hub/search_private/datagroups/search_term_sanitization_job_metadata_daily_last_updated.datagroup.lkml	2026-04-08 20:53:01.042843103 +0000
+++ /tmp/workspace/generated-lookml/search_private/datagroups/search_term_sanitization_job_metadata_daily_last_updated.datagroup.lkml	1970-01-01 00:00:00.000000000 +0000
@@ -1,20 +0,0 @@
-# *Do not manually modify this file*
-
-# This file has been generated via https://github.com/mozilla/lookml-generator
-
-# Using a datagroup in an Explore: https://cloud.google.com/looker/docs/reference/param-explore-persist-with
-# Using a datagroup in a derived table: https://cloud.google.com/looker/docs/reference/param-view-datagroup-trigger
-
-datagroup: search_term_sanitization_job_metadata_daily_last_updated {
-  label: "search_term_sanitization_job_metadata_daily Last Updated"
-  sql_trigger: SELECT MAX(storage_last_modified_time)
-    FROM (
-        
-    SELECT MAX(storage_last_modified_time) AS storage_last_modified_time
-    FROM `moz-fx-data-shared-prod`.`region-us`.INFORMATION_SCHEMA.TABLE_STORAGE
-    WHERE (table_schema = 'search_terms_derived' AND table_name = 'sanitization_job_metadata_v2')
-
-    ) ;;
-  description: "Updates for search_term_sanitization_job_metadata_daily when referenced tables are modified."
-  max_cache_age: "24 hours"
-}
\ No newline at end of file
diff -x '.*' -bur --no-dereference --new-file /tmp/workspace/looker-hub/user_journey/views/events_daily.view.lkml /tmp/workspace/generated-lookml/user_journey/views/events_daily.view.lkml
--- /tmp/workspace/looker-hub/user_journey/views/events_daily.view.lkml	2026-04-08 20:53:01.047050860 +0000
+++ /tmp/workspace/generated-lookml/user_journey/views/events_daily.view.lkml	1970-01-01 00:00:00.000000000 +0000
@@ -1,71 +0,0 @@
-view: events_daily {
-  sql_table_name: `mozdata.messaging_system.events_daily`
-    ;;
-
-  dimension: addon_version {
-    type: string
-    sql: ${TABLE}.addon_version ;;
-  }
-
-  dimension: app_version {
-    type: string
-    sql: ${TABLE}.app_version ;;
-  }
-
-  dimension: city {
-    type: string
-    sql: ${TABLE}.city ;;
-  }
-
-  dimension: client_id {
-    type: string
-    sql: ${TABLE}.client_id ;;
-  }
-
-  dimension: country {
-    type: string
-    map_layer_name: countries
-    sql: ${TABLE}.country ;;
-  }
-
-  dimension: events {
-    type: string
-    sql: ${TABLE}.events ;;
-    hidden: yes
-  }
-
-  dimension: experiments {
-    hidden: yes
-    sql: ${TABLE}.experiments ;;
-  }
-
-  dimension: locale {
-    type: string
-    sql: ${TABLE}.locale ;;
-  }
-
-  dimension: normalized_channel {
-    type: string
-    sql: ${TABLE}.normalized_channel ;;
-  }
-
-  dimension: os {
-    type: string
-    sql: ${TABLE}.os ;;
-  }
-
-  dimension: sample_id {
-    type: number
-    sql: ${TABLE}.sample_id ;;
-  }
-
-  dimension: subdivision1 {
-    type: string
-    sql: ${TABLE}.subdivision1 ;;
-  }
-
-  dimension: submission_date {
-    type: date
-    sql: CAST(${TABLE}.submission_date AS TIMESTAMP) ;;
-  }
-}

Link to full diff

@mikewilli mikewilli added this pull request to the merge queue Apr 8, 2026
Merged via the queue into main with commit fe06a21 Apr 8, 2026
18 checks passed
@mikewilli mikewilli deleted the experiment-srm-view branch April 8, 2026 21:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants