From f2228ec544b7a3d5311980aaa53e4f69fb4dcbb0 Mon Sep 17 00:00:00 2001 From: Jamie Rodriguez <65564846+fivetran-jamie@users.noreply.github.com> Date: Tue, 24 Jun 2025 14:49:21 -0400 Subject: [PATCH] update freshness config --- .../maintainer_pull_request_template.md | 52 +++++++++++-------- CHANGELOG.md | 26 ++++++++++ LICENSE | 2 +- README.md | 3 +- dbt_project.yml | 2 +- integration_tests/dbt_project.yml | 2 +- models/src_reddit_ads.yml | 9 ++-- 7 files changed, 64 insertions(+), 32 deletions(-) diff --git a/.github/PULL_REQUEST_TEMPLATE/maintainer_pull_request_template.md b/.github/PULL_REQUEST_TEMPLATE/maintainer_pull_request_template.md index 3220674..99c450c 100644 --- a/.github/PULL_REQUEST_TEMPLATE/maintainer_pull_request_template.md +++ b/.github/PULL_REQUEST_TEMPLATE/maintainer_pull_request_template.md @@ -1,29 +1,35 @@ -## PR Overview -**This PR will address the following Issue/Feature:** + +- `dbt run --full-refresh && dbt test` +- `dbt run` && `dbt test` (if incremental models are present) +- The related issue is linked, tagged, and appropriately assigned +- Documentation and version updates are included, if applicable +- `docs` have been regenerated (unless there are no code or YAML changes) +- BuildKite integration tests are passing +--> -**Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:** - +## PR Overview +**Package version introduced in this PR:** + +**This PR addresses the following Issue/Feature(s):** + -## PR Checklist -### Basic Validation -Please acknowledge that you have successfully performed the following commands locally: -- [ ] dbt run –full-refresh && dbt test -- [ ] dbt run (if incremental models are present) && dbt test +**Summary of changes:** + -Before marking this PR as "ready for review" the following have been applied: -- [ ] The appropriate issue has been linked, tagged, and properly assigned -- [ ] All necessary documentation and version upgrades have been applied -- [ ] docs were regenerated (unless this PR does not include any code or yml updates) -- [ ] BuildKite integration tests are passing -- [ ] Detailed validation steps have been provided below -### Detailed Validation -Please share any and all of your validation steps: - +### Submission Checklist +- [ ] Alignment meeting with the reviewer (if needed) + - [ ] Timeline and validation requirements discussed +- [ ] Provide validation details: + - [ ] **Validation Steps:** Check for unintentional effects (e.g., add/run consistency & integrity tests) + - [ ] **Testing Instructions:** Confirm the change addresses the issue(s) + - [ ] **Focus Areas:** Complex logic or queries that need extra attention -### If you had to summarize this PR in an emoji, which would it be? - -:dancer: \ No newline at end of file +### Changelog + +- [ ] Draft changelog for PR +- [ ] Final changelog for release review \ No newline at end of file diff --git a/CHANGELOG.md b/CHANGELOG.md index 2df0694..0a6564c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,29 @@ +# dbt_reddit_ads_source v0.5.0 + +[PR #13](https://github.com/fivetran/dbt_reddit_ads_source/pull/13) includes the following updates: + +## Breaking Change for dbt Core < 1.9.6 +> *Note: This is not relevant to Fivetran Quickstart users.* + +Migrated `freshness` from a top-level source property to a source `config` in alignment with [recent updates](https://github.com/dbt-labs/dbt-core/issues/11506) from dbt Core. This will resolve the following deprecation warning that users running dbt >= 1.9.6 may have received: + +``` +[WARNING]: Deprecated functionality +Found `freshness` as a top-level property of `reddit_ads` in file +`models/src_reddit_ads.yml`. The `freshness` top-level property should be moved +into the `config` of `reddit_ads`. +``` + +**IMPORTANT:** Users running dbt Core < 1.9.6 will not be able to utilize freshness tests in this release or any subsequent releases, as older versions of dbt will not recognize freshness as a source `config` and therefore not run the tests. + +If you are using dbt Core < 1.9.6 and want to continue running Reddit Ads freshness tests, please elect **one** of the following options: + 1. (Recommended) Upgrade to dbt Core >= 1.9.6 + 2. Do not upgrade your installed version of the `reddit_ads_source` package. Pin your dependency on v0.4.1 in your `packages.yml` file. + 3. Utilize a dbt [override](https://docs.getdbt.com/reference/resource-properties/overrides) to overwrite the package's `reddit_ads` source and apply freshness via the [old](https://github.com/fivetran/dbt_reddit_ads_source/blob/v0.4.1/models/src_reddit_ads.yml#L11-L13) top-level property route. This will require you to copy and paste the entirety of the `src_reddit_ads.yml` [file](https://github.com/fivetran/dbt_reddit_ads_source/blob/v0.4.1/models/src_reddit_ads.yml#L4-L505) and add an `overrides: reddit_ads_source` property. + +## Under the Hood +- Updated the package maintainer PR template. + # dbt_reddit_ads_source v0.4.1 [PR #11](https://github.com/fivetran/dbt_reddit_ads_source/pull/11) includes the following updates: diff --git a/LICENSE b/LICENSE index 261eeb9..689e239 100644 --- a/LICENSE +++ b/LICENSE @@ -186,7 +186,7 @@ same "printed page" as the copyright notice for easier identification within third-party archives. - Copyright [yyyy] [name of copyright owner] + Copyright 2025 Fivetran, Inc. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. diff --git a/README.md b/README.md index a92eeeb..a140156 100644 --- a/README.md +++ b/README.md @@ -16,6 +16,7 @@ - Materializes [Reddit Ads staging tables](https://fivetran.github.io/dbt_reddit_ads_source/#!/overview/reddit_ads_source/models/?g_v=1&g_e=seeds) which leverage data in the format described by [this ERD](https://fivetran.com/docs/applications/reddit-ads#schemainformation). These staging tables clean, test, and prepare your reddit_ads data from [Fivetran's connector](https://fivetran.com/docs/applications/reddit-ads) for analysis by doing the following: - Naming the columns for consistency across all packages and for easier analysis - Adding freshness tests to source data + > dbt Core >= 1.9.6 is required to run freshness tests out of the box. See other options [here](https://github.com/fivetran/dbt_reddit_ads_source/blob/main/CHANGELOG.md#breaking-change-for-dbt-core--196). - Adding column-level testing where applicable. For example, all primary keys are tested for uniqueness and non-null values. - Generates a comprehensive data dictionary of your Reddit Ads data through the [dbt docs site](https://fivetran.github.io/dbt_reddit_ads_source/). - These tables are designed to work simultaneously with our [Reddit Ads transformation package](https://github.com/fivetran/dbt_reddit_ads). @@ -40,7 +41,7 @@ If you are _not_ using the [Reddit Ads](https://github.com/fivetran/dbt_reddit_a ```yaml packages: - package: fivetran/reddit_ads_source - version: [">=0.4.0", "<0.5.0"] + version: [">=0.5.0", "<0.6.0"] ``` ### Step 3: Define database and schema variables By default, this package runs using your destination and the `reddit_ads` schema. If this is not where your Reddit Ads data is (for example, if your `reddit_ads` schema is named `reddit_ads_fivetran`), add the following configuration to your root `dbt_project.yml` file: diff --git a/dbt_project.yml b/dbt_project.yml index 8282a60..dec3172 100644 --- a/dbt_project.yml +++ b/dbt_project.yml @@ -1,5 +1,5 @@ name: 'reddit_ads_source' -version: '0.4.1' +version: '0.5.0' config-version: 2 require-dbt-version: [">=1.3.0", "<2.0.0"] diff --git a/integration_tests/dbt_project.yml b/integration_tests/dbt_project.yml index 092ec5e..eb111a9 100644 --- a/integration_tests/dbt_project.yml +++ b/integration_tests/dbt_project.yml @@ -1,5 +1,5 @@ name: 'reddit_ads_source_integration_tests' -version: '0.4.1' +version: '0.5.0' profile: 'integration_tests' config-version: 2 diff --git a/models/src_reddit_ads.yml b/models/src_reddit_ads.yml index d0cfcee..72f8c78 100644 --- a/models/src_reddit_ads.yml +++ b/models/src_reddit_ads.yml @@ -8,13 +8,12 @@ sources: loader: Fivetran loaded_at_field: _fivetran_synced - freshness: - warn_after: {count: 48, period: hour} - error_after: {count: 168, period: hour} - config: enabled: "{{ var('ad_reporting__reddit_ads_enabled', true) }}" - + freshness: + warn_after: {count: 48, period: hour} + error_after: {count: 168, period: hour} + tables: - name: account identifier: "{{ var('reddit_ads_account_identifier', 'account') }}"