diff --git a/.github/PULL_REQUEST_TEMPLATE/maintainer_pull_request_template.md b/.github/PULL_REQUEST_TEMPLATE/maintainer_pull_request_template.md index 3220674..0899dc6 100644 --- a/.github/PULL_REQUEST_TEMPLATE/maintainer_pull_request_template.md +++ b/.github/PULL_REQUEST_TEMPLATE/maintainer_pull_request_template.md @@ -1,29 +1,38 @@ -## PR Overview -**This PR will address the following Issue/Feature:** + +- `dbt run --full-refresh && dbt test` +- `dbt run` && `dbt test` (if incremental models are present) +- The related issue is linked, tagged, and appropriately assigned +- Documentation and version updates are included, if applicable +- `docs` have been regenerated (unless there are no code or YAML changes) +- BuildKite integration tests are passing +--> -**Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:** - +## PR Overview +**Package version introduced in this PR:** +- -## PR Checklist -### Basic Validation -Please acknowledge that you have successfully performed the following commands locally: -- [ ] dbt run –full-refresh && dbt test -- [ ] dbt run (if incremental models are present) && dbt test +**This PR addresses the following Issue/Feature(s):** + +- -Before marking this PR as "ready for review" the following have been applied: -- [ ] The appropriate issue has been linked, tagged, and properly assigned -- [ ] All necessary documentation and version upgrades have been applied -- [ ] docs were regenerated (unless this PR does not include any code or yml updates) -- [ ] BuildKite integration tests are passing -- [ ] Detailed validation steps have been provided below +**Summary of changes:** + +- -### Detailed Validation -Please share any and all of your validation steps: - +### Submission Checklist +- [ ] Alignment meeting with the reviewer (if needed) + - [ ] Timeline and validation requirements discussed +- [ ] Provide validation details: + - [ ] **Validation Steps:** Check for unintentional effects (e.g., add/run consistency & integrity tests) + - [ ] **Testing Instructions:** Confirm the change addresses the issue(s) + - [ ] **Focus Areas:** Complex logic or queries that need extra attention +- [ ] Merge any relevant open PRs into this PR -### If you had to summarize this PR in an emoji, which would it be? - -:dancer: \ No newline at end of file +### Changelog + +- [ ] Draft changelog for PR +- [ ] Final changelog for release review diff --git a/.github/workflows/auto-release.yml b/.github/workflows/auto-release.yml index 8ed5853..e0a7100 100644 --- a/.github/workflows/auto-release.yml +++ b/.github/workflows/auto-release.yml @@ -3,11 +3,12 @@ on: pull_request: types: - closed - branches: - - main + - labeled jobs: - call-workflow-passing-data: - if: github.event.pull_request.merged + release: + if: | + (github.event.pull_request.merged == true && github.event.pull_request.base.ref == 'main') || + github.event.label.name == 'pre-release' uses: fivetran/dbt_package_automations/.github/workflows/auto-release.yml@main - secrets: inherit \ No newline at end of file + secrets: inherit diff --git a/.github/workflows/generate-docs.yml b/.github/workflows/generate-docs.yml new file mode 100644 index 0000000..6b92a7f --- /dev/null +++ b/.github/workflows/generate-docs.yml @@ -0,0 +1,14 @@ +name: 'generate dbt docs' +on: + pull_request: + types: + - labeled + +jobs: + generate-docs: + if: github.event.label.name == 'docs:ready' + uses: fivetran/dbt_package_automations/.github/workflows/generate-docs.yml@main + secrets: inherit + with: + # This needs to be kept a long list like this, and ': public' should not be included in the last var of the list. + schema_var_name: "apple_store_schema: 'public', google_play_schema" diff --git a/.gitignore b/.gitignore index 2f4710d..06f4e77 100644 --- a/.gitignore +++ b/.gitignore @@ -1,7 +1,73 @@ - -target/ +# dbt +**/package-lock.yml +package-lock.yml +.dbt/ +dbt_modules/ dbt_packages/ logs/ +profiles.yml +target/ +*.log + +# IDE files +.idea/ +.vscode/ +*~ +*.swp +*.swo + +# Jupyter Notebook +.ipynb_checkpoints + +# OS generated files +**/.DS_Store .DS_Store -package-lock.yml -env/ \ No newline at end of file +.Spotlight-V100 +.Trashes +._* +Thumbs.db +ehthumbs.db + +# Python +*.egg +*.egg-info/ +*.py[cod] +*.so +*$py.class +.Python +__pycache__/ +build/ +develop-eggs/ +dist/ +downloads/ +eggs/ +.env +.installed.cfg +lib/ +lib64/ +MANIFEST +parts/ +sdist/ +var/ +wheels/ + +# Secrets and credentials +.env.* +.secrets +credentials.json +service-account.json + +# Temporary files +.cache/ +*.temp +*.tmp + +# Virtual environments +.conda/ +.env +.venv +ENV/ +env/ +env.bak/ +venv/ +venv.bak/ diff --git a/CHANGELOG.md b/CHANGELOG.md index da8cca9..22649ec 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,9 +5,12 @@ - Updated package to maintain compatibility with dbt-core versions both before and after v1.10.6, which introduced a breaking change to multi-argument test syntax (e.g., `unique_combination_of_columns`). - Temporarily removed unsupported tests to avoid errors and ensure smoother upgrades across different dbt-core versions. These tests will be reintroduced once a safe migration path is available. - Removed all `dbt_utils.unique_combination_of_columns` tests. - - Removed all accepted_values tests. - Moved `loaded_at_field: _fivetran_synced` under the `config:` block in `src_app_reporting.yml`. +### Under the Hood +- Updated conditions in `.github/workflows/auto-release.yml`. +- Added `.github/workflows/generate-docs.yml`. + # dbt_app_reporting v0.6.0 [PR #28](https://github.com/fivetran/dbt_app_reporting/pull/28) includes the following updates: diff --git a/LICENSE b/LICENSE index 261eeb9..f9554a9 100644 --- a/LICENSE +++ b/LICENSE @@ -186,7 +186,7 @@ same "printed page" as the copyright notice for easier identification within third-party archives. - Copyright [yyyy] [name of copyright owner] + Copyright © 2025 Fivetran Inc. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. diff --git a/README.md b/README.md index 4356aeb..58bbf0d 100644 --- a/README.md +++ b/README.md @@ -1,16 +1,21 @@ -

+ +# App Reporting dbt Package ([Docs](https://fivetran.github.io/dbt_app_reporting/)) + +

+ href="https://github.com/fivetran/dbt_app_reporting/blob/main/LICENSE"> - + + +

-# App Reporting dbt Package ([Docs](https://fivetran.github.io/dbt_app_reporting/)) ## What does this dbt package do? - Standardizes schemas from various app platform connectors and creates reporting models for all activity aggregated to the device, country, OS version, app version, traffic source and subscription levels. - Currently supports the following Fivetran app platform connectors: @@ -70,7 +75,7 @@ By default, this package also looks for your connection data in specific schemas vars: apple_store_schema: itunes_connect apple_store_database: your_database_name - + google_play_schema: google_play google_play_database: your_database_name ``` @@ -115,7 +120,7 @@ models: +schema: apple_store # default schema suffix apple_store_source: +schema: apple_store_source # default schema suffix - + google_play: +schema: google_play # default schema suffix google_play_source: @@ -146,13 +151,13 @@ To connect your multiple schema/database sources to the package models, follow t If an individual source table has a different name than the package expects, add the table name as it appears in your destination to the respective variable. This is not available when running the package on multiple unioned connections. > IMPORTANT: See the Apple Store [`dbt_project.yml`](https://github.com/fivetran/dbt_apple_store_source/blob/main/dbt_project.yml) and Google Play [`dbt_project.yml`](https://github.com/fivetran/dbt_google_play_source/blob/main/dbt_project.yml) variable declarations to see the expected names. - + ```yml vars: apple_store__identifier: your_table_name google_play__identifier: your_table_name ``` - +
@@ -178,7 +183,7 @@ packages: - package: fivetran/google_play version: [">=0.5.0", "<0.6.0"] - + - package: fivetran/google_play_source version: [">=0.5.0", "<0.6.0"] diff --git a/integration_tests/ci/sample.profiles.yml b/integration_tests/ci/sample.profiles.yml index 4b9abfd..cda89da 100644 --- a/integration_tests/ci/sample.profiles.yml +++ b/integration_tests/ci/sample.profiles.yml @@ -2,7 +2,6 @@ # HEY! This file is used in the dbt package integrations tests with Buildkite. # You should __NEVER__ check credentials into version control. Thanks for reading :) - integration_tests: target: redshift outputs: diff --git a/integration_tests/dbt_project.yml b/integration_tests/dbt_project.yml index 4df9da7..494b140 100644 --- a/integration_tests/dbt_project.yml +++ b/integration_tests/dbt_project.yml @@ -3,9 +3,9 @@ version: '0.7.0' profile: 'integration_tests' config-version: 2 vars: - # apple_store__using_subscriptions: true # uncomment this line when generating docs! - # google_play__using_subscriptions: true # uncomment this line when regenerating docs! - # google_play__using_earnings: true # uncomment this line when regenerating docs! + # apple_store__using_subscriptions: true # @docs-include + # google_play__using_subscriptions: true # @docs-include + # google_play__using_earnings: true # @docs-include google_play_schema: app_reporting_integrations_test_9 apple_store_schema: app_reporting_integrations_test_9 @@ -54,6 +54,8 @@ models: +schema: "app_reporting_{{ var('directed_schema','dev') }}" ## To be used for validation testing seeds: + +docs: + show: False app_reporting_integration_tests: +quote_columns: "{{ true if target.type == 'redshift' else false }}" +column_types: @@ -66,3 +68,5 @@ seeds: app_version_code: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}" flags: send_anonymous_usage_stats: False + + use_colors: True \ No newline at end of file