Add Data Metric Function cleanup support

[jonhopper-dataengineers]

Summary

• Added clean_data_metric_functions macro to reconcile dbt-defined Data Metric Functions (DMFs) against deployed DMFs in Snowflake and drop orphaned ones
• Added data_metric_functions as a supported object_type in the clean_objects orchestrator (included in default list)
• Modified clean_functions to exclude DMFs (is_data_metric = 'NO' filter) so they are handled exclusively by the new dedicated macro
• Updated version to 1.0.3, CHANGELOG, README, and clean.yml documentation

Details

DMFs live in information_schema.functions with is_data_metric = 'YES' and have TABLE(...) argument signatures. They must be dropped with DROP FUNCTION (not DROP DATA METRIC FUNCTION). The new clean_data_metric_functions macro:

1. Queries information_schema.functions filtered to is_data_metric = 'YES'
2. Matches against dbt graph nodes with config.materialized = 'data_metric_function' (from the companion dbt-snowflake-datops-materilizations package)
3. Drops unmatched DMFs using DROP FUNCTION IF EXISTS with the full signature

Test plan

• Verify clean_data_metric_functions identifies and drops orphaned DMFs in a test database
• Verify clean_functions no longer picks up DMFs (the is_data_metric = 'NO' filter excludes them)
• Verify clean_objects with default object_types includes data_metric_functions in the cleanup run
• Verify dry-run mode logs the drop statements without executing

.... Generated with Cortex Code

Summary by Sourcery

Add dedicated cleanup support for Snowflake Data Metric Functions and wire it into the existing clean orchestration while updating package metadata.

New Features:

• Introduce a clean_data_metric_functions macro to reconcile dbt-defined Data Metric Functions with deployed ones and drop orphans.
• Add data_metric_functions as a supported object_type in the clean_objects orchestrator, enabled by default.

Enhancements:

• Update clean_functions to explicitly exclude Data Metric Functions so they are handled only by the new macro.
• Extend documentation and macro catalog to describe Data Metric Function cleanup and clarify that clean_functions skips DMFs.
• Set the default clean_objects configuration to invoke Data Metric Function cleanup alongside other object types.

Build:

• Bump project version from 1.0.2 to 1.0.3 in dbt_project.yml.

Documentation:

• Revise README with the new version, updated package revision, and documentation of clean_data_metric_functions and supported object_types for clean_objects.
• Add clean_data_metric_functions to clean.yml with public docs for its arguments and behavior.
• Update CHANGELOG with a 1.0.3 entry describing Data Metric Function cleanup support and related changes.

[sourcery-ai]

Reviewer's Guide

Adds a dedicated clean_data_metric_functions dbt macro and wires it into the clean_objects orchestrator while ensuring regular function cleanup excludes Data Metric Functions, plus associated versioning and docs updates.

Sequence diagram for clean_objects invoking clean_data_metric_functions

sequenceDiagram
    actor DbtUser
    participant clean_objects
    participant clean_functions
    participant clean_data_metric_functions
    participant Snowflake
    participant dbt_graph as dbt_graph_nodes
    participant drop_object

    DbtUser->>clean_objects: clean_objects(database, clean_targets, object_types)
    clean_objects->>clean_functions: clean_functions(database, dry_run)
    Note over clean_functions: Filters functions with is_data_metric = 'NO'

    clean_objects->>clean_data_metric_functions: clean_data_metric_functions(database, dry_run)
    clean_data_metric_functions->>Snowflake: SELECT FROM information_schema.functions
    Note over Snowflake: WHERE is_data_metric = 'YES'

    clean_data_metric_functions->>dbt_graph: iterate graph.nodes
    Note over dbt_graph: Find nodes with materialized = data_metric_function

    clean_data_metric_functions->>drop_object: drop_object("FUNCTION", database, snowflake_dmfs_to_drop, dry_run)
    drop_object-->>Snowflake: DROP FUNCTION IF EXISTS ...

Loading

File-Level Changes

Change	Details	Files
Introduce dedicated cleanup macro for Snowflake Data Metric Functions (DMFs) and reconcile against dbt graph.	Add clean_data_metric_functions macro to query information_schema.functions for is_data_metric = 'YES' DMFs, excluding system/meta schemas. Iterate over dbt graph nodes with materialized = 'data_metric_function' and optional override_name to determine which DMFs are still defined in the project. Collect unmatched DMFs by full signature and pass them to drop_object("FUNCTION", ...) supporting dry_run behavior.	macros/clean/clean_data_metric_functions.sql macros/clean/clean.yml
Update orchestrator and existing function cleanup to separate DMFs from regular functions and procedures.	Extend clean_objects default object_types list and dispatch logic to include data_metric_functions and invoke clean_data_metric_functions when selected. Tighten clean_functions query to exclude DMFs by adding is_data_metric = 'NO' filter so they are not dropped by generic function cleanup.	macros/clean/clean_objects.sql macros/clean/clean_functions.sql
Refresh documentation and package metadata for the new DMF cleanup feature.	Document clean_data_metric_functions in clean.yml and README, including its signature and description, and add data_metric_functions to the supported clean_objects object_types list. Update CHANGELOG with v1.0.3 section describing DMF cleanup behavior and function filter change. Bump package version references from 1.0.2/1.0.0 to 1.0.3 in dbt_project.yml and README usage snippet.	macros/clean/clean.yml README.md CHANGELOG.md dbt_project.yml

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've left some high level feedback:

• Consider wrapping clean_data_metric_functions in the same if execute/flags.WHICH/target.name guards used by clean_objects and other clean macros so it doesn’t run in unexpected contexts (e.g. parsing or docs generation).
• The node.config.get(...) calls in clean_data_metric_functions assume config is a dict, but in dbt it is a config object; accessing meta/override_name via attributes (and/or getattr) will be safer and avoid runtime errors.
• The iteration over snowflake_dmf_results using result.values()[0/1/2] relies on implicit ordering; using explicit column access (e.g. snowflake_dmf_results.columns and rows or named indices) would make the macro more robust to changes in the query shape.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- Consider wrapping `clean_data_metric_functions` in the same `if execute`/`flags.WHICH`/`target.name` guards used by `clean_objects` and other clean macros so it doesn’t run in unexpected contexts (e.g. parsing or docs generation).
- The `node.config.get(...)` calls in `clean_data_metric_functions` assume `config` is a dict, but in dbt it is a config object; accessing meta/override_name via attributes (and/or `getattr`) will be safer and avoid runtime errors.
- The iteration over `snowflake_dmf_results` using `result.values()[0/1/2]` relies on implicit ordering; using explicit column access (e.g. `snowflake_dmf_results.columns` and `rows` or named indices) would make the macro more robust to changes in the query shape.

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}