direct: ignore UC-managed schema property defaults#5195
Merged
Conversation
UC auto-populates system-managed property keys (e.g. `unity.catalog.managed.delta.defaults.delta.enableRowTracking`) on schema creation. Without a backend_defaults rule, the planner sees the remote map as drift, emits Update, and DoUpdate sends an empty payload which UC rejects with "UpdateSchema Nothing to update". The rule only applies when both saved and new are nil, so user-set properties still drive real drift detection. Also mirror the UC behavior in the fake testserver so the no-drift invariant is exercised locally; added acceptance/.../schemas/drift/ managed_properties covering the reproducer. Co-authored-by: Isaac
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
The fake populated UC-managed properties on every property-less schema. The direct engine ignores them as backend defaults, but terraform treats them as drift, which broke unrelated schema acceptance tests across both engines. Gate the injection on a dedicated schema name so only the backend-default drift test opts in, and drop recreate's now-incidental property line. Co-authored-by: Isaac
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
janniklasrose
commented
Jun 9, 2026
Collaborator
Integration test reportCommit: 9a90424
24 interesting tests: 15 SKIP, 7 KNOWN, 2 flaky
Top 26 slowest tests (at least 2 minutes):
|
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
denik
reviewed
Jun 11, 2026
Flattens the shouldSkipBackendDefault -> matchesBackendDefaultMap -> matchesAnyBackendDefault chain: the nil-vs-map walk now sits directly in shouldSkipBackendDefault, and the per-rule prefix/values check is merged into matchesAnyBackendDefault, which remains a function because the map case calls it once per entry. Co-authored-by: Isaac
Build the test's lifecycle config inline instead of reading the real schemas config via GetResourceConfig, so future resources.yml edits can't break the unit test. The real wiring stays covered by the schemas/drift acceptance test. Co-authored-by: Isaac
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
The GitHub web conflict editor left the <<<<<<< / ======= / >>>>>>> markers in resources.yml, breaking YAML parsing (lint, direct tests, and validate-generated). Keep both sides: backend_defaults from this branch plus normalize_case/normalize_slash from main (#5531), ordered to match the volumes section. Co-authored-by: Isaac
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
shreyas-goenka
approved these changes
Jun 15, 2026
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
denik
reviewed
Jun 15, 2026
denik
approved these changes
Jun 15, 2026
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
Collaborator
Integration test reportCommit: eae333b
395 interesting tests: 304 MISS, 38 FAIL, 31 RECOVERED, 16 flaky, 3 PANIC, 2 SKIP, 1 KNOWN
Top 50 slowest tests (at least 2 minutes):
|
deco-sdk-tagging Bot
added a commit
that referenced
this pull request
Jun 17, 2026
## Release v1.4.0 ### CLI * Improved error messages for `ssh connect`: when an SSH connection attempt fails, the client now fetches and prints the server's recent error logs ([#5555](#5555)). * Increase the SSH server startup timeout from 10 to 45 minutes when a GPU accelerator is requested via `databricks ssh connect --accelerator` ([#5569](#5569)). * Fix authentication falling back to the default profile in `.databrickscfg` when a host is already configured via the environment (e.g. `DATABRICKS_HOST` with `DATABRICKS_TOKEN`) ([#5616](#5616)). * ssh: fix opening remote environment in Cursor, which previously hung on default-extension install and never opened the editor ([#5619](#5619)). * Improve the error shown when `databricks labs install` cannot find a project's `labs.yml`: the message now explains that either the requested version does not exist or the project is not installable with the CLI, and links to the repository ([#5559](#5559)). ### Bundles * Remove API enum values and types that are still in development from the `databricks-bundles` Python package; these were never accepted by the backend ([#5484](#5484)). * direct: Fix resolving a resource reference that is used more than once within the same field ([#5558](#5558)). * Bundle variable references now accept Unicode letters in path segments (e.g. `${var.变量}`). ([#5532](#5532)) * Ignore remote changes for vector search direct_access_index_spec.schema_json to prevent drift when the backend normalizes the schema ([#5481](#5481)). * Remove hidden, never-functional `--existing-dashboard-id`, `--existing-dashboard-path`, `--existing-alert-id`, and `--existing-genie-space-id` alias flags from `bundle generate`; use the documented `--existing-id` / `--existing-path` flags instead ([#5591](#5591)). * engine/direct: Fix WAL corruption after two consecutive failed deploys ([#5606](#5606)). * engine/direct: Don't open the deployment state WAL when a deploy's plan fails ([#5607](#5607)). * Ignore unity catalog managed schema property defaults to avoid unnecessary drift ([#5195](#5195)). * Add `postgres_roles` and `postgres_databases` resources to create Postgres roles and databases on a Lakebase branch ([#5467](#5467), [#5627](#5627)). * direct: Stop spurious recreate/rename on redeploy when the backend normalizes a resource's name-based ID (e.g. Unity Catalog lowercasing a schema or volume name) ([#5599](#5599)). * Fix the generated pipeline README to suggest `databricks bundle run <pipeline> --refresh <table>` for running a single transformation; the previously documented `--select` flag is not supported by `bundle run` ([#5252](#5252)).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
backend_defaultsfor two fieldsproperties['...']structdiffwhere a remote-only map change is emitted at the parent path: only skip the parent if every remote entry matches a configured child rule. Synthesize child paths withstructpath.NewBracketStringto match whatstructdiffproduces (rather thanNewStringKey, which would diverge for identifier-like keys).Why
UC auto-populates system-managed property keys out of band (e.g. setting
unity.catalog.managed.delta.defaults.delta.enableRowTracking). This causes the planner to output Update when there is nothing to doTests
This pull request and its description were written by Isaac.