bundle/direct: add a feature-flag list to the direct deployment state by shreyas-goenka · Pull Request #5403 · databricks/cli

shreyas-goenka · 2026-06-01T17:59:12Z

Why

The direct deployment state needs a forward-compatibility mechanism: as the engine gains capabilities (the first is experimental.record_deployment_history / DMS), a state written by a newer CLI may rely on something an older CLI doesn't understand. Rather than bump the schema version for every such capability, we add a generic feature-flag list to the state. An older CLI that finds a feature it doesn't recognize fails with an actionable message instead of silently mishandling the state.

What

Bumps the direct state schema to version 3, which adds Header.Features — a map of feature name → the CLI version that recorded it. (Older CLIs, max v2, reject v3 with the existing "upgrade the CLI" error; this is the one schema bump, after which capabilities are feature flags, not version bumps.)
On load, Open rejects any state that records a feature this CLI doesn't know, naming the feature and the minimum CLI version recorded in the state: the deployment state requires feature "X" which this CLI (<v>) does not support; upgrade to <minVersion> or newer.
A deploy that opts into a feature records it (RecordFeature), stamping the current CLI version as the minimum. record_deployment_history is the first feature.
record_deployment_history is still rejected under the terraform engine (it drives DMS, which only the direct engine supports).

Tests

Unit (dstate): schema-version pin + migrations-completeness tripwire, legacy→current migration, reject-newer-version, RecordFeature round-trip + panic-if-WAL-started, and checkSupportedFeatures (known accepted / unknown rejected).
Acceptance:
- bundle/state/unknown_feature — a state requiring an unknown feature is rejected with the recorded min version.
- record-deployment-history/state-upgrade — deploy without the flag (no features) → with the flag (feature recorded) → older CLI (v1.0.0) rejects v3.
- record-deployment-history/terraform-unsupported — the flag is rejected under terraform.
- All direct-engine goldens regenerated for state_version: 3.

This pull request and its description were written by Isaac.

eng-dev-ecosystem-bot · 2026-06-01T18:39:50Z

Integration test report

Commit: 17b0ef2

Run: 27814010649

	Env	🟨KNOWN	🔄flaky	💚RECOVERED	🙈SKIP	✅pass	🙈skip	Time
🟨	aws linux	7			13	265	1013	7:01
💚	aws windows			7	13	267	1011	8:16
💚	aws-ucws linux			7	13	361	927	6:17
💚	aws-ucws windows			7	13	363	925	8:52
💚	azure linux			1	15	268	1011	5:30
🔄	azure windows		3		15	268	1009	9:06
💚	azure-ucws linux			1	15	366	923	7:06
💚	azure-ucws windows			1	15	368	921	9:02
💚	gcp linux			1	15	264	1014	6:29
💚	gcp windows			1	15	266	1012	8:29

22 interesting tests: 13 SKIP, 7 KNOWN, 2 flaky

	Test Name	aws linux	aws windows	aws-ucws linux	aws-ucws windows	azure linux	azure windows	azure-ucws linux	azure-ucws windows	gcp linux	gcp windows
🟨	TestAccept	🟨K	💚R	💚R	💚R	💚R	🔄f	💚R	💚R	💚R	💚R
🙈	TestAccept/bundle/invariant/no_drift	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/permissions	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🟨	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions	🟨K	💚R	💚R	💚R	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🟨	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct	🟨K	💚R	💚R	💚R
🟨	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform	🟨K	💚R	💚R	💚R
🟨	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions	🟨K	💚R	💚R	💚R	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🟨	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct	🟨K	💚R	💚R	💚R
🟨	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform	🟨K	💚R	💚R	💚R
🙈	TestAccept/bundle/resources/postgres_branches/basic	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_branches/recreate	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_branches/replace_existing	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_branches/update_protected	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_branches/without_branch_id	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_endpoints/basic	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_projects/update_display_name	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/synced_database_tables/basic	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🔄	TestAccept/selftest/record_cloud/basic	✅p	✅p	✅p	✅p	✅p	🔄f	✅p	✅p	✅p	✅p
🔄	TestAccept/selftest/record_cloud/basic/DATABRICKS_BUNDLE_ENGINE=terraform	✅p	✅p	✅p	✅p	✅p	🔄f	✅p	✅p	✅p	✅p
🙈	TestAccept/ssh/connection	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S

Top 24 slowest tests (at least 2 minutes):

duration	env	testname
4:31	gcp windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:05	gcp linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:03	gcp windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:58	gcp linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:44	aws windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:29	aws-ucws windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:22	gcp windows	TestAccept
3:16	azure-ucws windows	TestAccept
3:15	aws-ucws windows	TestAccept
3:15	aws windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:13	aws-ucws windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:12	aws windows	TestAccept
2:52	azure-ucws linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:45	aws linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:44	azure windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:40	azure-ucws windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:34	azure linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:33	azure windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:33	azure-ucws windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:32	aws-ucws linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:31	aws linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:28	azure-ucws linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:21	azure linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:21	aws-ucws linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct

shreyas-goenka · 2026-06-01T21:47:42Z

+
+			// Enforce the recorded DMS protocol version only when this bundle has
+			// opted into DMS; a bundle that has not opted in does not act on it.
+			if b.Config.Experimental != nil && b.Config.Experimental.RecordDeploymentHistory {


if the config is not set - the deployment just falls back to direct deplioyment.

github-actions · 2026-06-01T21:51:43Z

Approval status: pending

`/acceptance/bundle/` - needs approval

26 files changed
Suggested: @denik
Also eligible: @pietern, @janniklasrose, @lennartkats-db, @andrewnester, @anton-107

`/bundle/` - needs approval

5 files changed
Suggested: @denik
Also eligible: @pietern, @janniklasrose, @lennartkats-db, @andrewnester, @anton-107

`/cmd/bundle/` - needs approval

Files: cmd/bundle/utils/process.go
Suggested: @denik
Also eligible: @pietern, @janniklasrose, @lennartkats-db, @andrewnester, @anton-107

General files (require maintainer)

Files: acceptance/acceptance_test.go
Based on git history:

@denik -- recent work in bundle/direct/dstate/, bundle/phases/, cmd/bundle/utils/

_{Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @simonfaltum, @renaudhartert-db) can approve all areas.

See OWNERS for ownership rules.}

denik · 2026-06-02T12:26:07Z

+func TestStateSchemaVersions(t *testing.T) {
+	assert.Equal(t, 2, currentStateVersion)
+	assert.Equal(t, 3, dmsStateVersion)
+	assert.Equal(t, 1, currentDmsVersion)


What does this test? We know constants in golang work.

This is a tripwire test that will trigger anytime someone does a version bump. This forces the author and reviewer to read the instructions above and not miss them.

denik · 2026-06-02T12:32:36Z

+
+title "without the flag, the same state is accepted (the check is gated on opt-in)"
+trace update_file.py databricks.yml "record_deployment_history: true" "record_deployment_history: false"
+trace errcode $CLI bundle plan 2>&1 | contains.py "!is newer than supported version"


What happens if experimental flag is removed? Do we stop calling DMS but keep using normal direct engine? Do we keep version 3 or 2 in that case?

Do we stop calling DMS but keep using normal direct engine?

Yes, we fall back to normal direct engine. It works fine because we'll continue to maintain direct state with WAL and resources.json as we preview.

Do we keep version 3 or 2 in that case?

We keep version 3. The configuration remains the source of truth for whether to use direct or DMS (provide easy fallback for customers).

Bumps the direct state schema to version 3, which adds a per-state feature list (Header.Features) mapping each recorded feature flag to the CLI version that wrote it. From v3 on, additive capabilities are recorded as feature flags rather than schema-version bumps: a CLI that loads a state recording a feature it does not understand errors and points the user at the minimum CLI version recorded in the state. record_deployment_history is the first such feature. Older CLIs (max state version 2) reject v3 state with the existing "upgrade the CLI" error; record_deployment_history is still rejected under the terraform engine, which does not support it. Co-authored-by: Isaac

shreyas-goenka · 2026-06-19T08:43:02Z

Superseded by #5660, which is a clean, minimal version focused solely on the feature-flag mechanism (no DMS/record_deployment_history cruft).

shreyas-goenka temporarily deployed to test-trigger-is June 1, 2026 17:59 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/dms-state-version branch from ad20bc3 to ccbd8f1 Compare June 1, 2026 19:55

shreyas-goenka changed the base branch from shreyas-goenka/dms-version to main June 1, 2026 19:55

shreyas-goenka force-pushed the shreyas-goenka/dms-state-version branch from ccbd8f1 to cefabe0 Compare June 1, 2026 20:10

shreyas-goenka temporarily deployed to test-trigger-is June 1, 2026 20:11 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/dms-state-version branch from cefabe0 to e70bc2c Compare June 1, 2026 20:11

shreyas-goenka temporarily deployed to test-trigger-is June 1, 2026 20:12 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/dms-state-version branch from e70bc2c to e1babf2 Compare June 1, 2026 20:23

shreyas-goenka temporarily deployed to test-trigger-is June 1, 2026 20:24 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/dms-state-version branch from e1babf2 to d15b537 Compare June 1, 2026 20:27

shreyas-goenka temporarily deployed to test-trigger-is June 1, 2026 20:28 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/dms-state-version branch from d15b537 to fab41cc Compare June 1, 2026 20:42

shreyas-goenka temporarily deployed to test-trigger-is June 1, 2026 20:43 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/dms-state-version branch from fab41cc to ea32c1d Compare June 1, 2026 20:57

shreyas-goenka temporarily deployed to test-trigger-is June 1, 2026 20:58 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/dms-state-version branch from ea32c1d to cf5f00d Compare June 1, 2026 21:17

shreyas-goenka temporarily deployed to test-trigger-is June 1, 2026 21:17 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/dms-state-version branch from cf5f00d to 69c7de4 Compare June 1, 2026 21:18

shreyas-goenka temporarily deployed to test-trigger-is June 1, 2026 21:18 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/dms-state-version branch from 69c7de4 to 522d49e Compare June 1, 2026 21:25

shreyas-goenka temporarily deployed to test-trigger-is June 1, 2026 21:26 — with GitHub Actions Inactive

shreyas-goenka changed the title ~~bundle/direct: conditionally upgrade state version when opted into DMS~~ Add direct state v3 - opt in only for record_deployment_history preview Jun 1, 2026

shreyas-goenka force-pushed the shreyas-goenka/dms-state-version branch from 522d49e to 47fbd29 Compare June 1, 2026 21:40

shreyas-goenka temporarily deployed to test-trigger-is June 1, 2026 21:41 — with GitHub Actions Inactive

shreyas-goenka commented Jun 1, 2026

View reviewed changes

shreyas-goenka marked this pull request as ready for review June 1, 2026 21:51

shreyas-goenka requested a review from denik June 1, 2026 21:51

denik reviewed Jun 2, 2026

View reviewed changes

shreyas-goenka force-pushed the shreyas-goenka/dms-state-version branch from 47fbd29 to 94d066e Compare June 2, 2026 12:53

shreyas-goenka temporarily deployed to test-trigger-is June 2, 2026 12:54 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/dms-state-version branch from 94d066e to cf6ef38 Compare June 2, 2026 15:01

shreyas-goenka temporarily deployed to test-trigger-is June 2, 2026 15:02 — with GitHub Actions Inactive

shreyas-goenka requested a review from denik June 2, 2026 15:02

shreyas-goenka force-pushed the shreyas-goenka/dms-state-version branch from cf6ef38 to 230c84d Compare June 9, 2026 14:30

shreyas-goenka changed the title ~~Add direct state v3 - opt in only for record_deployment_history preview~~ bundle/direct: add a feature-flag list to the direct deployment state Jun 9, 2026

shreyas-goenka removed the request for review from denik June 19, 2026 08:07

shreyas-goenka force-pushed the shreyas-goenka/dms-state-version branch from 230c84d to 17b0ef2 Compare June 19, 2026 08:13

shreyas-goenka temporarily deployed to test-trigger-is June 19, 2026 08:13 — with GitHub Actions Inactive

shreyas-goenka closed this Jun 19, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bundle/direct: add a feature-flag list to the direct deployment state#5403

bundle/direct: add a feature-flag list to the direct deployment state#5403
shreyas-goenka wants to merge 1 commit into
mainfrom
shreyas-goenka/dms-state-version

shreyas-goenka commented Jun 1, 2026 •

edited

Loading

Uh oh!

eng-dev-ecosystem-bot commented Jun 1, 2026 •

edited

Loading

Uh oh!

shreyas-goenka Jun 1, 2026

Uh oh!

github-actions Bot commented Jun 1, 2026 •

edited

Loading

Uh oh!

Uh oh!

denik Jun 2, 2026

Uh oh!

shreyas-goenka Jun 2, 2026

Uh oh!

Uh oh!

Uh oh!

denik Jun 2, 2026

Uh oh!

shreyas-goenka Jun 2, 2026

Uh oh!

shreyas-goenka Jun 2, 2026

Uh oh!

Uh oh!

shreyas-goenka commented Jun 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

shreyas-goenka commented Jun 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What

Tests

Uh oh!

eng-dev-ecosystem-bot commented Jun 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Integration test report

Uh oh!

shreyas-goenka Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Jun 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Approval status: pending

/acceptance/bundle/ - needs approval

/bundle/ - needs approval

/cmd/bundle/ - needs approval

General files (require maintainer)

Uh oh!

Uh oh!

denik Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

shreyas-goenka Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

denik Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

shreyas-goenka Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

shreyas-goenka Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

shreyas-goenka commented Jun 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

shreyas-goenka commented Jun 1, 2026 •

edited

Loading

eng-dev-ecosystem-bot commented Jun 1, 2026 •

edited

Loading

github-actions Bot commented Jun 1, 2026 •

edited

Loading

`/acceptance/bundle/` - needs approval

`/bundle/` - needs approval

`/cmd/bundle/` - needs approval