Skip to content

bundle: record and read deployment state via DMS - #6094

Merged
shreyas-goenka merged 167 commits into
mainfrom
isaac/pr6052-fixes
Sep 9, 2026
Merged

shreyas-goenka merged 167 commits into
mainfrom
isaac/pr6052-fixes

Conversation

@shreyas-goenka

@shreyas-goenka shreyas-goenka commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds DMS as a supported backend for direct deployments.

Known bug

acceptance/bundle/dms/empty-grants documents it: emptying a grants node records a succeeded UPDATE with no state, which the service rejects. That test is Cloud = false; the fix (record an update carrying the emptied state instead of a delete) is pending.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 3aabe82

Run: 34286030123

Env 🔄​flaky 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux 1 317 48 8:18
💚​ aws windows 1 319 46 12:17
🔄​ azure linux 3 314 48 10:47
💚​ azure windows 1 318 46 12:58
💚​ gcp linux 1 317 48 9:39
💚​ gcp windows 1 319 46 11:56
Test Name aws linux aws windows azure linux azure windows gcp linux gcp windows
🔄​ TestAccept 💚​R 💚​R 🔄​f 💚​R 💚​R 💚​R
🔄​ TestAccept/bundle/resources/dashboards/detect-change ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/dashboards/detect-change/DATABRICKS_BUNDLE_ENGINE=direct/DMS= ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
Top 13 slowest tests (at least 2 minutes):
duration env testname
5:11 azure windows TestAccept
3:43 aws windows TestAccept
3:34 gcp windows TestAccept
2:49 gcp windows TestAccept/bundle/dms/existing-state/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
2:34 gcp windows TestAccept/bundle/dms/declined-deploy/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
2:29 aws windows TestAccept/bundle/resources/quality_monitors/change_table_name/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
2:14 azure windows TestAccept/bundle/dms/no-drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
2:09 azure windows TestAccept/bundle/resources/dashboards/detect-change/DATABRICKS_BUNDLE_ENGINE=terraform/DMS=
2:08 aws windows TestAccept/bundle/dms/stale-plan/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true/READPLAN=
2:06 aws windows TestAccept/bundle/dms/no-drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
2:06 azure linux TestFilerWorkspaceFilesExtensionsReadDir
2:04 azure windows TestAccept/bundle/resources/quality_monitors/change_assets_dir/DATABRICKS_BUNDLE_ENGINE=terraform/DMS=
2:02 aws windows TestAccept/bundle/resources/quality_monitors/change_assets_dir/DATABRICKS_BUNDLE_ENGINE=terraform/DMS=

Comment thread bundle/direct/bundle_apply.go Outdated
// of and the next deploy would create them a second time. Checked here rather
// than only where operations are recorded, which is after the resource has
// already been modified.
if err := opQueue.firstErr(); err != nil {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could eventually extend this to record and return all multiple errors that happened.

@@ -0,0 +1,4 @@

=== An operation upload failure fails the deploy instead of reporting only at the end

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's hard to make a assert more here because we cannot control how many requests went through. We could harden this test by making the number of workers configurable and 1. Omitting for now.

Comment thread acceptance/bundle/dms/test.toml Outdated
@@ -0,0 +1,18 @@
Local = true
Cloud = false

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll rollout the service to our test env by next week at which point we can start turning these tests on on cloud as well.


# Deployment Metadata Service (DMS) recording is only supported by the direct
# engine; it is a no-op on terraform.
EnvMatrix.DATABRICKS_BUNDLE_ENGINE = ["direct"]

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eventually we can also make DMS part of the default test matrix - to get more coverage - this can be rolled out slowly.

Comment thread bundle/direct/opqueue_test.go Outdated
assert.Empty(t, f.recorded())
}

func TestOperationQueueCloseIsIdempotent(t *testing.T) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not strictly necessary given we only have one close site.

Comment thread libs/dms/recorder.go Outdated
return &apiVersionCreator{client: c}
}

func (a *apiVersionCreator) CreateVersion(ctx context.Context, deploymentID, versionID string, body createVersionRequest) (*bundledeployments.Version, error) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will be replaced with SDK calls soon in a followup.

Comment thread libs/testserver/bundledeployments.go Outdated
// The extra field cannot be added by embedding Deployment in a wrapper struct:
// Deployment has its own MarshalJSON, which is promoted to the wrapper and
// silently drops any sibling field.
func deploymentBody(d *dmsDeployment) (map[string]any, error) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

goes away on a SDK bump

Comment thread cmd/bundle/summary.go Outdated
Comment thread libs/dms/recording_test.go Outdated
r := NewRecorder(RecorderOptions{Service: f, Versions: fakeVersions{requests: &f.versions}, DeploymentID: "stored-id", StatePath: testStatePath, Metadata: Metadata{TargetName: "dev", DisplayName: testDisplayName}, VersionType: VersionTypeDeploy})

err := r.CreateVersion(t.Context())
assert.ErrorContains(t, err, "internal error: no deployment found for the file with object id stored-id")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should never happen if the service is working properly, since htis indicates that the file exists but there was no deployment returned by GET deployment.

Comment thread bundle/direct/bundle_apply.go Outdated
// (sv.Value) come from the write just performed; GetResourceID reads
// the ID assigned by Deploy. depends_on is recorded alongside the config
// because it cannot be recomputed from it (see dstate.RecordedState).
if err := opQueue.record(ctx, resourceKey, action, b.StateDB.GetResourceID(resourceKey), sv.Value, d.DependsOn); err != nil {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct me if I'm wrong, this records "recreate" once, correct?

We discussed sending two requests - one for Delete and one for subsequent Create.

We also discussed moving recording hooks inside SaveState, so that we record the event ASAP. I'm considering for some resources to record state earlier and/or multiple times, by moving recording into SaveState we'll be able to handle that with DMS #5391

Comment thread bundle/direct/opclient.go Outdated
"github.com/databricks/databricks-sdk-go/service/bundledeployments"
)

// The CLI calls the operations API directly rather than through the generated

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to fix this. I'll investigate in a followup.

Comment thread libs/dms/recording.go Outdated

// The server expires a version's lease if it does not receive a heartbeat
// within a 2-minute TTL; we heartbeat well inside that window.
const defaultHeartbeatInterval = 30 * time.Second

@shreyas-goenka shreyas-goenka Aug 11, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parts of the code here should be removed and replaced by the SDK soon.

Comment thread bundle/phases/destroy.go Outdated
// node under the state directory, so files.Delete removes it and any later call
// fails with 404. CompleteVersion is idempotent, so the deferred call in Destroy
// is a no-op after this.
if err := recorder.CompleteVersion(ctx, true); err != nil {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its a bit of a chicken and egg problem. We can improve this by first cleaning up worksapce.file_path + artifact_path. Then completing the version and then deleting state_path + root_path.

Left as a followup exercise, the current PR is already big enough in scope.

Comment thread acceptance/bundle/test.toml Outdated
# unset; `deploy --plan` then creates the resources without it and the next plan reports
# drift. Stamping at plan time would mean `bundle plan` creating the deployment record,
# which is a design decision, so the saved-plan path is left out of the DMS run for now.
EnvMatrixExclude.dms_no_readplan = ["DATABRICKS_BUNDLE_RECORD_DEPLOYMENT_HISTORY=true", "READPLAN=1"]

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll fix this in a followup.

Comment thread bundle/direct/oprecorder.go Outdated
// failureFields. A failure that arrives before any operation exists still
// goes through CreateOperation below, carrying the prior state.
fields = failureFields
body = updateOperationRequest{

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By omitting state here we ensure that the latest state that was uploaded in a partial operation is retained.

@shreyas-goenka
shreyas-goenka requested a review from denik August 11, 2026 12:46
varundeepsaini pushed a commit to varundeepsaini/cli that referenced this pull request Aug 17, 2026
…lates (databricks#6220)

## Changes

`bundle init` now writes `experimental.record_deployment_history: true`
into the generated `databricks.yml` when
`DATABRICKS_BUNDLE_INIT_RECORD_DEPLOYMENT_HISTORY` is set. It covers
every built-in template that generates a `databricks.yml`, and is off by
default so template output is otherwise unchanged.

## Why

Lets new projects opt into recording deployment history at creation time
instead of editing the config by hand afterwards. The variable is
temporary and deliberately scoped to `bundle init`: it persists the
setting in the generated project rather than toggling it for the bundle
being run.

Note the setting is still gated off (databricks#6094), so a generated project also
needs `DATABRICKS_BUNDLE_FORCE_ALLOW_RECORD_DEPLOYMENT_HISTORY` to
validate until that gate is lifted. The acceptance test sets both and
runs `bundle validate`, so it will fail if the two ever drift apart.

## Tests

New acceptance test: all five templates with the variable on, plus the
default-off and explicitly-false cases.
Comment thread libs/dms/client.go
Service bundledeployments.BundleDeploymentsInterface

// raw sends what the generated client cannot; see requester.
raw requester

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will go away once we get everything in the SDK

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

general comment: please add such comments directly to code, otherwise they will be lost.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

q: What's still missing from SDK?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

q: What's still missing from SDK?

A couple of that need to be propogated to the SDK from the API spec. Some are still marked as development and need to be marked as PrPr.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's mark it as PrPr and use generated client?

Comment thread libs/dms/recording.go Outdated
if err != nil {
// The service caps how many operations one version may stage, so a bundle past the
// cap cannot be recorded at all. Say so rather than passing the raw API error on.
if isResourceExhaustedErr(err) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

users might also see this if they exhaust their versions. We'll fix this soon though.

chenyuem-db pushed a commit to chenyuem-db/cli that referenced this pull request Sep 9, 2026
…atabricks#6549)

Adds a plan version repl to make the diff for
databricks#6094 smaller.

Co-authored-by: Isaac <no-reply@databricks.com>
shreyas-goenka and others added 2 commits September 9, 2026 10:28
readplan/lineage-mismatch fed one committed plan to both variants, and a committed plan
has to match the shape of the target it is fed to: a recorded target carries the
deployment_history feature, and validatePlan rejects a features mismatch before it looks
at lineage. There are now two fixtures - plan.direct.json and plan.dms.json - and the
script picks by variant, so the lineage error is asserted either way.

volume_path_job_ref captured its plan with trace and &> together, so the trace header
landed in the file alongside the JSON and nothing could filter it. Captured untraced into
a raw file and stamped out into the golden, same as change_assets_dir.

Also drops deployment_id and next_version_id from a plan-header assertion and from the
nostamp docstring: neither is part of a plan any more.

Co-authored-by: Isaac <no-reply@databricks.com>
The deploy runs as the test service principal, but the summary that reads the job id back
ran as the user. Under recording the state lives in the deploying identity deployment
record, which another identity does not resolve, so the id came back null.

Read the summary as the same identity that deployed. The test is about the permission
error on destroy, so which identity reads the id back is incidental - the goldens are
unchanged, and both variants now run.

Co-authored-by: Isaac <no-reply@databricks.com>
job_id=$($CLI bundle summary --output json | jq -r '.resources.jobs.foo.id')
# Read the summary as the deploying identity: under recording the state lives in that
# identity's deployment record, which another identity does not resolve.
job_id=$(as-test-sp $CLI bundle summary --output json | jq -r '.resources.jobs.foo.id')

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This worked previously because summary would use local state, which is not available with DMS.

shreyas-goenka and others added 2 commits September 9, 2026 10:39
Same fix as change_assets_dir: the plan was captured with trace and &> together, so the
trace header landed in the file alongside the JSON and no filter could touch it. Capture
it untraced into a raw file and stamp that out into the golden.

Co-authored-by: Isaac <no-reply@databricks.com>
The script removes plan.json itself, so there is nothing left for the harness to flag.

Co-authored-by: Isaac <no-reply@databricks.com>
Comment thread bundle/direct/bundle_plan.go Outdated
Comment thread bundle/direct/bundle_plan.go Outdated
Comment thread cmd/bundle/utils/process.go Outdated
if !cmdctx.HasWorkspaceClient(ctx) {
ctx = cmdctx.SetWorkspaceClient(ctx, b.WorkspaceClient(ctx))
}
if err := b.DeploymentBundle.StateDB.Open(ctx, localPath, dstate.WithRecovery(false), dstate.WithWrite(false), dstate.WithDeploymentHistory(true), dstate.OpenDmsArgs{ID: dmsDeploymentID, LastVersionID: lastVersionID}); err != nil {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dstate.WithDeploymentHistory(true), dstate.OpenDmsArgs{ID: dmsDeploymentID, LastVersionID: lastVersionID}

nit: seems unnecessary to have both -- dms args could be a pointer.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done for consistency really. That's how we pass other boolean options to the Open function.

if logdiag.HasError(ctx) {
return b, stateDesc, root.ErrAlreadyPrinted
}
// StateDB.Open builds the DMS client from the workspace client on the context.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you pass it via context and not as part of OpenDmsArgs?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's how we set the worspace client in other places as well. The context becomes the single place where we inititalize the workspace client.

Comment thread bundle/direct/dstate/state.go Outdated
Comment thread bundle/direct/dstate/state.go
Comment thread bundle/direct/dstate/state.go Outdated
Comment thread bundle/direct/dstate/state.go
Comment thread bundle/direct/dstate/state.go Outdated
2. run "databricks bundle destroy" to delete the existing resources
3. add experimental.deployment_history back and deploy again`)
case !recording && recorded:
return ErrUnsettingRecording

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should not a feature set in the state be definitive and sticky? Once your state is on DMS, your env var is not relevant anymore, you'll continue to use DMS.

@shreyas-goenka shreyas-goenka Sep 9, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure. We might allow unsetting this feature to opt-out of DMS (unlikely but still a possible direction to take).

For now I think lets keep the conifugration required. We can always easily relax this in the future.

shreyas-goenka and others added 2 commits September 9, 2026 11:16
CalculatePlan copied features only for a recorded deployment, but a plan should carry
whatever features its target has - the check on the other side compares both sides
regardless. Copying unconditionally needs one way to read the counter for any backend,
which GetSerial provides: the recorded version when the bundle records history, since its
state file persists no serial of its own, and the state serial otherwise.

validatePlan reads the same accessor, so the plan and the check that validates it can no
longer disagree about where the number comes from.

StateFeatures returns a copy. It handed out the live map, which the caller stamps into a
plan that outlives the lock, so the lock was protecting nothing.

Co-authored-by: Isaac <no-reply@databricks.com>
CalculatePlan copies features unconditionally now - a plan should carry whatever features
its target has, and the check on the other side compares both sides regardless. That
needs one way to read the counter for any backend, which GetSerial provides: the recorded
version when the bundle records history, since its state file persists no serial of its
own, and the state serial otherwise. validatePlan reads the same accessor, so it no longer
branches on the backend at all and one equality comparison covers both.

StateFeatures returns a copy so callers cannot modify the state feature set.

A recording open now refuses a leftover WAL rather than deleting it. A recorded deployment
writes none, so one can only have come from a deploy that did not record - and that file
is the only record of its writes.

DeleteState no longer clears Data.State, matching the direct path, and the shared stateIDs
delete is hoisted out of both branches.

Addresses review feedback.

Co-authored-by: Isaac <no-reply@databricks.com>
Comment thread libs/dms/operation.go
// and their values. It is built where the outcome is known, so a malformed one fails the
// resource that produced it rather than the upload at the end of apply.
type OperationUpdate struct {
// Fields is the mask to send. It is taken literally: a field named here is written,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean update_mask? Do we ever want non-star update here? Operation is a really small atomic model.

@shreyas-goenka shreyas-goenka Sep 9, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can look into this as a followup. Right now the backend API does not support update_mask: *.

Do we ever want non-star update here? Operation is a really small atomic model.

Sometimes yes. Like when recording an error message. In that case sometimes you want to preserve the last state. State can be big.

Comment thread libs/dms/fields.go
Comment thread libs/dms/metadata.go Outdated
Comment thread libs/dms/operation_buffer.go
Comment thread libs/dms/operation_buffer.go
shreyas-goenka and others added 2 commits September 9, 2026 11:36
A recording open refuses a leftover WAL now instead of deleting it, so the file survives
the refusal. The destroy that follows runs without recording, so it takes the recovery
path and died replaying that stale WAL - which aborted the script and left its recorded
requests behind, committed by accident as a golden. Remove the WAL first and untrack that
file.

Co-authored-by: Isaac <no-reply@databricks.com>
…quence id TODO

sameWorkspaceInfo copied both structs to blank their ForceSendFields before a DeepEqual,
which only works while WorkspaceInfo has no nested structs - true today, not guaranteed.
structdiff.IsEqual handles ForceSendFields recursively and needs no copy; its doc comment
records the same reason DeepEqual is wrong for SDK structs.

Also records a TODO on the operation buffer's sequence ids: they guard against an earlier
update overwriting a later one, which cannot happen when updates for a resource are sent
one at a time from a single goroutine. UpdateOperation requires them today, so the client
cannot simply stop sending them.

Addresses review feedback.

Co-authored-by: Isaac <no-reply@databricks.com>
Comment thread bundle/config/deployment.go Outdated
Comment thread bundle/config/deployment.go Outdated
Comment thread bundle/direct/dstate/dms.go Outdated
Comment thread bundle/direct/dstate/state.go Outdated
Comment thread bundle/direct/dstate/state.go Outdated
Comment thread bundle/direct/dstate/state.go Outdated
Comment thread bundle/direct/dstate/state.go Outdated
Comment thread bundle/direct/dstate/state.go
shreyas-goenka and others added 12 commits September 9, 2026 11:51
Co-authored-by: Isaac <no-reply@databricks.com>
The service reports last_version_id as a string. Parse it once where the deployment is
fetched, so OpenDmsArgs.LastVersionID, DeploymentHistory.LatestVersionID and everything
below carry a number instead of passing the string around and parsing it again in Open.
bundle summary now reports latest_version_id as a number.

OpenDmsArgs.ID becomes DeploymentID: easier to grep than an overloaded ID.

applyDMSState asserts the file it is filling in is the empty shell a recorded deployment
writes, and says "fills in" rather than "replaces", since there is nothing to replace. Its
test cases follow: a file carrying resources is now the refusal case rather than a merge.

getOperationBuffer replaces three hand-rolled lock-read-unlock sequences. RecordFailure
dereferenced the buffer relying on a comment that a version is always open; it checks now.

Also trims the DeploymentID and VersionID comments; VersionID leads with what it is - the
DMS counterpart of Data.Serial, owned by the service, so it never reaches resources.json.

Addresses review feedback.

Co-authored-by: Isaac <no-reply@databricks.com>
The state's features already say whether a deployment records history, so the
StorageBackend type, its two constants and the field mirroring them were a second copy of
that fact. IsDeploymentMetadataService is a map lookup on Data.Features, with an unlocked
twin for the callers that already hold db.mu.

Two places cannot use it, and both are after Finalize resets Data, so the features are no
longer there to read: CompleteVersion and the deferred calls to it in the deploy and
destroy phases. CompleteVersion gates on the operation buffer instead, which exists only
for a recorded deployment and survives the reset, and its callers no longer gate at all -
it already no-ops when no version was created. The old field survived the reset, which is
why this was not visible until the switch.

Co-authored-by: Isaac <no-reply@databricks.com>
…he destroy version

deployment_id and latest_version_id moved onto Deployment itself, so the generated schema
needed regenerating and the two new fields needed descriptions rather than PLACEHOLDER -
what validate-generated was failing on. bundle/dms/no-drift reads them from their new
place.

destroyCore's version completion was still gated on the state's features, which are gone
by then: Finalize resets Data before it runs. That deferred completion and the deployment
record's deletion until after files.Delete had removed the node they depend on.

sameWorkspaceInfo is gone - structdiff.IsEqual compares the pointers directly and handles
nil on either side, so the wrapper only restated it.

Co-authored-by: Isaac <no-reply@databricks.com>
It was dropped along with the comment rewrite in 83f339e, which left its unit case
failing - and unit tests run in every CI cell, so all of them went red regardless of
engine.

Co-authored-by: Isaac <no-reply@databricks.com>
A deployment is usable only when both the workspace node and the service's record exist. A
half-created one - node present, record missing - blocks the bundle here, even though
CreateDeployment already recovers from it. Records the shape proposed in review: a
dms.ReadDeployment(ctx, statePath) that reports a deployment only when both halves are
present, leaving the caller to call CreateDeployment to create or finalize it.

Co-authored-by: Isaac <no-reply@databricks.com>
…_id reason

Probed the real service: it rejects resource_id on a delete with "state must be in
update_mask when 'resource_id' is", and means state with a value. Dropping state from the
mask entirely gets the same error, so no request shape carries an id alongside a cleared
state. The ClearsState comment said something close but not quite that; it now quotes what
the service actually answers, with a TODO to record the deleted resource's id once that is
allowed.

StatusPending gets the follow-up that was only on the PR: use IN_PROGRESS once the service
supports it, since it describes a half-applied recreate better.

Co-authored-by: Isaac <no-reply@databricks.com>
Recovering a half-created deployment and collapsing the two round trips into one are
independent asks; keep them as separate entries.

Co-authored-by: Isaac <no-reply@databricks.com>
Open already decides this from the state's features, so it records the answer instead of
every caller re-deriving it. That also outlives the features themselves - Finalize resets
Data - so CompleteVersion and its callers in the deploy and destroy phases can gate on the
feature again rather than on the operation buffer.

Cleared by Open rather than by reset: the same state is reopened (see bind.go), so a stale
value would misroute a non-recording open, while reset is exactly what it has to survive.

Co-authored-by: Isaac <no-reply@databricks.com>
=== a state depending on a feature this CLI does not recognize is rejected
>>> errcode [CLI] bundle plan
Error: migrating state [TEST_TMP_DIR]/.databricks/bundle/default/resources.json: the deployment state requires features this CLI does not support: future_feature; upgrade to the latest CLI version and see https://docs.databricks.com/aws/en/dev-tools/bundles/state-features#state-features for more information
Error: the deployment state requires features this CLI does not support: future_feature; upgrade to the latest CLI version and see https://docs.databricks.com/aws/en/dev-tools/bundles/state-features#state-features for more information

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

intentionally changed to state version to 2. Because that is what DMS stamps.

shreyas-goenka and others added 2 commits September 9, 2026 14:17
…ainstState

The checks sat in a validatePlan wrapper in cmd/bundle/utils that added nothing but a
hop: everything they read is on the state. They go back to ValidatePlanAgainstState next
to the state they read, and the caller calls it directly.

Also drops a stale comment on templates/deployment-history: the DMS variant is not there to
permit the field, the field is written by bundle init and DMS is what the test exercises.

Co-authored-by: Isaac <no-reply@databricks.com>
…g thing

The plan runs after the recreate, so it shows nothing to do - the title said it showed the
recreate action. The recreate is asserted by the recorded requests above it and the
resource listing below.

Co-authored-by: Isaac <no-reply@databricks.com>
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 7bf1485

Run: 34355098711

Env ❌​FAIL 🤯​MISS 🙈​SKIP ✅​pass 🙈​skip Time
❌​ aws linux 25 314 1 1241 1023 355:26
❌​ aws windows 34 312 1 1140 1046 343:45
❌​ azure linux 9 298 1 1090 1071 355:01
❌​ azure windows 10 236 1 1057 1094 346:39
🤯​ gcp linux 285 1 1098 1075 355:24
❌​ gcp windows 15 319 1 955 1098 347:58
844 interesting tests: 772 MISS, 71 FAIL, 1 SKIP
Test Name aws linux aws windows azure linux azure windows gcp linux gcp windows
🤯​ TestAccept 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/apps/job_permissions 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/apps/job_permissions/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/apps/job_permissions/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/apps/job_permissions/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_doesnot_exist 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_doesnot_exist/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_doesnot_exist/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_doesnot_exist/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_not_deployed 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_not_deployed/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_not_deployed/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_not_deployed/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/config-remote-sync/cli_defaults 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/cli_defaults/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cli_defaults/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cli_defaults/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/config_edits 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/config_edits/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/config_edits/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/config_edits/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/dashboard_etag 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/dashboard_etag/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/dashboard_etag/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/dashboard_etag/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/flushed_cache 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/flushed_cache/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/flushed_cache/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/flushed_cache/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/formatting_preserved 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/formatting_preserved/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/formatting_preserved/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/formatting_preserved/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_fields 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/job_fields/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_fields/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_fields/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_multiple_tasks 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/job_multiple_tasks/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_multiple_tasks/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_multiple_tasks/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_params_variables 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/job_params_variables/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_params_variables/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_params_variables/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_pipeline_task 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/job_pipeline_task/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_pipeline_task/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_pipeline_task/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/json_status_missing_state 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/json_status_missing_state/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/json_status_selectors 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/json_status_selectors/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/json_status_selectors/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/json_status_selectors/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_files 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/multiple_files/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_files/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_files/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_resources 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_json 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/output_json/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_json/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_json/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_no_changes 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/output_no_changes/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_no_changes/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_no_changes/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/pipeline_fields 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/pipeline_fields/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/pipeline_fields/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/pipeline_fields/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/resolve_variables 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/resolve_variables/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/resolve_variables/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/resolve_variables/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_basic 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/select_basic/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_basic/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_basic/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_multiple 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/select_multiple/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_multiple/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_multiple/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_deployment 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/skip_deployment/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_deployment/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_deployment/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_permissions 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/skip_permissions/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_permissions/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_permissions/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/target_override 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/target_override/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/target_override/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/target_override/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/task_rename_revert 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/task_rename_revert/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/task_rename_revert/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/task_rename_revert/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/debug/fetch-repository-info 🤯​M 🤯​M 🤯​M ✅​p 🤯​M 🤯​M
🤯​ TestAccept/bundle/debug/fetch-repository-info/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M 🤯​M
🤯​ TestAccept/bundle/debug/fetch-repository-info/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M ✅​p 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/empty-bundle 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=true/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=true/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=true/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/files/no-snapshot-sync 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/files/no-snapshot-sync/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/files/no-snapshot-sync/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/files/no-snapshot-sync/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/mlops-stacks 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/snapshot-comparison 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/snapshot-comparison/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/spark-jar-task 🤯​M ✅​p 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M ✅​p 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M ✅​p 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M ✅​p 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/alert 🤯​M ✅​p 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/alert/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M ✅​p 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/alert/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M ✅​p 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/catalog 🤯​M ✅​p 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/catalog/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M ✅​p 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/cluster 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/cluster/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/cluster/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/dashboard 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/dashboard/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/READPLAN= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/dashboard/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/READPLAN=1 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/dashboard/DATABRICKS_BUNDLE_ENGINE=terraform/DMS=/READPLAN= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/dashboard/recreation 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/dashboard/recreation/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/READPLAN= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/dashboard/recreation/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/READPLAN=1 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/dashboard/recreation/DATABRICKS_BUNDLE_ENGINE=terraform/DMS=/READPLAN= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/experiment 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/experiment/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/experiment/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/job/generate-and-bind 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/job/generate-and-bind/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/job/generate-and-bind/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/job/job-abort-bind 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/job/job-abort-bind/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/job/job-abort-bind/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/job/job-spark-python-task 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/job/job-spark-python-task/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/job/job-spark-python-task/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/model-serving-endpoint 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/model-serving-endpoint/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/model-serving-endpoint/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
❌​ TestAccept/bundle/deployment/bind/pipelines/recreate 🤯​M ✅​p 🤯​M ❌​F 🤯​M ❌​F
❌​ TestAccept/bundle/deployment/bind/pipelines/recreate/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M ✅​p 🤯​M ❌​F 🤯​M ❌​F
🤯​ TestAccept/bundle/deployment/bind/pipelines/recreate/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/registered-model 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/registered-model/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/registered-model/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/schema 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/schema/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/schema/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/secret-scope 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/secret-scope/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M ✅​p 🤯​M ✅​p 🤯​M ✅​p
Top 50 slowest tests (at least 2 minutes):
duration env testname
41:46 aws windows TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform/DMS=
32:52 aws linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
23:43 aws windows TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
22:35 aws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
19:05 aws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
18:49 gcp linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
17:49 aws windows TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
17:20 aws windows TestAccept/bundle/resources/clusters/deploy/simple/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
16:33 aws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
15:53 gcp windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
15:36 gcp linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
15:21 aws windows TestAccept/bundle/resources/clusters/resize-terminated-fallback/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
15:17 aws linux TestAccept/bundle/resources/clusters/resize-terminated-fallback/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
15:03 gcp windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
14:34 aws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform/DMS=
14:34 aws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
14:00 aws linux TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
13:29 aws windows TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
12:51 aws linux TestAccept/bundle/resources/clusters/deploy/simple/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
12:46 azure windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
12:28 aws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
12:15 aws windows TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
12:00 azure linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
11:39 gcp linux TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
11:33 aws windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
11:22 aws linux TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
10:56 aws linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
10:42 azure windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
10:33 aws windows TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
10:06 aws windows TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
10:06 aws linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
9:52 gcp windows TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=app.yml.tmpl/READPLAN=1
9:49 aws windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
9:49 aws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
9:40 gcp linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
9:35 aws windows TestAccept/bundle/resources/clusters/resize-terminated-fallback/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
9:33 azure windows TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
9:19 gcp linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
9:19 gcp linux TestAccept/bundle/resources/apps/lifecycle-started-omitted/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
9:08 azure linux TestAccept/bundle/quiet-levels/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
9:07 gcp linux TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
8:33 gcp windows TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=job_pydabs_1000_tasks.yml.tmpl/READPLAN=1
8:33 azure windows TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
8:30 aws windows TestAccept/bundle/resources/experiments/basic/DATABRICKS_BUNDLE_ENGINE=terraform/DMS=
8:26 azure linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
8:26 aws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
8:17 aws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform/DMS=
8:12 azure linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true/INPUT_CONFIG=vector_search_endpoint.yml.tmpl/READPLAN=1
8:06 gcp linux TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=job_with_task.yml.tmpl/READPLAN=1
8:04 gcp windows TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
(671 table rows omitted to keep the report under 60000 bytes)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants