Skip to content

aitools: categorize install errors - #6482

Merged
rclarey merged 10 commits into
mainfrom
aitools-install-error-categories-stacked
Sep 8, 2026
Merged

rclarey merged 10 commits into
mainfrom
aitools-install-error-categories-stacked

Conversation

@rclarey

@rclarey rclarey commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Stacked on #6481

Changes

Categorize aitools install errors, and emit those in telemetry and JSON output

Why

To better understand why installations failed

Tests

Added unit tests

@rclarey
rclarey force-pushed the aitools-install-output-json branch from 583618b to 5dbbee3 Compare September 2, 2026 11:10
@rclarey
rclarey force-pushed the aitools-install-error-categories-stacked branch from 1bec626 to 475a007 Compare September 2, 2026 11:10
@rugpanov

rugpanov commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Review — multi-reviewer pass

Reviewed the incremental diff (against the stacked parent aitools-install-output-json) with several independent reviewers and verified each finding against the code. Design and correctness are sound overall, and CI (including integration tests) is green. One blocking wire-format bug, plus a few low-risk cleanups.

🔴 Blocking — per-agent category serializes under the wrong JSON key

libs/telemetry/protos/aitools_install.go:53

type AitoolsAgentResult struct {
	Agent         AitoolsAgentType     `json:"agent"`
	ErrorCategory AitoolsErrorCategory `json:"errorCategory"`   // should be "error_category"
}

This is the only camelCase JSON tag in the whole libs/telemetry/protos package (111 other tags are snake_case), and it disagrees with its own sibling field on the parent event (error_category, line 71) and the existing precedents in ssh_tunnel.go / bundle_config_remote_sync.go. The marshaled payload becomes agent_results:[{"agent":"CODEX","errorCategory":...}], so the per-agent category won't map to the lumberjack proto's error_category field and is likely dropped on ingestion — which is exactly the per-agent signal this PR adds. Fix: json:"error_category".

🟡 Nice to have

  • A sibling specific-skill failure is left uncategorizedlibs/aitools/installer/installer.go:487. The "experimental skill; use --experimental" specific-failure still returns a plain fmt.Errorf, so it classifies as UNCATEGORIZED, while the two adjacent failures in the same isSpecific branch (not-found :475, version-incompatible :495) were converted to *SkillError. Consider a category (e.g. EXPERIMENTAL_SKILL) or making it a SkillError for consistency.

  • Defensive nil checkcmd/aitools/telemetry.go:58. o.agent == nil in agentResultsField guards against a state that can't occur (every outcome is built from a non-nil plan agent). Per the repo convention on unjustified nil checks, consider removing it (and the synthetic nil test case) or adding a comment on why the invariant might break. (It does mirror the pre-existing pattern in agentsField, so it's at least locally consistent.)

  • No test asserts the telemetry wire format — nothing marshals AitoolsInstallEvent/AitoolsAgentResult to JSON and checks the keys. A serialization assertion would have caught the blocking finding above.

  • UNSUPPORTED_SCOPE on exit-0 skipscmd/aitools/install.go:391,400,555. An ordinary scope-incompatible skip (status skipped, exit 0) still lands in agent_results with UNSUPPORTED_SCOPE. Looks intentional per the doc comment, but conflating expected skips with error rows may inflate error analytics — worth confirming the intended semantics.

⚪ Nits

  • cmd/aitools/install.go:606if o.errorCategory != "" { … } is redundant given the omitempty tag, and inconsistent with the unconditional top-level set just below it.
  • cmd/aitools/install.go:64 — the field skipError holds a category, not an error; mildly misleading next to errorCategory.
  • libs/aitools/installer/errors.go:25SkillError.Error() produces a trailing space when Detail is empty (only reachable from a unit test today).

@anton-107 anton-107 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice change — the classification logic itself looks right to me. I built the branch and ran go test ./cmd/aitools/... ./libs/aitools/... ./libs/telemetry/..., go vet and gofmt: all clean. I couldn't find a correctness bug; the defer closure capture of outcomes/runErr, the topLevelFailure gate, and the SkillError message reconstruction (byte-identical to the strings it replaces) all check out.

Requesting changes on naming only — two new errorCategory keys are camelCase where the surrounding code is snake_case, and one of them is a public output contract for aitools install --output json that can't be renamed after release. Details inline.

The remaining inline comments are non-blocking robustness/consistency notes; take or leave them as you see fit.

Two things I checked and cleared: both deliverySkip branches in planItemFor genuinely mean unsupported scope (mapAgentScope's only ok == false return is the project-scope case), and ReasonNoPluginUNCATEGORIZED is unreachable since plugin-less agents route to deliverySkills.

Comment thread libs/telemetry/protos/aitools_install.go Outdated
Comment thread cmd/aitools/install.go Outdated
Comment thread cmd/aitools/install.go Outdated
Comment thread cmd/aitools/telemetry.go Outdated
Comment thread libs/aitools/installer/errors.go Outdated
Comment thread libs/telemetry/protos/aitools_install.go Outdated
@rclarey
rclarey force-pushed the aitools-install-output-json branch from 5dbbee3 to 0f9f04a Compare September 3, 2026 12:21
@rclarey
rclarey force-pushed the aitools-install-error-categories-stacked branch from 475a007 to 7c97955 Compare September 3, 2026 12:21
@rclarey
rclarey force-pushed the aitools-install-output-json branch from 0f9f04a to af4bb2a Compare September 3, 2026 13:46
@rclarey
rclarey force-pushed the aitools-install-error-categories-stacked branch from 7c97955 to 929fe31 Compare September 3, 2026 13:46
Add JSON output to `aitools install`, driven entirely by flags so the
run is fully non-interactive: require --scope and --agents (erroring and
naming the missing flags otherwise) so no scope prompt, agent picker, or
confirm is shown. executePlan now returns a per-agent outcome (name,
delivery, status, message) that the JSON payload lists. A top-level
failure with no per-agent entry (e.g. a skills-group install failure) is
surfaced in a top-level "error" field; per-agent failures stay in their
agent entry and are not duplicated there (executePlan wraps them so the
two are distinguishable). Once the JSON result is rendered, silence
cobra's text "Error:"/usage output so a failure is not reported twice;
the non-zero exit still comes from returning the run error. Share the
indented-JSON encoder between install and list as renderJSON.

Co-authored-by: Isaac <no-reply@databricks.com>
@rclarey
rclarey force-pushed the aitools-install-error-categories-stacked branch from 929fe31 to 4882a9b Compare September 3, 2026 14:04
@rclarey
rclarey force-pushed the aitools-install-output-json branch from af4bb2a to 047ee54 Compare September 3, 2026 14:04
The changelog validator now requires each fragment to be a single line
starting with a `* ` bullet and ending with a period before the trailing
PR link group. Reformat the install --output json fragment accordingly.

Co-authored-by: Isaac <no-reply@databricks.com>
@rclarey
rclarey force-pushed the aitools-install-error-categories-stacked branch from 4882a9b to 0c9f703 Compare September 4, 2026 10:27
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: ef859bc

Run: 34234494191

Env 🔄​flaky 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux 1 275 19 8:41
🔄​ aws windows 1 1 276 17 6:34
🔄​ azure linux 2 1 272 19 10:03
💚​ azure windows 1 276 17 6:53
💚​ gcp linux 1 275 19 8:34
💚​ gcp windows 1 277 17 7:32
Test Name aws linux aws windows azure linux azure windows gcp linux gcp windows
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🔄​ TestSyncIncrementalSyncPythonNotebookToFile ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p
🔄​ TestFilerReadDir ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestFilerReadDir/files ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
Top 6 slowest tests (at least 2 minutes):
duration env testname
5:48 aws windows TestAccept
3:16 azure windows TestAccept
3:15 gcp windows TestAccept
3:14 azure linux TestFilerWorkspaceFilesExtensionsStat
2:04 azure linux TestImportDir
2:03 azure linux TestFilerWorkspaceFilesExtensionsRead

--output json is meant to emit only the structured JSON document, but the
installer still wrote progress lines ("Using skills version", "Fetching
skills manifest...", "Installed N skills.") to stderr, so a consumer saw
non-JSON interleaved with the result. Mark the context quiet in JSON mode
and route those library messages through cmdio.LogProgress, which respects
it. Text mode is unaffected. Update the acceptance test to use the
non-deprecated `aitools install` and assert only JSON is emitted.

Co-authored-by: Isaac <no-reply@databricks.com>
@rclarey
rclarey force-pushed the aitools-install-error-categories-stacked branch from 5a6cd9f to 49636b6 Compare September 4, 2026 14:34
rclarey and others added 5 commits September 7, 2026 10:48
Classify why an `aitools install` run, or one agent within it, failed
into a stable AitoolsErrorCategory, so install failures can be aggregated
in telemetry without sending any user-authored error text. Introduce
SkillError alongside the existing BlockedError, map both to categories via
classifyInstallError, and record the per-agent categories on the install
event. Surface the per-agent category in the `--output json` result too.

The top-level errorCategory (telemetry and JSON) is set only for a
failure with no per-agent entry; a per-agent failure keeps its category
in its own entry and leaves the top-level category Unspecified, so it is
never counted twice.

Co-authored-by: Isaac <no-reply@databricks.com>
- Rename the JSON/telemetry field errorCategory -> error_category
  (snake_case), matching the CLI's --output json convention and the rest
  of libs/telemetry/protos. Update the acceptance golden accordingly.
- Drop omitempty on AitoolsInstallEvent.ErrorCategory: it is always
  populated (Unspecified on success), so the tag never fired.
- agentResultsField: key on status == outcomeInstalled instead of
  errorCategory == "", and drop the dead o.agent == nil guard (agents
  always come from the validated registry, matching buildInstallOutput).
- SkillError.Error() falls back to Reason when Detail is empty so the
  message stays self-describing.

Co-authored-by: Isaac <no-reply@databricks.com>
Use the non-deprecated `aitools install` command. With progress now
silenced in JSON mode (see the --output json branch), stdout carries only
the JSON document, so the golden no longer has non-JSON text before it.

Co-authored-by: Isaac <no-reply@databricks.com>
Cover the --output json path where a failure is reported per agent rather
than at the top level:
- install-output-json-agent-error: claude-code installs while cursor is
  skipped with an UNSUPPORTED_SCOPE category in its own agents[] entry, so
  the array mixes a success with a per-agent error and the top-level
  error_category stays unset.
- install-output-json-agents-skipped: every named agent (cursor, codex) is
  skipped for scope, giving multiple per-agent categories with no successful
  install and no top-level error.

Both exit non-zero because the agents were named explicitly.

Co-authored-by: Isaac <no-reply@databricks.com>
@rclarey
rclarey force-pushed the aitools-install-error-categories-stacked branch from 2d006b5 to c0840bd Compare September 7, 2026 08:54
@rclarey
rclarey requested a review from anton-107 September 7, 2026 08:55
Base automatically changed from aitools-install-output-json to main September 7, 2026 12:42
@rclarey
rclarey requested a review from rugpanov September 7, 2026 13:14

@anton-107 anton-107 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks — all six threads from my previous pass are properly addressed, and the rename went further than the structs (acceptance golden, the script's explanatory comment, and the changelog fragment are all updated; git grep 'json:"[a-z]*[A-Z]' over cmd/aitools, libs/aitools, libs/telemetry/protos is empty). Both blocking items are resolved. I'm leaving my requested-changes in place for now only pending the installer.go:487 question at the bottom — everything else here is non-blocking.

I re-verified rather than taking the resolutions on trust: gofmt and go vet clean, go test ./cmd/aitools/... ./libs/aitools/... ./libs/telemetry/... green, and go test ./acceptance -run TestAccept/experimental/aitools green, so the goldens genuinely match. Two details I liked: you took the drop the guard fork on the nil check (so agentResultsField and buildInstallOutput now agree), and status == outcomeInstalled is an exact success test rather than an approximation, since outcomeStatus has exactly three values.

Two small leftovers from my point on agentResultsField, inline. Both non-blocking.

One request before merge: could you take another look at Grigory's review comment? Its blocking item was the same camelCase tag, so that's covered, but four of its items got no change and no reply, and one of them looks substantive to me:

  • libs/aitools/installer/installer.go:487"skill %q is experimental; use --experimental to install" is still a plain fmt.Errorf, so classifyInstallError lands it in UNCATEGORIZED, while the two sibling failures in the same isSpecific branch were converted to *SkillError. That's a classification gap in exactly the signal this PR adds — a real, reachable user error that will show up as uncategorized noise. Worth a SkillError reason, or an explicit note that uncategorized is intended for it.
  • cmd/aitools/install.go:65skipError holds a protos.AitoolsErrorCategory, not an error; mildly confusing sitting next to errorCategory.
  • The UNSUPPORTED_SCOPE-on-exit-0 skip semantics question. Your two new acceptance tests document the explicit-agents (exit 1) case nicely, but the auto-detected-agents case still records an error category on a successful run — fine if intended, just never answered.
  • The missing wire-format test, which I've also flagged inline.

Happy for all of these to be follow-ups if you'd rather keep this PR tight — I'd just like the installer.go:487 one either fixed or explicitly deferred rather than dropped, and then I'll clear the block.

Comment thread cmd/aitools/install.go Outdated
Status: string(o.status),
Message: o.message,
}
if o.errorCategory != "" {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking, follow-up to my earlier note on agentResultsField: the trap moved rather than disappeared. Telemetry now keys success off status == outcomeInstalled, but this side still keys off errorCategory != "", so the two consumers of the same slice disagree again — just in the opposite direction from before. A future successful outcome carrying AitoolsErrorCategoryUnspecified would be correctly dropped from agent_results and still emit "error_category": "TYPE_UNSPECIFIED" here, on an agent whose status is installed.

Separately, the if is redundant as written: string("") is "" and the tag is omitempty, so an unconditional assignment is byte-identical on the wire. Keying on o.status != outcomeInstalled (matching telemetry.go) or just dropping the branch both work; the former keeps the two functions aligned.

// user-authored text.
type AitoolsAgentResult struct {
Agent AitoolsAgentType `json:"agent"`
ErrorCategory AitoolsErrorCategory `json:"error_category"`

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: with the tag fixed, nothing guards it. No test marshals AitoolsInstallEvent/AitoolsAgentResult and asserts the key names, so the next hand-edit to these structs can reintroduce exactly the bug we just fixed — and it fails silently, on the ingestion side, where nobody sees it.

libs/telemetry/protos/ssh_tunnel_test.go is the precedent in this package: it marshals the event and compares against a literal JSON string. A ~10-line equivalent covering an event with one agent_results entry would pin both error_category fields. Grigory raised this too.

@rugpanov

rugpanov commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Heads up: lint is red because of a formatting slip. After the rebase onto main, cmd/aitools/install.go (right after the executePlan call, ~line 219) has a blank line made of spaces instead of being empty, so gofmt rejects it. CI shows:

cmd/aitools/install.go:221:1: File is not properly formatted (gofmt)

Fix is just ./task fmt (or gofmt -w cmd/aitools/install.go) and repush — no code change. Everything else looks good.

Address review feedback on PR #6482:
- Add EXPERIMENTAL_SKILL error category so a specific experimental skill
  requested without --experimental no longer classifies as UNCATEGORIZED;
  installer returns a *SkillError with ReasonExperimentalSkill.
- Key buildInstallOutput's category emission on status != outcomeInstalled
  to match agentResultsField, so the JSON and telemetry views agree.
- Rename skipError to skipErrorCategory (it holds a category, not an error).
- Add a telemetry wire-format test pinning the snake_case error_category keys.
- Add an acceptance test for the experimental-skill JSON output.

Co-authored-by: Isaac <no-reply@databricks.com>
@rclarey

rclarey commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

@anton-107 about the "UNSUPPORTED_SCOPE on exit-0 skips" feedback, yes this is intentional. The marker for errors in telemetry is exit code 1, so this doesn't misattribute these skips as errors. There doesn't seem any reason why we shouldn't include "UNSUPPORTED_SCOPE" skips in success telemetry, so I didn't explicitly drop it

@rclarey
rclarey requested a review from anton-107 September 8, 2026 13:50

@anton-107 anton-107 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything I was holding the block on is resolved — clearing it. ef859bcb fixes the installer.go:487 classification gap, and it also picked up the four items from Grigory's review that had been left without a change or a reply, plus the gofmt slip.

I re-verified rather than taking the commit on trust: gofmt -l empty, go vet clean, go test ./cmd/aitools/... ./libs/aitools/... ./libs/telemetry/... green, and go test ./acceptance -run TestAccept/experimental/aitools green — so the new golden genuinely matches.

Details I checked on the new commit:

  • EXPERIMENTAL_SKILL — the message is byte-identical to the fmt.Errorf it replaces (skill "test-exp" is experimental; use --experimental to install), since Detail is set and SkillError.Error() takes the Detail branch. resolveSkills is now internally consistent: all three isSpecific failures are *SkillError with distinct reasons, so the classification gap in exactly the signal this PR adds is closed.
  • The two views now agreebuildInstallOutput keys on o.status != outcomeInstalled, matching agentResultsField. Behaviour is unchanged today: errorCategory is assigned at only two sites (install.go:547 outcomeFailed, install.go:582 outcomeSkipped), and both outcomeInstalled sites leave it zero — so the new predicate is exactly equivalent to the old errorCategory != "", minus the trap.
  • The ReasonNoPlugin"some-future-reason" test swap loses no coverage. The constant is still asserted at plugin_test.go:114, and the classify test now exercises the default branch honestly instead of documenting an unreachable path — InstallPluginForAgent only returns it when agent.Plugin == nil, and planItemFor's default arm (the only one that yields deliveryPlugin) requires a.Plugin != nil.
  • The wire-format test pins what it needs toerror_category on the event and on the agent_results entry, plus the agent_results key itself. That's the guard the camelCase bug slipped through.
  • The new acceptance test covers the path end-to-end, including the empty agents array with the top-level category and exit 1.

No new findings. One thing I looked at and cleared, so it doesn't come back as a follow-up: installer.go:309 ("no agents support project-scoped skills") is a semantically scope-related failure that would classify as UNCATEGORIZED, but it's unreachable from both callers — cmd/aitools/install.go only ever passes agents that planItemFor already routed to deliverySkills (project-incompatible ones become deliverySkip/UNSUPPORTED_SCOPE before the group call), and InstallAllSkills passes InstallOptions{}, whose zero-value scope isn't ScopeProject. So there's nothing to categorize there.

Thanks for taking the UNSUPPORTED_SCOPE-on-exit-0 question head-on too — the exit-code-1 marker is the right answer, and it means the skip rows can't inflate error analytics.

@rclarey
rclarey added this pull request to the merge queue Sep 8, 2026
Merged via the queue into main with commit d472995 Sep 8, 2026
36 checks passed
@rclarey
rclarey deleted the aitools-install-error-categories-stacked branch September 8, 2026 15:07
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: d472995

Run: 34242722673

Env ❌​FAIL 🤯​MISS 🙈​SKIP ✅​pass 🙈​skip Time
❌​ aws linux 49 427 1 1032 992 356:40
❌​ aws windows 64 398 1 952 1015 349:34
❌​ azure linux 49 368 1 916 1038 357:11
❌​ azure windows 30 410 1 799 1061 349:40
❌​ gcp linux 56 447 1 816 1042 356:25
❌​ gcp windows 4 399 1 822 1065 349:55
1048 interesting tests: 876 MISS, 171 FAIL, 1 SKIP
Test Name aws linux aws windows azure linux azure windows gcp linux gcp windows
🤯​ TestAccept 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/apps/job_permissions 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/apps/job_permissions/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/apps/job_permissions/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/apps/job_permissions/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_doesnot_exist 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_doesnot_exist/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_doesnot_exist/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_doesnot_exist/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_not_deployed 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_not_deployed/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_not_deployed/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_not_deployed/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/config-remote-sync/cli_defaults 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/cli_defaults/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cli_defaults/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cli_defaults/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/config_edits 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/config_edits/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/config_edits/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/config_edits/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/dashboard_etag 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/dashboard_etag/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/dashboard_etag/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/dashboard_etag/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/flushed_cache 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/flushed_cache/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/flushed_cache/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/flushed_cache/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/formatting_preserved 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/formatting_preserved/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/formatting_preserved/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/formatting_preserved/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_fields 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/job_fields/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_fields/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_fields/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_multiple_tasks 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/job_multiple_tasks/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_multiple_tasks/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_multiple_tasks/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_params_variables 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/job_params_variables/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_params_variables/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_params_variables/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_pipeline_task 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/job_pipeline_task/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_pipeline_task/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_pipeline_task/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/json_status_missing_state 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/json_status_missing_state/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/json_status_selectors 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/json_status_selectors/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/json_status_selectors/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/json_status_selectors/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_files 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/multiple_files/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_files/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_files/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_resources 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_json 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/output_json/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_json/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_json/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_no_changes 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/output_no_changes/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_no_changes/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_no_changes/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/pipeline_fields 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/pipeline_fields/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/pipeline_fields/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/pipeline_fields/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/resolve_variables 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/resolve_variables/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/resolve_variables/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/resolve_variables/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_basic 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/select_basic/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_basic/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_basic/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_multiple 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/select_multiple/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_multiple/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_multiple/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_deployment 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/skip_deployment/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_deployment/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_deployment/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_permissions 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/skip_permissions/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_permissions/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_permissions/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/target_override 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/target_override/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/target_override/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/target_override/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/task_rename_revert 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/task_rename_revert/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/task_rename_revert/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/task_rename_revert/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/debug/fetch-repository-info 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/debug/fetch-repository-info/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/debug/fetch-repository-info/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/empty-bundle 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=true/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=true/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=true/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/files/no-snapshot-sync 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/files/no-snapshot-sync/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/files/no-snapshot-sync/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/files/no-snapshot-sync/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/mlops-stacks 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
❌​ TestAccept/bundle/deploy/snapshot-comparison 🤯​M 🤯​M ❌​F ❌​F 🤯​M ✅​p
❌​ TestAccept/bundle/deploy/snapshot-comparison/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M ❌​F ❌​F 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/spark-jar-task 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/alert 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/alert/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/alert/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/catalog 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/catalog/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/cluster 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/cluster/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/cluster/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/dashboard 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/dashboard/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/READPLAN= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/dashboard/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/READPLAN=1 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/dashboard/DATABRICKS_BUNDLE_ENGINE=terraform/DMS=/READPLAN= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/dashboard/recreation 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/dashboard/recreation/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/READPLAN= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/dashboard/recreation/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/READPLAN=1 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/dashboard/recreation/DATABRICKS_BUNDLE_ENGINE=terraform/DMS=/READPLAN= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/experiment 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/experiment/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/experiment/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/job/generate-and-bind 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/job/generate-and-bind/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/job/generate-and-bind/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/job/job-abort-bind 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/job/job-abort-bind/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/job/job-abort-bind/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/job/job-spark-python-task 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/job/job-spark-python-task/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/job/job-spark-python-task/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/model-serving-endpoint 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/model-serving-endpoint/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/model-serving-endpoint/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/pipelines/recreate 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/pipelines/recreate/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/pipelines/recreate/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/registered-model 🤯​M 🤯​M 🤯​M ✅​p ✅​p ✅​p
🤯​ TestAccept/bundle/deployment/bind/registered-model/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M ✅​p ✅​p ✅​p
🤯​ TestAccept/bundle/deployment/bind/registered-model/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M ✅​p ✅​p ✅​p
🤯​ TestAccept/bundle/deployment/bind/schema 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/schema/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/schema/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/secret-scope 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/secret-scope/DATABRICKS_BUNDLE_ENGINE=direct/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/secret-scope/DATABRICKS_BUNDLE_ENGINE=terraform/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
🤯​ TestAccept/bundle/deployment/bind/vector_search_endpoint 🤯​M 🤯​M 🤯​M ✅​p 🤯​M ✅​p
Top 50 slowest tests (at least 2 minutes):
duration env testname
23:53 aws windows TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
21:06 aws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
18:44 aws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
16:56 aws windows TestAccept/bundle/resources/clusters/resize-terminated-fallback/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
16:49 gcp windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
16:44 gcp linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
16:38 gcp windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
16:37 azure windows TestAccept/bundle/quiet-levels/DATABRICKS_BUNDLE_ENGINE=terraform/DMS=
14:34 gcp linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
13:36 aws linux TestAccept/bundle/resources/clusters/resize-terminated-fallback/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
13:32 aws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform/DMS=
13:28 azure windows TestAccept/bundle/quiet-levels/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
13:28 azure linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
13:18 aws windows TestAccept/bundle/resources/clusters/deploy/simple/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
13:07 aws windows TestAccept/bundle/resources/clusters/deploy/simple/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
12:20 azure windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
11:48 azure linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
11:47 azure windows TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=schema.yml.tmpl/READPLAN=1
10:58 aws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
10:57 gcp windows TestAccept/bundle/quiet-levels/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
10:50 azure linux TestAccept/bundle/resources/clusters/deploy/simple/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
10:40 aws windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
10:34 aws windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
10:28 gcp windows TestAccept/bundle/resources/apps/lifecycle-started-omitted/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
10:21 gcp windows TestAccept/bundle/quiet-levels/DATABRICKS_BUNDLE_ENGINE=terraform/DMS=
10:09 aws linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
10:09 aws linux TestAccept/bundle/resources/clusters/deploy/simple/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
9:46 gcp linux TestAccept/bundle/resources/models/basic/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
9:38 azure linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
9:37 aws linux TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
9:27 aws linux TestAccept/bundle/resources/clusters/deploy/simple/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
9:19 azure windows TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=app.yml.tmpl/READPLAN=
9:17 aws linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
9:11 gcp windows TestAccept/bundle/quiet-levels/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
9:10 azure linux TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=yes/DMS=/NBOOK=yes/PY=yes/READPLAN=
9:03 azure windows TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
8:51 gcp linux TestAccept/bundle/quiet-levels/DATABRICKS_BUNDLE_ENGINE=terraform/DMS=
8:47 aws linux TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
8:47 gcp linux TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
8:31 aws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
8:29 azure windows TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=pipeline.yml.tmpl/READPLAN=1
8:27 gcp windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
8:24 gcp windows TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=volume_uppercase_name.yml.tmpl/READPLAN=1
8:24 azure linux TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/UV_PYTHON=3.9
8:24 gcp linux TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/DMS=/NBOOK=no/PY=yes/READPLAN=
8:22 gcp linux TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/DMS=/NBOOK=no/PY=no/READPLAN=
8:22 gcp linux TestAccept/bundle/resources/clusters/resize-terminated-fallback/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
8:19 gcp windows TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
8:17 azure windows TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=secret_scope.yml.tmpl/READPLAN=1
8:17 azure windows TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=registered_model.yml.tmpl/READPLAN=1
(873 table rows omitted to keep the report under 60000 bytes)

chenyuem-db pushed a commit to chenyuem-db/cli that referenced this pull request Sep 9, 2026
## Changes
Add `--output json` to `aitools install`. This requires `--scope` and
`--agents` to avoid needing to prompt users on stdout

## Why
Needed for databricks#6482

## Tests
Added unit tests

<!-- If your PR needs to be included in the release notes for next
release,
add a changelog fragment: create .nextchanges/<section>/<name>.md with a
one-line description (e.g. .nextchanges/cli/quickstart.md). See
.nextchanges/README.md. -->

---------

Co-authored-by: Isaac <no-reply@databricks.com>
deco-sdk-tagging Bot added a commit that referenced this pull request Sep 9, 2026
## Release v1.16.0

### CLI

 * `aitools install` now registers the official Claude marketplace if it is missing before installing the Databricks Claude plugin. ([#6485](#6485))
 * `databricks aitools install --output json` now reports an `error_category` for a failed or skipped install (per agent, and at the top level for a failure with no per-agent entry), giving coding agents and CI a stable classification of why an install did not complete. ([#6482](#6482))
 * `databricks aitools install` honors `--output json`, emitting a structured `{scope, agents[...]}` document that reports each agent's delivery and install status so coding agents and CI can consume the result without scraping the text output. JSON mode requires `--scope` and `--agents` so the command runs without interactive prompts. ([#6481](#6481))
 * `databricks bundle sync` now prints sync progress (`Action: PUT`, `Uploaded ...`) by default, matching `databricks sync`. Previously it was silent unless `--output` was passed. Use `--output json` for machine-readable output. ([#6568](#6568))
 * Support major-only DBR runtime versions such as `19.x-scala2.13` in the cluster picker used by `databricks auth login --configure-cluster` and `databricks labs`. ([#6574](#6574))
 * Deprecated the `databricks environments setup-local --constraints-only` flag in favour of the orthogonal `--no-dbconnect`; the flag still works as a hidden alias but is hidden from `--help` and prints a one-line deprecation notice, and will be removed in a later release. ([#6470](#6470))
 * Add orthogonal `--no-constraints` and `--no-dbconnect` flags to `databricks environments setup-local`: `--no-constraints` skips writing the remote Python-version and dependency pins, and `--no-dbconnect` skips the databricks-connect dependency. ([#6464](#6464))
 * `databricks environments setup-local` now reports a distinct `E_PROVISION_CONFLICT` error code in `--output json` when the project's dependencies conflict with the pins written for the target environment, making the requirements unsatisfiable (the same conflict surfaced as a `W_USER_CONSTRAINT_CONFLICT` warning); it is reported after the project files are written, without attempting the doomed provisioning, while other provisioning failures continue to report `E_PROVISION`. ([#6479](#6479))
 * `databricks ssh connect` and `ssh setup` now verify the tunnel's SSH host key against the key the workspace published for the connection, recorded in `~/.databricks/ssh-tunnel-known-hosts/<name>` instead of `~/.ssh/known_hosts`. Reconnecting with a name used before no longer fails with `Host key verification failed` when the compute behind that name changed, and no longer needs a manual `ssh-keygen -R`; host blocks written by an earlier `databricks ssh setup` pick this up once you re-run it. ([#6557](#6557))
 * Stop `databricks ssh connect --ide` from adding a duplicate entry to the IDE's Remote Explorer on every connect: the remote authority is now the SSH host alias alone, instead of embedding the per-instance remote OS user. ([#6550](#6550))
 * Add `--max-clients` and `--server-timeout` flags to `databricks ssh setup`, and `--server-timeout` to `databricks ssh connect`. Both are fixed when the SSH tunnel server job is submitted, so `ssh setup` now serializes them into the generated `ProxyCommand` instead of falling back to the built-in defaults. ([#6547](#6547))
 * `ssh connect` sessions no longer end when the tunnel's websocket connection is lost. The CLI reattaches to the running session and replays the bytes that were missed, so the shell and everything running in it stay intact, and a transient failure to open a replacement connection for the periodic auth refresh is retried rather than ending the session. Reattaching requires an SSH server started by a CLI that supports it; against an older server the connection behaves as before. ([#6558](#6558))

### Bundles

 * Added PyDABs (Python) support for secrets: `Resources.add_secret` and the `secret_mutator` decorator. ([#6553](#6553))
 * Fix job and pipeline environment dependencies with a `*` version wildcard (e.g. `numpy==2.5.*`) being treated as local file paths. ([#6555](#6555))
 * Add the `postgres_snapshot_schedules` bundle resource for managing a Lakebase Postgres branch's automatic-snapshot schedule (direct deployment engine only). ([#6449](#6449))

### Dependency Updates

 * Bump `github.com/databricks/databricks-sdk-go` from v0.175.0 to v0.177.0. ([#6448](#6448))
 * Bump Terraform provider from v1.128.0 to v1.131.0. ([#6544](#6544))
janniklasrose pushed a commit that referenced this pull request Sep 15, 2026
## Changes
Add `--output json` to `aitools install`. This requires `--scope` and
`--agents` to avoid needing to prompt users on stdout

## Why
Needed for #6482

## Tests
Added unit tests

<!-- If your PR needs to be included in the release notes for next
release,
add a changelog fragment: create .nextchanges/<section>/<name>.md with a
one-line description (e.g. .nextchanges/cli/quickstart.md). See
.nextchanges/README.md. -->

---------

Co-authored-by: Isaac <no-reply@databricks.com>
janniklasrose pushed a commit that referenced this pull request Sep 15, 2026
Stacked on #6481 
## Changes
Categorize `aitools install` errors, and emit those in telemetry and
JSON output

## Why
To better understand why installations failed

## Tests
Added unit tests

<!-- If your PR needs to be included in the release notes for next
release,
add a changelog fragment: create .nextchanges/<section>/<name>.md with a
one-line description (e.g. .nextchanges/cli/quickstart.md). See
.nextchanges/README.md. -->

---------

Co-authored-by: Isaac <no-reply@databricks.com>
janniklasrose pushed a commit that referenced this pull request Sep 15, 2026
## Release v1.16.0

### CLI

 * `aitools install` now registers the official Claude marketplace if it is missing before installing the Databricks Claude plugin. ([#6485](#6485))
 * `databricks aitools install --output json` now reports an `error_category` for a failed or skipped install (per agent, and at the top level for a failure with no per-agent entry), giving coding agents and CI a stable classification of why an install did not complete. ([#6482](#6482))
 * `databricks aitools install` honors `--output json`, emitting a structured `{scope, agents[...]}` document that reports each agent's delivery and install status so coding agents and CI can consume the result without scraping the text output. JSON mode requires `--scope` and `--agents` so the command runs without interactive prompts. ([#6481](#6481))
 * `databricks bundle sync` now prints sync progress (`Action: PUT`, `Uploaded ...`) by default, matching `databricks sync`. Previously it was silent unless `--output` was passed. Use `--output json` for machine-readable output. ([#6568](#6568))
 * Support major-only DBR runtime versions such as `19.x-scala2.13` in the cluster picker used by `databricks auth login --configure-cluster` and `databricks labs`. ([#6574](#6574))
 * Deprecated the `databricks environments setup-local --constraints-only` flag in favour of the orthogonal `--no-dbconnect`; the flag still works as a hidden alias but is hidden from `--help` and prints a one-line deprecation notice, and will be removed in a later release. ([#6470](#6470))
 * Add orthogonal `--no-constraints` and `--no-dbconnect` flags to `databricks environments setup-local`: `--no-constraints` skips writing the remote Python-version and dependency pins, and `--no-dbconnect` skips the databricks-connect dependency. ([#6464](#6464))
 * `databricks environments setup-local` now reports a distinct `E_PROVISION_CONFLICT` error code in `--output json` when the project's dependencies conflict with the pins written for the target environment, making the requirements unsatisfiable (the same conflict surfaced as a `W_USER_CONSTRAINT_CONFLICT` warning); it is reported after the project files are written, without attempting the doomed provisioning, while other provisioning failures continue to report `E_PROVISION`. ([#6479](#6479))
 * `databricks ssh connect` and `ssh setup` now verify the tunnel's SSH host key against the key the workspace published for the connection, recorded in `~/.databricks/ssh-tunnel-known-hosts/<name>` instead of `~/.ssh/known_hosts`. Reconnecting with a name used before no longer fails with `Host key verification failed` when the compute behind that name changed, and no longer needs a manual `ssh-keygen -R`; host blocks written by an earlier `databricks ssh setup` pick this up once you re-run it. ([#6557](#6557))
 * Stop `databricks ssh connect --ide` from adding a duplicate entry to the IDE's Remote Explorer on every connect: the remote authority is now the SSH host alias alone, instead of embedding the per-instance remote OS user. ([#6550](#6550))
 * Add `--max-clients` and `--server-timeout` flags to `databricks ssh setup`, and `--server-timeout` to `databricks ssh connect`. Both are fixed when the SSH tunnel server job is submitted, so `ssh setup` now serializes them into the generated `ProxyCommand` instead of falling back to the built-in defaults. ([#6547](#6547))
 * `ssh connect` sessions no longer end when the tunnel's websocket connection is lost. The CLI reattaches to the running session and replays the bytes that were missed, so the shell and everything running in it stay intact, and a transient failure to open a replacement connection for the periodic auth refresh is retried rather than ending the session. Reattaching requires an SSH server started by a CLI that supports it; against an older server the connection behaves as before. ([#6558](#6558))

### Bundles

 * Added PyDABs (Python) support for secrets: `Resources.add_secret` and the `secret_mutator` decorator. ([#6553](#6553))
 * Fix job and pipeline environment dependencies with a `*` version wildcard (e.g. `numpy==2.5.*`) being treated as local file paths. ([#6555](#6555))
 * Add the `postgres_snapshot_schedules` bundle resource for managing a Lakebase Postgres branch's automatic-snapshot schedule (direct deployment engine only). ([#6449](#6449))

### Dependency Updates

 * Bump `github.com/databricks/databricks-sdk-go` from v0.175.0 to v0.177.0. ([#6448](#6448))
 * Bump Terraform provider from v1.128.0 to v1.131.0. ([#6544](#6544))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants