Emit E_PROVISION_CONFLICT for uv sync resolution conflicts - #6479
Merged
Merged
Conversation
rugpanov
marked this pull request as ready for review
September 2, 2026 08:36
rclarey
reviewed
Sep 3, 2026
| // keeps. Gating on the merge signal rather than uv's stderr keeps the code | ||
| // precise: it fires only when a conflict the CLI itself detected is present. | ||
| // | ||
| // The warning means the merged pins are provably unsatisfiable, so a real |
Contributor
There was a problem hiding this comment.
If we already know the pins are unsatisfiable why do we wait for uv sync to fail? Why not error right away?
Contributor
Author
There was a problem hiding this comment.
Good call — now we error right away.
rugpanov
force-pushed
the
setup-local/orthogonal-flags
branch
from
September 3, 2026 15:24
7d3ec5f to
f9ffddd
Compare
rugpanov
force-pushed
the
setup-local/provision-conflict
branch
2 times, most recently
from
September 4, 2026 15:03
e6fe8cd to
dc380a0
Compare
`databricks environments setup-local` wrapped every provisioning failure as E_PROVISION. The extension's recovery flow needs to tell a dependency version conflict — the project's dependencies can't be satisfied against the pins this command wrote for the target environment — apart from a generic failure it can't fix by relaxing constraints. The merge phase already detects a provable version conflict and records it as the W_USER_CONSTRAINT_CONFLICT warning. When that fires, uv sync would deterministically fail to resolve, so report the new E_PROVISION_CONFLICT right after writing the project files (diskMutated=true, at the provision phase) instead of spending a doomed Python install and sync. Gating on the CLI's own detection keeps the code precise — no uv-stderr matching and no false positive on an unrelated sync failure. Adds the matching telemetry enum value and its coverage case, a unit test asserting the fail-fast path reports the conflict without invoking provisioning, and an acceptance golden. Co-authored-by: Isaac <no-reply@databricks.com>
rugpanov
force-pushed
the
setup-local/provision-conflict
branch
from
September 4, 2026 15:13
dc380a0 to
6c07a2b
Compare
Collaborator
Integration test reportCommit: 6c07a2b
Top 6 slowest tests (at least 2 minutes):
|
rclarey
approved these changes
Sep 7, 2026
deco-sdk-tagging Bot
added a commit
that referenced
this pull request
Sep 9, 2026
## Release v1.16.0 ### CLI * `aitools install` now registers the official Claude marketplace if it is missing before installing the Databricks Claude plugin. ([#6485](#6485)) * `databricks aitools install --output json` now reports an `error_category` for a failed or skipped install (per agent, and at the top level for a failure with no per-agent entry), giving coding agents and CI a stable classification of why an install did not complete. ([#6482](#6482)) * `databricks aitools install` honors `--output json`, emitting a structured `{scope, agents[...]}` document that reports each agent's delivery and install status so coding agents and CI can consume the result without scraping the text output. JSON mode requires `--scope` and `--agents` so the command runs without interactive prompts. ([#6481](#6481)) * `databricks bundle sync` now prints sync progress (`Action: PUT`, `Uploaded ...`) by default, matching `databricks sync`. Previously it was silent unless `--output` was passed. Use `--output json` for machine-readable output. ([#6568](#6568)) * Support major-only DBR runtime versions such as `19.x-scala2.13` in the cluster picker used by `databricks auth login --configure-cluster` and `databricks labs`. ([#6574](#6574)) * Deprecated the `databricks environments setup-local --constraints-only` flag in favour of the orthogonal `--no-dbconnect`; the flag still works as a hidden alias but is hidden from `--help` and prints a one-line deprecation notice, and will be removed in a later release. ([#6470](#6470)) * Add orthogonal `--no-constraints` and `--no-dbconnect` flags to `databricks environments setup-local`: `--no-constraints` skips writing the remote Python-version and dependency pins, and `--no-dbconnect` skips the databricks-connect dependency. ([#6464](#6464)) * `databricks environments setup-local` now reports a distinct `E_PROVISION_CONFLICT` error code in `--output json` when the project's dependencies conflict with the pins written for the target environment, making the requirements unsatisfiable (the same conflict surfaced as a `W_USER_CONSTRAINT_CONFLICT` warning); it is reported after the project files are written, without attempting the doomed provisioning, while other provisioning failures continue to report `E_PROVISION`. ([#6479](#6479)) * `databricks ssh connect` and `ssh setup` now verify the tunnel's SSH host key against the key the workspace published for the connection, recorded in `~/.databricks/ssh-tunnel-known-hosts/<name>` instead of `~/.ssh/known_hosts`. Reconnecting with a name used before no longer fails with `Host key verification failed` when the compute behind that name changed, and no longer needs a manual `ssh-keygen -R`; host blocks written by an earlier `databricks ssh setup` pick this up once you re-run it. ([#6557](#6557)) * Stop `databricks ssh connect --ide` from adding a duplicate entry to the IDE's Remote Explorer on every connect: the remote authority is now the SSH host alias alone, instead of embedding the per-instance remote OS user. ([#6550](#6550)) * Add `--max-clients` and `--server-timeout` flags to `databricks ssh setup`, and `--server-timeout` to `databricks ssh connect`. Both are fixed when the SSH tunnel server job is submitted, so `ssh setup` now serializes them into the generated `ProxyCommand` instead of falling back to the built-in defaults. ([#6547](#6547)) * `ssh connect` sessions no longer end when the tunnel's websocket connection is lost. The CLI reattaches to the running session and replays the bytes that were missed, so the shell and everything running in it stay intact, and a transient failure to open a replacement connection for the periodic auth refresh is retried rather than ending the session. Reattaching requires an SSH server started by a CLI that supports it; against an older server the connection behaves as before. ([#6558](#6558)) ### Bundles * Added PyDABs (Python) support for secrets: `Resources.add_secret` and the `secret_mutator` decorator. ([#6553](#6553)) * Fix job and pipeline environment dependencies with a `*` version wildcard (e.g. `numpy==2.5.*`) being treated as local file paths. ([#6555](#6555)) * Add the `postgres_snapshot_schedules` bundle resource for managing a Lakebase Postgres branch's automatic-snapshot schedule (direct deployment engine only). ([#6449](#6449)) ### Dependency Updates * Bump `github.com/databricks/databricks-sdk-go` from v0.175.0 to v0.177.0. ([#6448](#6448)) * Bump Terraform provider from v1.128.0 to v1.131.0. ([#6544](#6544))
janniklasrose
pushed a commit
that referenced
this pull request
Sep 15, 2026
## Why `databricks environments setup-local` wrapped every provisioning failure as `E_PROVISION`. The VPEX extension's recovery flow needs to distinguish a **dependency version conflict** — the project's dependencies can't be satisfied against the pins this command wrote for the target environment — from a generic failure it can't fix by relaxing constraints (a build-backend error, a permissions problem, a transport error). Fixes DECO-28365. ## What - New error code **`E_PROVISION_CONFLICT`** (`libs/localenv/result.go`), reported from the provision phase. - The merge phase already detects a **provable** version conflict between the user's dependencies and the environment pins and records it as the `W_USER_CONSTRAINT_CONFLICT` warning (`libs/localenv/warnings.go`, PEP 440 interval math on provably-disjoint ranges, scanning every requirement uv locks). - When that warning is present, `uv sync` would deterministically fail to resolve, so the pipeline **fails fast**: right after the merge writes the project files it reports `E_PROVISION_CONFLICT` (`failurePhase=provision`, `diskMutated=true`) **without** attempting the Python install or `uv sync` (`libs/localenv/pipeline.go`, `hasConstraintConflictWarning`). Every other provisioning failure keeps `E_PROVISION`. - Gating on the CLI's own merge detection (rather than parsing uv's stderr) keeps the code **precise** — no false positive on an unavailable package or an unrelated sync failure — and needs no brittle stderr matching. `diskMutated=true` is preserved because the constraints are written before the check, which the extension's recovery flow relies on. - `--dry-run` is unchanged: it computes a plan and does not evaluate provisioning, so the same conflict surfaces there as the `W_USER_CONSTRAINT_CONFLICT` warning (`ok=true`) — a test pins that intended divergence. - Telemetry: adds `SetupLocalErrorCodeProvisionConflict` and the mapping case (`cmd/environments/telemetry.go`, `libs/telemetry/protos/setup_local.go`), kept exhaustive by the linter and `TestErrorCodeCoversLocalenv`. ### Consumer-facing note Adding `E_PROVISION_CONFLICT` is additive, but this exact scenario previously surfaced as `E_PROVISION`. Any consumer that special-cased `E_PROVISION` for the conflict case will see the new code. The universe lumberjack proto should add the matching `SetupLocalErrorCode` enum value (ingestion ignores unknown values, so the two changes can land in either order). ## Testing - **Unit** (`libs/localenv/pipeline_test.go`): a conflict warning makes the run report `E_PROVISION_CONFLICT` at the provision phase with `diskMutated=true` and **without** invoking Python install or sync (`recordingPM` records neither); `--dry-run` with the same pins reports `ok=true` with the warning and no error, writing nothing. `TestPipelineRetainsFallbackResolutionWhenProvisioningFails` still covers a generic sync failure → `E_PROVISION`. - **Acceptance** (`acceptance/localenv/provision-conflict/`): a real `setup-local` run where the user pins `pip==24.0` and the remote constraints pin `pip<24`; the merge flags the conflict and the command reports `E_PROVISION_CONFLICT` before provisioning. Cross-platform (only preflight `uv --version` runs; the doomed sync never does), so it runs on every OS with no fake binary. - `gofmt`, `go vet`, `golangci-lint`, and the local `check-changelog` pass. This pull request and its description were written by Isaac. Co-authored-by: Isaac <no-reply@databricks.com>
janniklasrose
pushed a commit
that referenced
this pull request
Sep 15, 2026
## Release v1.16.0 ### CLI * `aitools install` now registers the official Claude marketplace if it is missing before installing the Databricks Claude plugin. ([#6485](#6485)) * `databricks aitools install --output json` now reports an `error_category` for a failed or skipped install (per agent, and at the top level for a failure with no per-agent entry), giving coding agents and CI a stable classification of why an install did not complete. ([#6482](#6482)) * `databricks aitools install` honors `--output json`, emitting a structured `{scope, agents[...]}` document that reports each agent's delivery and install status so coding agents and CI can consume the result without scraping the text output. JSON mode requires `--scope` and `--agents` so the command runs without interactive prompts. ([#6481](#6481)) * `databricks bundle sync` now prints sync progress (`Action: PUT`, `Uploaded ...`) by default, matching `databricks sync`. Previously it was silent unless `--output` was passed. Use `--output json` for machine-readable output. ([#6568](#6568)) * Support major-only DBR runtime versions such as `19.x-scala2.13` in the cluster picker used by `databricks auth login --configure-cluster` and `databricks labs`. ([#6574](#6574)) * Deprecated the `databricks environments setup-local --constraints-only` flag in favour of the orthogonal `--no-dbconnect`; the flag still works as a hidden alias but is hidden from `--help` and prints a one-line deprecation notice, and will be removed in a later release. ([#6470](#6470)) * Add orthogonal `--no-constraints` and `--no-dbconnect` flags to `databricks environments setup-local`: `--no-constraints` skips writing the remote Python-version and dependency pins, and `--no-dbconnect` skips the databricks-connect dependency. ([#6464](#6464)) * `databricks environments setup-local` now reports a distinct `E_PROVISION_CONFLICT` error code in `--output json` when the project's dependencies conflict with the pins written for the target environment, making the requirements unsatisfiable (the same conflict surfaced as a `W_USER_CONSTRAINT_CONFLICT` warning); it is reported after the project files are written, without attempting the doomed provisioning, while other provisioning failures continue to report `E_PROVISION`. ([#6479](#6479)) * `databricks ssh connect` and `ssh setup` now verify the tunnel's SSH host key against the key the workspace published for the connection, recorded in `~/.databricks/ssh-tunnel-known-hosts/<name>` instead of `~/.ssh/known_hosts`. Reconnecting with a name used before no longer fails with `Host key verification failed` when the compute behind that name changed, and no longer needs a manual `ssh-keygen -R`; host blocks written by an earlier `databricks ssh setup` pick this up once you re-run it. ([#6557](#6557)) * Stop `databricks ssh connect --ide` from adding a duplicate entry to the IDE's Remote Explorer on every connect: the remote authority is now the SSH host alias alone, instead of embedding the per-instance remote OS user. ([#6550](#6550)) * Add `--max-clients` and `--server-timeout` flags to `databricks ssh setup`, and `--server-timeout` to `databricks ssh connect`. Both are fixed when the SSH tunnel server job is submitted, so `ssh setup` now serializes them into the generated `ProxyCommand` instead of falling back to the built-in defaults. ([#6547](#6547)) * `ssh connect` sessions no longer end when the tunnel's websocket connection is lost. The CLI reattaches to the running session and replays the bytes that were missed, so the shell and everything running in it stay intact, and a transient failure to open a replacement connection for the periodic auth refresh is retried rather than ending the session. Reattaching requires an SSH server started by a CLI that supports it; against an older server the connection behaves as before. ([#6558](#6558)) ### Bundles * Added PyDABs (Python) support for secrets: `Resources.add_secret` and the `secret_mutator` decorator. ([#6553](#6553)) * Fix job and pipeline environment dependencies with a `*` version wildcard (e.g. `numpy==2.5.*`) being treated as local file paths. ([#6555](#6555)) * Add the `postgres_snapshot_schedules` bundle resource for managing a Lakebase Postgres branch's automatic-snapshot schedule (direct deployment engine only). ([#6449](#6449)) ### Dependency Updates * Bump `github.com/databricks/databricks-sdk-go` from v0.175.0 to v0.177.0. ([#6448](#6448)) * Bump Terraform provider from v1.128.0 to v1.131.0. ([#6544](#6544))
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
databricks environments setup-localwrapped every provisioning failure asE_PROVISION. The VPEX extension's recovery flow needs to distinguish adependency version conflict — the project's dependencies can't be satisfied
against the pins this command wrote for the target environment — from a generic
failure it can't fix by relaxing constraints (a build-backend error, a permissions
problem, a transport error). Fixes DECO-28365.
What
E_PROVISION_CONFLICT(libs/localenv/result.go), reportedfrom the provision phase.
user's dependencies and the environment pins and records it as the
W_USER_CONSTRAINT_CONFLICTwarning (libs/localenv/warnings.go, PEP 440interval math on provably-disjoint ranges, scanning every requirement uv locks).
uv syncwould deterministically fail to resolve,so the pipeline fails fast: right after the merge writes the project files it
reports
E_PROVISION_CONFLICT(failurePhase=provision,diskMutated=true)without attempting the Python install or
uv sync(
libs/localenv/pipeline.go,hasConstraintConflictWarning). Every otherprovisioning failure keeps
E_PROVISION.the code precise — no false positive on an unavailable package or an
unrelated sync failure — and needs no brittle stderr matching.
diskMutated=trueis preserved because the constraints are written before the check, which the
extension's recovery flow relies on.
--dry-runis unchanged: it computes a plan and does not evaluate provisioning,so the same conflict surfaces there as the
W_USER_CONSTRAINT_CONFLICTwarning(
ok=true) — a test pins that intended divergence.SetupLocalErrorCodeProvisionConflictand the mapping case(
cmd/environments/telemetry.go,libs/telemetry/protos/setup_local.go), keptexhaustive by the linter and
TestErrorCodeCoversLocalenv.Consumer-facing note
Adding
E_PROVISION_CONFLICTis additive, but this exact scenario previouslysurfaced as
E_PROVISION. Any consumer that special-casedE_PROVISIONfor theconflict case will see the new code. The universe lumberjack proto should add the
matching
SetupLocalErrorCodeenum value (ingestion ignores unknown values, so thetwo changes can land in either order).
Testing
libs/localenv/pipeline_test.go): a conflict warning makes the runreport
E_PROVISION_CONFLICTat the provision phase withdiskMutated=trueandwithout invoking Python install or sync (
recordingPMrecords neither);--dry-runwith the same pins reportsok=truewith the warning and no error,writing nothing.
TestPipelineRetainsFallbackResolutionWhenProvisioningFailsstill covers a generic sync failure →
E_PROVISION.acceptance/localenv/provision-conflict/): a realsetup-localrun where the user pins
pip==24.0and the remote constraints pinpip<24; themerge flags the conflict and the command reports
E_PROVISION_CONFLICTbeforeprovisioning. Cross-platform (only preflight
uv --versionruns; the doomed syncnever does), so it runs on every OS with no fake binary.
gofmt,go vet,golangci-lint, and the localcheck-changelogpass.This pull request and its description were written by Isaac.