Repository navigation

bundle: record and read deployment state via DMS - #6094

Merged

shreyas-goenka merged 167 commits into

mainfrom

isaac/pr6052-fixes

Sep 9, 2026

Commits on Sep 4, 2026

bundle: record and read deployment state via DMS

shreyas-goenka
and
Isaac
committed
f0673ef
acceptance/dms: list-resources shows empty tombstone for no-resources

shreyas-goenka
and
Isaac
committed
a9ce3a2
acceptance/dms: assert declined deploy leaves recorded resources unchanged

shreyas-goenka
and
Isaac
committed
4abbea0
acceptance/dms: list-resources coverage for depends-on and emptied-resource

shreyas-goenka
and
Isaac
committed
2a56b8b
acceptance/dms: inline bundle summary into no-drift + declined-deploy, drop dedicated summary test

shreyas-goenka
and
Isaac
committed
af3ffde
libs/dms: document the three SDK gaps the raw client works around; drop redundant operation-upload-message test

shreyas-goenka
and
Isaac
committed
0978bb8
acceptance/dms: fold serialized-plan header assertion into stale-plan, drop serialized-plan
shreyas-goenka
committed
6163285
acceptance/bin: centralize recorded-state reconstruction in dms_resources.get_recorded_state
shreyas-goenka
committed
632b8ea
acceptance/dms: record-failure uses a real bogus node type + a scoped stub instead of fault.py
shreyas-goenka
committed
1b0b45f
libs/dms: gofmt the rawClient doc comment
shreyas-goenka
committed
09f61c3
bundle: make the recorded-state marker the source of truth for DMS; Open reads the workspace client from context

shreyas-goenka
and
Isaac
committed
d42b08e
bundle: move the DMS client and operation buffer into StateDB; drop them from DeploymentBundle

shreyas-goenka
and
Isaac
committed
8dc139d
bundle/dms: StorageBackend enum as source of truth; hide operation buffer in StateDB

shreyas-goenka
and
Isaac
committed
8ec86f8
bundle/dms: rename first-version stamp, inline env predicate, state-gate unbind, restore v3 read

shreyas-goenka
and
Isaac
committed
3683aed
acceptance/dms: record plan JSON, list-resources, and raw recorded state

shreyas-goenka
and
Isaac
committed
c0bc24d
acceptance/dms: record plan JSON, list-resources, and raw recorded state across the suite

shreyas-goenka
and
Isaac
committed
eaa31f4
bundle/dms: guard StorageBackend() gate on direct engine; bump future-version golden

shreyas-goenka
and
Isaac
committed
db6a166
bundle/dms: StateDB owns plan lineage; unbind-specific message; fail version on recording error

shreyas-goenka
and
Isaac
committed
0afcb13
acceptance/dms: drop dedicated readplan test; cover deploy --plan via the readplan option

shreyas-goenka
and
Isaac
committed
0f92c14
bundle/dms: trim redundant comment on the cmdctx workspace-client bridge

shreyas-goenka
and
Isaac
committed
5f10805
acceptance/dms: record plan/list-resources/state for the failure tests

shreyas-goenka
and
Isaac
committed
abd7357
bundle/dms: review fixes - naming, single lineage fields, user-facing wording

shreyas-goenka
and
Isaac
committed
b4d23d2

Commits on Sep 6, 2026

hand edits
shreyas-goenka
committed
97ba52a
bundle/dms: fix hand-edit regressions in unlockedOpen and declined-deploy test

shreyas-goenka
and
Isaac
committed
dad2993
acceptance/dms: serialize deployment_id and version_id in plan output

shreyas-goenka
and
Isaac
committed
bb5abcd
acceptance/dms: assert tombstone state via print_state.py --no-dms

shreyas-goenka
and
Isaac
committed
aadf972
acceptance/dms: assert tombstone state via print_state.py --no-dms; pre-deploy plan in depends-on

shreyas-goenka
and
Isaac
committed
5e7a846
acceptance/dms: put record_deployment_history feature in existing-state WAL header

shreyas-goenka
and
Isaac
committed
ab79f3b
acceptance/dms: drop two racy mid-flow titles in deployment-metadata-change

shreyas-goenka
and
Isaac
committed
bbade26

Commits on Sep 7, 2026

acceptance/dms: drop redundant resources.json guard in print_state.py

shreyas-goenka
and
Isaac
committed
55b1256
acceptance/dms: assert failed delete records its error via list-operations

shreyas-goenka
and
Isaac
committed
41b1adf
bundle/direct/dstate: restore original migrateState feature-version comment

shreyas-goenka
and
Isaac
committed
4563cbc
acceptance/dms: failed-recreate fails naturally via nonexistent catalog (cloud-enabled)

shreyas-goenka
and
Isaac
committed
5435321
bundle/direct/dstate: restore TestEmptyFeatureStateAcceptedWithoutFlippingVersion

shreyas-goenka
and
Isaac
committed
ae90e56
bundle/direct/dstate: use lock/defer-unlock in RecordingError

shreyas-goenka
and
Isaac
committed
6a21a0e
acceptance/dms: cover bind/unbind refusal while recording deployment history

shreyas-goenka
and
Isaac
committed
9806599
libs/dms: fold latestState update into record under one lock

shreyas-goenka
and
Isaac
committed
8d65ab2
acceptance/dms: print plan header via jq + readable plan instead of full -o json

shreyas-goenka
and
Isaac
committed
c0a2e50
libs/dms: shorten record/RecordFailure comments, rename flag to keepExistingUpdateIfAny

shreyas-goenka
and
Isaac
committed
d76883e
acceptance/dms: bind test uses a real job so it records the DMS refusal

shreyas-goenka
and
Isaac
committed
f839765
acceptance/dms: rename emptied-resource to empty-grants, record the known bug, Cloud=false

shreyas-goenka
and
Isaac
committed
f8894d5
acceptance/dms: assert failed permissions update records its error via list-operations

shreyas-goenka
and
Isaac
committed
4b01fa8
acceptance/dms: remove operation-upload-fails test

shreyas-goenka
and
Isaac
committed
7bf906a
acceptance/dms: remove redundant record and failed-update tests

shreyas-goenka
and
Isaac
committed
bdcc887
libs/testserver: gofmt fake_workspace.go after seeding the main catalog

shreyas-goenka
and
Isaac
committed
cc4d4cc
Class A: Revert catalog-existence validation regression

shreyas-goenka
and
Isaac
committed
983777e
Regenerate failed-recreate test golden after fault injection rewrite

shreyas-goenka
and
Isaac
committed
ab2ccc6
Add storage_backend to nostamp.py plan header keys

shreyas-goenka
and
Isaac
committed
7b84ffe
Class B: Pipe bundle plan JSON through nostamp.py in 32 tests

shreyas-goenka
and
Isaac
committed
101fa48
Fix failed-recreate: ensure out.requests.txt cleanup and suppress verbose plan

shreyas-goenka
and
Isaac
committed
ef650bc
Regenerate acceptance test goldens after nostamp.py fixes

shreyas-goenka
and
Isaac
committed
a85053c
libs/testserver: gofmt struct alignment after reverting seeded main catalog

shreyas-goenka
and
Isaac
committed
2488374
acceptance: strip leaked CLI_VERSION commit suffix from regenerated goldens

shreyas-goenka
and
Isaac
committed
eba427d
acceptance: harden CLI_VERSION repl against +sha suffix; skip empty-grants under DMS

shreyas-goenka
and
Isaac
committed
818e972
acceptance/dms: fix failed-recreate recreate trigger; skip readplan+DMS variants

shreyas-goenka
and
Isaac
committed
669fe1f
Enable DMS testing for additional bundle tests with saved plans

shreyas-goenka
and
Isaac
committed
a42fcc3
Enable DMS testing for resources/models/readplan-permissions

shreyas-goenka
and
Isaac
committed
d32d92e
Enable DMS testing for dashboards/change-serialized-dashboard

shreyas-goenka
and
Isaac
committed
ff8c67d
Enable DMS testing for more dashboard tests

shreyas-goenka
and
Isaac
committed
0319c54
Enable DMS testing for additional bundle tests

shreyas-goenka
and
Isaac
committed
20de1a7
Enable DMS testing for bundle/select tests

shreyas-goenka
and
Isaac
committed
5d98f2b
Simplify plan saves: remove redundant tee+nostamp pattern

shreyas-goenka
and
Isaac
committed
5863fda
Fix whitespace: remove trailing blank lines from test.toml files
shreyas-goenka
committed
0f14dd4
Update golden outputs for DMS-enabled tests

shreyas-goenka
and
Isaac
committed
620fcb9
Enable DMS for serialized dashboard test

shreyas-goenka
and
Isaac
committed
15c7df2
Enable DMS for dashboard tests: simple and detect-change

shreyas-goenka
and
Isaac
committed
9442cab
Enable DMS for more dashboard tests

shreyas-goenka
and
Isaac
committed
3dbda98
Add nostamp wrapper function to script.prepare

shreyas-goenka
and
Isaac
committed
0c743d9
Add explicit DMS field validation for serialized plans

shreyas-goenka
and
Isaac
committed
d1e2801
Fix select tests to run with all DMS/READPLAN combinations

shreyas-goenka
and
Isaac
committed
294bcc8
Update select tests to use engine-specific plan files

shreyas-goenka
and
Isaac
committed
4834820
Relax DMS plan validation to allow first-deploy plans

shreyas-goenka
and
Isaac
committed
07dce69
Update plan_file_survives_not_selected golden output

shreyas-goenka
and
Isaac
committed
00c17ce
Set NextVersionId for first deployments in plan generation

shreyas-goenka
and
Isaac
committed
041013c
Add DMS test: plan without DMS rejected when target has DMS

shreyas-goenka
and
Isaac
committed
94f0646
Update golden file for plan-without-dms test

shreyas-goenka
and
Isaac
committed
a0730e2
Simplify plan-without-dms test to use print_requests.py

shreyas-goenka
and
Isaac
committed
b5960ca
Remove unnecessary plan corruption from test

shreyas-goenka
and
Isaac
committed
4024655
acceptance/bundle/deploy/readplan: Enable DMS testing for grants-remove-principal

shreyas-goenka
and
Isaac
committed
6123585
acceptance/bin: consolidate DMS helpers into print_state.py

shreyas-goenka
and
Isaac
committed
e767ed8
acceptance/bundle/deploy/readplan: Fix tests to work with DMS by using raw plan + nostamp pattern

shreyas-goenka
and
Isaac
committed
bde2b0e
acceptance/bundle/deploy/readplan: Regenerate outputs for fixed tests
shreyas-goenka
committed
b0ae7f2
acceptance/readplan: normalize recorded requests with nostamp under DMS

shreyas-goenka
and
Isaac
committed
633e357
acceptance/readplan: nostamp recorded requests in cli-version-mismatch

shreyas-goenka
and
Isaac
committed
e55afd5
acceptance/bin: inline the deployment-history env check

shreyas-goenka
and
Isaac
committed
aab6337
Merge remote-tracking branch 'origin/main' into isaac/pr6052-fixes
shreyas-goenka
committed
d365f85
Revert first-deploy version stamping and the extra plan validation

shreyas-goenka
and
Isaac
committed
92c61c5
acceptance: match PLAN_VERSION repl to plan version 3; drop unused sys import

shreyas-goenka
and
Isaac
committed
37e5d6b
acceptance/bin: document why get_id_recorded lists instead of GET

shreyas-goenka
and
Isaac
committed
83c4fb8
acceptance: stop nostamping plans that feed deploy --plan

shreyas-goenka
and
Isaac
committed
0e452f4
Use state features as the DMS signal in the plan, drop storage_backend

shreyas-goenka
and
Isaac
committed
63b2c7c
Say "bundle target" in the bind/unbind recording errors

shreyas-goenka
and
Isaac
committed
0a2d4bf
acceptance/immutable: drop no-op nostamp

shreyas-goenka
and
Isaac
committed
2a9e879
acceptance/wal: trim the DMS skip comment

shreyas-goenka
and
Isaac
committed
a7d1301
acceptance/select: add missing trailing newline (whitespace lint)

shreyas-goenka
and
Isaac
committed
b3eebb9
acceptance: regenerate goldens stale after the nostamp/plan-file changes

shreyas-goenka
and
Isaac
committed
2bc157e
acceptance: stop nostamping plans captured with &> into a golden

shreyas-goenka
and
Isaac
committed
ed209b3
acceptance/bundle/resources/postgres_projects: Add compute_last_active_time field to plan outputs and project filter

shreyas-goenka
and
Isaac
committed
a76b434
acceptance/bundle/resources/grants/schemas/remove_all: Skip DMS=true variant due to known bug

shreyas-goenka
and
Isaac
committed
26f4329
acceptance/bundle/resources/postgres_*: Add out.requests.txt to Ignore list and fix compute_last_active_time handling

shreyas-goenka
and
Isaac
committed
62d461b
acceptance/bundle/resources/postgres_*: Remove --keep flag from print_requests.py calls

shreyas-goenka
and
Isaac
committed
9cbedde
Revert "acceptance/bundle/resources/postgres_*: Remove --keep flag from print_requests.py calls"
shreyas-goenka
committed
f6a0cfd
Revert "acceptance/bundle/resources/postgres_*: Add out.requests.txt to Ignore list and fix compute_last_active_time handling"
shreyas-goenka
committed
2d1eaa0
Revert "acceptance/bundle/resources/postgres_projects: Add compute_last_active_time field to plan outputs and project filter"
shreyas-goenka
committed
94e4135
acceptance/grants: regenerate remove_all golden after adding test.toml

shreyas-goenka
and
Isaac
committed
d74c107

Commits on Sep 8, 2026

Derive the deployment version from serial; no version for a no-op plan

shreyas-goenka
and
Isaac
committed
6138e60
Rename record_deployment_history to deployment_history

shreyas-goenka
and
Isaac
committed
f488c86
bundle/phases: trim the version-creation comment

shreyas-goenka
and
Isaac
committed
f7ac997
Validate serial against the deployment's last version; cover the stale-version case

shreyas-goenka
and
Isaac
committed
538852e
Reject a stale plan on serial vs the deployment's last version

shreyas-goenka
and
Isaac
committed
26792ff
Say "version" not "serial" in the stale-plan error

shreyas-goenka
and
Isaac
committed
243b3c4
acceptance/apps: run readplan-lifecycle under DMS

shreyas-goenka
and
Isaac
committed
873109b
bundle/direct/dstate: set DeploymentID in Open

shreyas-goenka
and
Isaac
committed
7d1a57a
bundle/direct: co-locate the plan-vs-state validations

shreyas-goenka
and
Isaac
committed
c345ea2
bundle/deployplan: drop deployment_id from the plan, keep plan version at 2

shreyas-goenka
and
Isaac
committed
90b5c3b
acceptance: revert readplan plan fixtures to plan version 2

shreyas-goenka
and
Isaac
committed
7750a32
bundle/direct/dstate: take the serial from the deployment's last_version_id

shreyas-goenka
and
Isaac
committed
cf4f400
cmd/bundle/utils: validate lineage and serial for a plan that has neither

shreyas-goenka
and
Isaac
committed
16ef4b2
bundle/direct/dstate: keep the created version on the state, not behind an accessor

shreyas-goenka
and
Isaac
committed
bc0a598
bundle/phases: compute the update mask only where it is used

shreyas-goenka
and
Isaac
committed
7033b09
cmd/bundle/utils: note the state-path deployment lookup could be one API

shreyas-goenka
and
Isaac
committed
3aee9af
better comment
shreyas-goenka
committed
2c91f6b
nit
shreyas-goenka
committed
3767ff3
bundle/direct/dstate: write no WAL for a recorded deployment

shreyas-goenka
and
Isaac
committed
c3b8e27
Merge remote-tracking branch 'origin/main' into isaac/pr6052-fixes
shreyas-goenka
committed
e9dbe85
bundle/direct/dstate: tidy up the recorded write path

shreyas-goenka
and
Isaac
committed
c3bd4ed
bundle/phases: read the deployment from the state, drop deploymentAndNextVersion

shreyas-goenka
and
Isaac
committed
17ed80e
bundle/direct/dstate: make VersionID the recorded version, int all the way

shreyas-goenka
and
Isaac
committed
a96c805
cmd/bundle/utils: check lineage before the version, set the deployment id directly

shreyas-goenka
and
Isaac
committed
77bd756
cmd/bundle/utils: assign the fetched deployment to its own variables

shreyas-goenka
and
Isaac
committed
f2c5ec6
acceptance: restore serial-mismatch script

shreyas-goenka
and
Isaac
committed
94bf4ed
acceptance/readplan: keep tmp plans out of the file count, run lineage-mismatch under recording

shreyas-goenka
and
Isaac
committed
4776d57
acceptance: run escaped_refs under recording, drop a needless nostamp in auto-migrate-clean

shreyas-goenka
and
Isaac
committed
73ca1d2
acceptance: run two more jobs tests under recording, document the grants badness

shreyas-goenka
and
Isaac
committed
d1aa9bd
Report the recorded version where the state serial used to be read

shreyas-goenka
and
Isaac
committed
c73985d
acceptance: run jobs/big_id under recording

shreyas-goenka
and
Isaac
committed
c7135c1
acceptance/telemetry: name the serial in the recording exclusion reason

shreyas-goenka
and
Isaac
committed
ab50d9d
acceptance/state_present: do not run it with recording

shreyas-goenka
and
Isaac
committed
a7e430a
acceptance: normalize the state serial in lineage-mismatch-after-redeploy

shreyas-goenka
and
Isaac
committed
320cd34
acceptance: say why the serial repl is needed

shreyas-goenka
and
Isaac
committed
9f03439
acceptance: resolve the deployment once, keep the lineage-mismatch fixture

shreyas-goenka
and
Isaac
committed
e1339e2
Scrub the deployment stamp inside print_state.py, run two more tests with recording

shreyas-goenka
and
Isaac
committed
324bd9d
acceptance/change_assets_dir: run it with recording

shreyas-goenka
and
Isaac
committed
fb1eb3c
Merge remote-tracking branch 'origin/main' into isaac/pr6052-fixes
shreyas-goenka
committed
c05cc55
bundle/direct/dstate: keep recorded state in memory so metadata.json gets resource ids

shreyas-goenka
and
Isaac
committed
3aabe82

Commits on Sep 9, 2026

acceptance: commit a DMS plan fixture, run two more tests with recording

shreyas-goenka
and
Isaac
committed
2e8baf1
acceptance/destroy_without_mgmtperms: run it with recording

shreyas-goenka
and
Isaac
committed
8bcdc81
acceptance/change_table_name: run it with recording

shreyas-goenka
and
Isaac
committed
96ffaa2
acceptance/lineage-mismatch: drop the redundant Ignore

shreyas-goenka
and
Isaac
committed
8f047e8
bundle/direct/dstate: add GetSerial, always copy features into the plan

shreyas-goenka
and
Isaac
committed
8af91e8
bundle/direct/dstate: GetSerial, refuse a leftover WAL, tidy DeleteState

shreyas-goenka
and
Isaac
committed
7a4424a
acceptance/dms/existing-state: clear the stale WAL before destroy

shreyas-goenka
and
Isaac
committed
fe3f963
libs/dms: compare workspace info with structdiff.IsEqual, note the sequence id TODO

shreyas-goenka
and
Isaac
committed
0aa6c53
libs/dms: shorten the sameWorkspaceInfo doc comment

shreyas-goenka
and
Isaac
committed
3aac3ab
Carry the deployment version as an int, add getOperationBuffer

shreyas-goenka
and
Isaac
committed
2467a05
bundle/direct/dstate: drop StorageBackend, read the features instead

shreyas-goenka
and
Isaac
committed
7c6a157
update comment
shreyas-goenka
committed
5d35498
better commentn
shreyas-goenka
committed
83f339e
remove wrapper struct for deployment ID and last version ID
shreyas-goenka
committed
33be751
Regenerate the schema for the flattened deployment fields, complete the destroy version

shreyas-goenka
and
Isaac
committed
a4540bf
bundle/direct/dstate: restore the empty-shell assertion in applyDMSState

shreyas-goenka
and
Isaac
committed
0dbffa2
cmd/bundle/utils: note the half-created deployment recovery in the TODO

shreyas-goenka
and
Isaac
committed
93ac803
libs/dms: record two deferred items in the code, correct the resource_id reason

shreyas-goenka
and
Isaac
committed
0883adc
cmd/bundle/utils: split the two deployment-lookup TODOs

shreyas-goenka
and
Isaac
committed
12e10e7
bundle/direct/dstate: cache whether the deployment records history

shreyas-goenka
and
Isaac
committed
9233825
bundle/direct: move the plan-vs-state checks back into ValidatePlanAgainstState

shreyas-goenka
and
Isaac
committed
792146f
acceptance/dms/successful-recreate: fix a title that claimed the wrong thing

shreyas-goenka
and
Isaac
committed
ecf5608