Skip to content

Normalize jobs PyDABs acceptance surface and drop coverage exception - #6548

Merged
Sankalp-Mittal merged 1 commit into
sankalp-mittal/pydabs-generate-all-resourcesfrom
sankalp-mittal/pydabs-jobs-tests-normalize
Sep 8, 2026
Merged

Sankalp-Mittal merged 1 commit into
sankalp-mittal/pydabs-generate-all-resourcesfrom
sankalp-mittal/pydabs-jobs-tests-normalize

Conversation

@Sankalp-Mittal

@Sankalp-Mittal Sankalp-Mittal commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Stacked on top of #6528.

test_python_support_coverage requires every PyDABs resource to have an acceptance/bundle/python/<plural>-support/ fixture. jobs was the lone exception: it predates the <plural>-support convention and was carried in a _LACKING shrink-only allowlist ("covered across the suite instead").

This PR covers that gap by:

  • adding jobs-support test
  • removing the _LACKING allowlist

Testing

  • go test ./acceptance -run 'TestAccept/bundle/python/jobs-support' passes without -update across all four variants (both engines × both wheel versions) — deterministic.
  • test_python_support_coverage passes for all 23 resources with no exceptions.

This pull request and its description were written by Isaac.

@Sankalp-Mittal
Sankalp-Mittal marked this pull request as draft September 7, 2026 13:10
@Sankalp-Mittal
Sankalp-Mittal marked this pull request as ready for review September 7, 2026 13:16
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 2bd8223

Run: 34243671785

Env 🔄​flaky 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux 1 275 33 11:04
💚​ aws windows 1 277 31 7:31
💚​ azure linux 1 274 33 14:54
🔄​ azure windows 2 1 274 31 10:39
🔄​ gcp linux 1 1 274 33 15:36
💚​ gcp windows 1 277 31 8:27
Test Name aws linux aws windows azure linux azure windows gcp linux gcp windows
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🔄​ TestSyncFullFileSync ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
🔄​ TestSyncIncrementalSyncPythonNotebookToFile ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
🔄​ TestSyncNestedFolderDoesntFailOnNonEmptyDirectory ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p
Top 24 slowest tests (at least 2 minutes):
duration env testname
5:57 aws linux TestFilerWorkspaceFilesExtensionsReadDir
4:39 gcp linux TestExportDir
4:32 gcp windows TestFilerWorkspaceFilesExtensionsReadDir
4:05 azure linux TestFilerWorkspaceFilesExtensionsDelete
3:46 azure linux TestFilerWorkspaceFilesExtensionsRead
3:43 azure linux TestFilerWorkspaceFilesExtensionsReadDir
3:22 gcp linux TestFilerRecursiveDelete/workspace_files
3:21 azure windows TestFilerRecursiveDelete/workspace_files_extensions
3:12 gcp windows TestAccept
3:12 aws windows TestAccept
3:10 azure windows TestAccept
3:06 azure linux TestFilerRecursiveDelete/workspace_files
2:59 aws linux TestWorkspaceFilesExtensions_ExportFormatIsPreserved/jupyter_scala
2:56 aws windows TestFilerWorkspaceFilesExtensionsReadDir
2:33 aws windows TestFilerWorkspaceFilesExtensionsRead
2:32 azure windows TestExportDir
2:31 azure linux TestSyncFullFileSync
2:29 gcp windows TestFilerReadWrite/workspace_files_extensions
2:28 azure windows TestLock
2:17 gcp linux TestFilerReadWrite/workspace_files
2:16 gcp linux TestFilerReadDir/workspace_files_extensions
2:04 gcp linux TestFilerWorkspaceFilesExtensionsReadDir
2:03 azure linux TestFilerWorkspaceNotebook/pythonJupyterNb.ipynb
2:02 gcp linux TestImportDirDoesNotOverwrite

@Sankalp-Mittal
Sankalp-Mittal force-pushed the sankalp-mittal/pydabs-jobs-tests-normalize branch from db909a0 to 37ecd9c Compare September 8, 2026 14:23
Add the standard jobs-support/ fixture so jobs matches every other PyDABs
resource, then remove the _LACKING exception list from
test_python_support_coverage. The guard now uniformly requires a
<plural>-support/ fixture for every resource.

Co-authored-by: Isaac <no-reply@databricks.com>
@Sankalp-Mittal
Sankalp-Mittal force-pushed the sankalp-mittal/pydabs-jobs-tests-normalize branch from 37ecd9c to 2bd8223 Compare September 8, 2026 15:14
@Sankalp-Mittal
Sankalp-Mittal added this pull request to the merge queue Sep 8, 2026
Merged via the queue into main with commit 897e551 Sep 8, 2026
32 checks passed
@Sankalp-Mittal
Sankalp-Mittal deleted the sankalp-mittal/pydabs-jobs-tests-normalize branch September 8, 2026 16:18
chenyuem-db pushed a commit to chenyuem-db/cli that referenced this pull request Sep 9, 2026
…atabricks#6553)

Stacked on databricks#6548.

## Changes

Enables PyDABs codegen to model the SDK `time.Time` type, which unblocks
generating `resources.Secret` (its `expire_time` field is a
`*time.Time`) and drops it from the codegen denylist.

`time.Time` is mapped to Python `str` — an RFC3339 timestamp string.
This mirrors how the Go side already models the type at its own config
boundary (`libs/dyn/convert/sdk_native_types.go` treats `time.Time` as a
string), so a PyDABs value flows unchanged through the same Go
normalization pipeline that `databricks.yml` uses.

Regenerated output adds the `secrets` module and its wiring; the
`databricks.bundles.core` public-API snapshot update is purely additive
(`add_secret`, `secret_mutator`, `secrets`).

## Tests

- New `acceptance/bundle/python/secrets-support/` fixture (required by
`test_python_support_coverage`): loads a Secret from YAML and Python,
runs a `secret_mutator` over both, and exercises `expire_time` (RFC3339
string) end to end. `value` is a default-less variable reference
supplied at validate time, as the secret value validator requires;
direct-engine only. Passes with and without `-update`.
- `pydabs-test` 423 passed; codegen unit tests 37 passed.

## Notes

- A changelog fragment (`.nextchanges/bundles/`) is still to be added.

This pull request and its description were written by Isaac.

---------

Co-authored-by: Isaac <no-reply@databricks.com>
janniklasrose pushed a commit that referenced this pull request Sep 15, 2026
…6548)

## Summary

Stacked on top of #6528.

`test_python_support_coverage` requires every PyDABs resource to have an
`acceptance/bundle/python/<plural>-support/` fixture. `jobs` was the
lone exception: it predates the `<plural>-support` convention and was
carried in a `_LACKING` shrink-only allowlist ("covered across the suite
instead").

This PR covers that gap by:
- adding `jobs-support` test
- removing the `_LACKING` allowlist

## Testing

- `go test ./acceptance -run 'TestAccept/bundle/python/jobs-support'`
passes without `-update` across all four variants (both engines × both
wheel versions) — deterministic.
- `test_python_support_coverage` passes for all 23 resources with no
exceptions.

This pull request and its description were written by Isaac.

Co-authored-by: Isaac <no-reply@databricks.com>
janniklasrose pushed a commit that referenced this pull request Sep 15, 2026
…6553)

Stacked on #6548.

## Changes

Enables PyDABs codegen to model the SDK `time.Time` type, which unblocks
generating `resources.Secret` (its `expire_time` field is a
`*time.Time`) and drops it from the codegen denylist.

`time.Time` is mapped to Python `str` — an RFC3339 timestamp string.
This mirrors how the Go side already models the type at its own config
boundary (`libs/dyn/convert/sdk_native_types.go` treats `time.Time` as a
string), so a PyDABs value flows unchanged through the same Go
normalization pipeline that `databricks.yml` uses.

Regenerated output adds the `secrets` module and its wiring; the
`databricks.bundles.core` public-API snapshot update is purely additive
(`add_secret`, `secret_mutator`, `secrets`).

## Tests

- New `acceptance/bundle/python/secrets-support/` fixture (required by
`test_python_support_coverage`): loads a Secret from YAML and Python,
runs a `secret_mutator` over both, and exercises `expire_time` (RFC3339
string) end to end. `value` is a default-less variable reference
supplied at validate time, as the secret value validator requires;
direct-engine only. Passes with and without `-update`.
- `pydabs-test` 423 passed; codegen unit tests 37 passed.

## Notes

- A changelog fragment (`.nextchanges/bundles/`) is still to be added.

This pull request and its description were written by Isaac.

---------

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants