Skip to content

Implement Custom Stacks RFC: allow docker:// URIs as stacks - #5492

Draft
johha wants to merge 4 commits into
mainfrom
custom-stacks
Draft

johha wants to merge 4 commits into
mainfrom
custom-stacks

Conversation

@johha

@johha johha commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

Feature: diego_custom_stacks (default: disabled)
Allow buildpack/CNB apps to use docker:// URIs as their stack,
enabling custom root filesystems from container registries.

Stack format: docker://host/path:tag (V3 style)
Internal conversion: docker://host/path#tag (Diego rootfs format)

Changes:

  • Feature flag: diego_custom_stacks (default false)
  • UriUtils: is_custom_stack_uri?, custom_stack_registry_host, parse helpers
  • CustomStackUriConverter: docker:// -> Diego rootfs format
  • BuildpackLifecycleDataMessage: stack_id field, credentials validation
    (strict username+password only for custom stack URIs)
  • BuildpackLifecycleDataModel: encrypted registry credentials storage
    with deep_stringify_keys normalization
  • DB migration: add encrypted credentials columns to buildpack_lifecycle_data
  • BuildpackLifecycleFetcher: skip DB lookup for docker:// stacks
  • LifecycleBase: memoized staging_stack with feature flag enforcement
  • AppBaseLifecycle: credentials update with explicit save trigger
  • AppBuildpackLifecycle/AppBaseLifecycle: persist credentials on create/update
  • StagingActionBuilder: docker rootfs, default lifecycle bundle fallback,
    skip buildpack cache upload for custom stacks
  • LifecycleProtocol: skip buildpack_cache download for custom stacks
  • TaskRecipeBuilder: atomic credential pair resolution for staging/tasks
  • AppRecipeBuilder: atomic credential pair resolution for LRPs
  • Buildpack/CNB DesiredLrpBuilder: docker rootfs with proper requires
  • Buildpack TaskActionBuilder: custom stack support for tasks
  • CNB StagingActionBuilder: optional CNB_STACK_ID via resolve_cnb_stack_id
  • CNB LifecycleData/Protocol: stack_id passthrough
  • AppsController: raise_unless_custom_stacks_enabled! for start/restart
  • AppManifestMessage: validate_custom_stacks_enabled! for manifests
  • BuildCreate: skip stack state validation for custom stacks
  • V3 API docs: feature_flags and lifecycles documentation

Test coverage:

  • 10 custom stacks spec files (uri_utils, converter, validator,
    desired_lrp_builder, message, task_action_builder, cnb_staging,
    feature_flag, lifecycle_data_model, task_recipe_builder)
  • Updated existing specs for compatibility (validator kwargs,
    CNB lifecycle_protocol mocks, controller/manifest test fixtures)Thanks for contributing to cloud_controller_ng. To speed up the process of reviewing your pull request please provide us with:
  • A short explanation of the proposed change:

  • An explanation of the use cases your change solves

  • Links to any other associated PRs

  • I have reviewed the contributing guide

  • I have viewed, signed, and submitted the Contributor License Agreement

  • I have made this pull request to the main branch

  • I have run all the unit tests using bundle exec rake

  • I have run CF Acceptance Tests

FloThinksPi and others added 3 commits September 28, 2026 16:00
Feature: diego_custom_stacks (default: disabled)
Allow buildpack/CNB apps to use docker:// URIs as their stack,
enabling custom root filesystems from container registries.

Stack format: docker://host/path:tag (V3 style)
Internal conversion: docker://host/path#tag (Diego rootfs format)

Changes:
- Feature flag: diego_custom_stacks (default false)
- UriUtils: is_custom_stack_uri?, custom_stack_registry_host, parse helpers
- CustomStackUriConverter: docker:// -> Diego rootfs format
- BuildpackLifecycleDataMessage: stack_id field, credentials validation
  (strict username+password only for custom stack URIs)
- BuildpackLifecycleDataModel: encrypted registry credentials storage
  with deep_stringify_keys normalization
- DB migration: add encrypted credentials columns to buildpack_lifecycle_data
- BuildpackLifecycleFetcher: skip DB lookup for docker:// stacks
- LifecycleBase: memoized staging_stack with feature flag enforcement
- AppBaseLifecycle: credentials update with explicit save trigger
- AppBuildpackLifecycle/AppBaseLifecycle: persist credentials on create/update
- StagingActionBuilder: docker rootfs, default lifecycle bundle fallback,
  skip buildpack cache upload for custom stacks
- LifecycleProtocol: skip buildpack_cache download for custom stacks
- TaskRecipeBuilder: atomic credential pair resolution for staging/tasks
- AppRecipeBuilder: atomic credential pair resolution for LRPs
- Buildpack/CNB DesiredLrpBuilder: docker rootfs with proper requires
- Buildpack TaskActionBuilder: custom stack support for tasks
- CNB StagingActionBuilder: optional CNB_STACK_ID via resolve_cnb_stack_id
- CNB LifecycleData/Protocol: stack_id passthrough
- AppsController: raise_unless_custom_stacks_enabled! for start/restart
- AppManifestMessage: validate_custom_stacks_enabled! for manifests
- BuildCreate: skip stack state validation for custom stacks
- V3 API docs: feature_flags and lifecycles documentation

Test coverage:
- 10 custom stacks spec files (uri_utils, converter, validator,
  desired_lrp_builder, message, task_action_builder, cnb_staging,
  feature_flag, lifecycle_data_model, task_recipe_builder)
- Updated existing specs for compatibility (validator kwargs,
  CNB lifecycle_protocol mocks, controller/manifest test fixtures)

Co-authored-by: 5863788+FloThinksPi@users.noreply.github.com
Bug fixes:
- Guard against nil buildpack_names in BuildpackLifecycleFetcher
- Reject empty buildpack list for custom stacks (vacuous all? bug)
- Move feature flag enforcement out of staging_stack getter
- Narrow rescue scope in AppManifestMessage to ApiError
- Fix nil lifecycle_data crash in controller start/restart

Refactoring:
- Extract ImageCredentialResolver (replaces ~90 lines of duplication)
- Extract CustomStackFallback module (shared resolved_stack_name)
- Convert CustomStackUriConverter to module_function
- Deduplicate UriUtils parsing (is_custom_stack_uri? delegates)
- Use ActiveSupport deep_stringify_keys instead of hand-rolled
- Unify controller custom stacks helper
- Memoize StagingActionBuilder#stack for custom path
- Simplify CNB lifecycle_protocol stack_id assignment
- Fix has_username_and_password? to reject empty strings

Also fixes asg_timestamps migration for Ruby 3.4 compatibility.
Migrate draft specs from Machinist (.make) to FactoryBot (create), and
remove the redundant deep_stringify_keys in credentials=: persistence
round-trips through Oj.dump/Oj.load, which stringifies keys regardless.
Add a spec asserting symbol-keyed credentials read back with string keys.

Co-authored-by: FloThinksPi <5863788+FloThinksPi@users.noreply.github.com>
- Redact lifecycle data.credentials in recorded app events
- Reject a token keyed to the stack registry host (Diego pulls the
  stack via HTTP Basic auth only; a token cannot be used)
- Reject credentials embedded in the stack URI
- Drop the unused process arg from lrp_image_credentials

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants