Skip to content

Prepare CI for Pulsar 5.x support with separate broker and connector images - #2153

Open
lhotari wants to merge 1 commit into
masterfrom
fix-integration-test-defaults
Open

lhotari wants to merge 1 commit into
masterfrom
fix-integration-test-defaults

Conversation

@lhotari

@lhotari lhotari commented Sep 18, 2026 •

Copy link
Copy Markdown
Member

Motivation

Prepare CI for Pulsar 5.x support by removing the integration tests' dependency on pulsar-all as the broker image. Pulsar 5.x will use a separate distribution of Pulsar IO connectors, so sink/source tests need to obtain connector files independently of the broker image and version.

This change adds that configuration and image assembly support while retaining apachepulsar/pulsar:latest for the broker and apachepulsar/pulsar-all:latest as the current connector source. CI can switch to the future connector image, including a scratch image with an independent release tag, through repository variables when it becomes available.

Modifications

  • Default the broker to apachepulsar/pulsar:latest and the connector source to apachepulsar/pulsar-all:latest.
  • Add PULSAR_IO_IMAGE and PULSAR_IO_IMAGE_VERSION; the connector tag defaults to PULSAR_VERSION but can follow an independent release cycle.
  • Use a separate Dockerfile target for sink/source tests that copies files from the connector image into the broker test image. The source image is never executed and can be a scratch image. PULSAR_IO_CONNECTORS_DIR configures the source path, defaulting to /pulsar/connectors.
  • Expose the image, connector tag, and connector directory settings through GitHub Actions repository variables. Preserve staging defaults for StreamNative bot runs.

Verifying this change

  • After rebasing onto 48d6ff9 (the policies environment fix in Fix policies tests with Pulsar 5 bundles #2155), the policies suite passed against docker-proxy.streamnative.io/snstage/pulsar-cloud:5.0.0-M1-SNAPSHOT without any script-level namespace-bundle override.
  • Shell syntax and command checks passed for all seven modes, including connector target selection and use of the upstream policies environment file.
  • Image composition validation from before the rebase: general, token, TLS, function, and package suites passed against apachepulsar/pulsar:latest; sink tests passed with connectors copied from apachepulsar/pulsar-all:latest; source tests passed with connectors copied from a scratch image using an independent fixture-v1 tag and / source directory.
  • The regular Dockerfile target built successfully with an intentionally invalid connector image, confirming that non-IO tests do not fetch it. All 35 image/tag configuration combinations passed.
  • Actionlint reported only the existing path-ignores and actions/checkout@v2 diagnostics; no new findings.

Documentation

  • doc — Documented image settings, tag defaults, and a scratch-image example in CONTRIBUTING.md.

@lhotari
lhotari requested review from a team, mattisonchao and zymap as code owners September 18, 2026 09:57
@github-actions github-actions Bot added the doc This pr contains a document label Sep 18, 2026
@lhotari
lhotari force-pushed the fix-integration-test-defaults branch from 15aaa52 to 89af72a Compare September 18, 2026 12:03
@lhotari lhotari changed the title Prepare integration tests for separate Pulsar and connector images Prepare CI for Pulsar 5.x support with separate broker and connector images Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc This pr contains a document

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant