Skip to content

Update watcher-s2i-tempest definition to match watcher-operator-validation-master tempest tests - #464

Open
morenod wants to merge 1 commit into
openstack-k8s-operators:mainfrom
morenod:watcher-s2i-job
Open

morenod wants to merge 1 commit into
openstack-k8s-operators:mainfrom
morenod:watcher-s2i-job

Conversation

@morenod

@morenod morenod commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Consolidate watcher validation jobs on CentOS Stream 10 s2i images

What

This change replaces the old CentOS 9 EDPM validation jobs with the s2i-based job, which already provides the same tempest coverage. As suggested in review, it also reorganizes the jobs into the project template and keeps the familiar watcher-operator-validation-master naming instead of watcher-s2i-tempest.

.zuul.yaml

  • Removed the CentOS 9 jobs: the CS9 watcher-operator-base, watcher-operator-validation-base, the meta-content watcher-operator-validation-master, and the CS9 periodic-watcher-operator-validation-master.
  • Renamed the CentOS Stream 10 base jobs to the canonical names (watcher-operator-base, watcher-operator-validation-base) now that CS9 is gone.
  • Renamed watcher-s2i-tempest → watcher-operator-validation-master (CS10, consuming s2i-built images, full tempest coverage against an NFS cinder backend). Unlike the old job it uses a regular EDPM parent rather than the s2i parent, which allows depends-on across repos.
  • Added periodic-watcher-operator-validation-master as the CS10 periodic equivalent of the above.
  • Consolidated the openstack-k8s-operators-content-provider, watcher-s2i-content-provider and watcher-operator-validation-master jobs into the opendev-master-watcher-operator-pipeline github-check template; the project github-check now only lists noop, watcher-operator-doc-preview and watcher-operator-kuttl.
  • Updated the opendev-watcher-edpm-pipeline and watcher-operator-s2i-component-jobs templates to reference the renamed job.
  • The jobs are now voting.

Test coverage

  • ci/tests/watcher-tempest-nfs.yml: enable optimize.run_bfv_tests, optimize.run_skipped_action_tests and optimize.run_delete_shelve_action_tests.
  • ci/scenarios/nfs.yml: register the nfs / nfs-2 cinder volume backends via cifmw_edpm_prepare_cindervolumes during edpm prepare.

Notes

  • watcher-operator-kuttl intentionally stays on CentOS 9 (different job type, out of scope).
  • The periodic job consumes s2i images; its triggering pipeline (external RDO config) must provide the watcher-s2i-content-provider dependency.

@openshift-ci
openshift-ci Bot requested review from dprince and stuggi September 28, 2026 08:27
@openshift-ci

openshift-ci Bot commented Sep 28, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign abays for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai

coderabbitai Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: Central YAML (base), Organization UI (inherited)
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 1e01f7e5-ff1e-4ac9-8f57-ba3579c1e451

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build succeeded (check pipeline).
https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/16a35feeea9e41c685799221962e4666

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 49m 42s
✔️ watcher-operator-validation-master SUCCESS in 2h 07m 56s
✔️ noop SUCCESS in 0s
✔️ watcher-operator-kuttl SUCCESS in 57m 08s
✔️ watcher-s2i-content-provider SUCCESS in 10h 11m 31s (non-voting)
✔️ watcher-s2i-tempest SUCCESS in 2h 04m 46s (non-voting)

@amoralej

Copy link
Copy Markdown
Contributor

github.com/openstack-k8s-operators/openstack-operator/pull/2070/ is merged so i think we may get rid of the watcher-operator-validation-master job too.

Also, i see we are skipping some tests that were added to the tempest plugin that we may run in the master job:

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/logs//c5e/rdoproject.org/c5ebe5dc020041c4a1259c71edc4ecb9/controller/ci-framework-data/tests/test_operator/tempest-tests-tempest/tempest_results.html

as:

run_bfv_tests: true
run_skipped_action_tests: true
run_delete_shelve_action_tests: true

Although these were also missing in the old job, may we use this PR to also enable them?

Comment thread .zuul.yaml Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should make them voting now too.

Comment thread .zuul.yaml Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

@centosinfra-prod-github-app

Copy link
Copy Markdown

Zuul encountered a syntax error while parsing its
configuration in the repo openstack-k8s-operators/watcher-operator on branch main. The
problem was:

expected str for dictionary value @ data['github-check']['jobs'][3]['watcher-s2i-content-provider']

The problem appears in the "openstack-k8s-operators/watcher-operator" project stanza:

project:
name: openstack-k8s-operators/watcher-operator
default-branch: main
merge-mode: rebase
templates:
- opendev-master-watcher-operator-pipeline
github-check:
jobs:
- noop
...

in "openstack-k8s-operators/watcher-operator/.zuul.yaml@main", line 379

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/4a0ac70267c14cbf9a5b0e808f8c311f

❌ openstack-k8s-operators-content-provider FAILURE in 6m 13s
⚠️ watcher-operator-validation-master SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
✔️ noop SUCCESS in 0s
⚠️ watcher-operator-kuttl SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
✔️ watcher-s2i-content-provider SUCCESS in 10m 39s
⚠️ watcher-s2i-tempest SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider

@morenod

morenod commented Sep 30, 2026

Copy link
Copy Markdown
Contributor Author

recheck

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/bba75beeba7040deb611986fbe767d14

❌ openstack-k8s-operators-content-provider FAILURE in 5m 27s
⚠️ watcher-operator-validation-master SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
✔️ noop SUCCESS in 0s
⚠️ watcher-operator-kuttl SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
✔️ watcher-s2i-content-provider SUCCESS in 16m 28s
⚠️ watcher-s2i-tempest SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider

@morenod

morenod commented Sep 30, 2026

Copy link
Copy Markdown
Contributor Author

recheck

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/c421f9a83aff4d3a86b24af47afe6ca2

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 44m 25s
✔️ watcher-operator-validation-master SUCCESS in 2h 28m 03s
✔️ noop SUCCESS in 0s
❌ watcher-operator-kuttl RETRY_LIMIT in 46m 46s
✔️ watcher-s2i-content-provider SUCCESS in 2h 20m 48s
❌ watcher-s2i-tempest FAILURE in 2h 00m 49s

@morenod

morenod commented Sep 30, 2026

Copy link
Copy Markdown
Contributor Author

recheck

1 similar comment
@morenod

morenod commented Sep 30, 2026

Copy link
Copy Markdown
Contributor Author

recheck

@centosinfra-prod-github-app

Copy link
Copy Markdown

This change depends on a change that failed to merge.

Change openstack-k8s-operators/ci-framework#4217 is needed.

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/45f59686a500497e92ac17db8259aacf

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 56m 05s
✔️ watcher-operator-validation-master SUCCESS in 2h 39m 32s
✔️ noop SUCCESS in 0s
❌ watcher-operator-kuttl RETRY_LIMIT in 45m 44s
✔️ watcher-s2i-content-provider SUCCESS in 2h 35m 25s
❌ watcher-s2i-tempest FAILURE in 2h 18m 11s

@morenod

morenod commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

recheck

@centosinfra-prod-github-app

Copy link
Copy Markdown

Zuul encountered a syntax error while parsing its
configuration in the repo openstack-k8s-operators/watcher-operator on branch main. The
problem was:

found duplicate anchor; first occurrence
in "openstack-k8s-operators/watcher-operator/.zuul.yaml@main", line 101, column 23
second occurrence
in "openstack-k8s-operators/watcher-operator/.zuul.yaml@main", line 131, column 23

@morenod

morenod commented Oct 5, 2026

Copy link
Copy Markdown
Contributor Author

recheck

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/4c6b8226947344c78d1616210888a730

✔️ openstack-k8s-operators-content-provider SUCCESS in 5h 14m 58s
✔️ watcher-operator-validation-master SUCCESS in 2h 21m 15s
✔️ noop SUCCESS in 0s
❌ watcher-operator-kuttl RETRY_LIMIT in 45m 23s
✔️ watcher-s2i-content-provider SUCCESS in 6h 15m 32s
✔️ watcher-s2i-tempest SUCCESS in 2h 20m 10s

@amoralej amoralej left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see the watcher-s2i-tempest now runs all the tests. Just missing removing the old one, and I'd suggest reorganizing it into the template.

Comment thread .zuul.yaml
- name: openstack-operator
src: "{{ ansible_user_dir }}/src/github.com/openstack-k8s-operators/openstack-operator"
image_base: watcher
- watcher-operator-validation-master:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea was to get rid of this job as the new s2i one has the same coverage but without s2i parent job which allows to do depends-on, etc..

Actually, i'd suggest reorganizing the new jobs for the github-check in this project-template.

Comment thread .zuul.yaml

- job:
name: watcher-operator-validation-master
name: periodic-watcher-operator-validation-master

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FTR, this job is not executed since https://review.rdoproject.org/r/c/config/+/59056

However we can not remove it while the template is still defined (both the template and this job should be removed).

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/9aa6061e0bce4f48babf5ea45b3d1851

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 40m 14s
✔️ watcher-s2i-content-provider SUCCESS in 16h 57m 54s
❌ watcher-operator-validation-master FAILURE in 1h 46m 24s
✔️ noop SUCCESS in 0s
✔️ watcher-operator-kuttl SUCCESS in 54m 12s

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/9431f40503ca473782649579b2670366

❌ openstack-k8s-operators-content-provider FAILURE in 6m 14s
✔️ watcher-s2i-content-provider SUCCESS in 11h 58m 24s
⚠️ watcher-operator-validation-master SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
✔️ noop SUCCESS in 0s
⚠️ watcher-operator-kuttl SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider

@morenod

morenod commented Oct 8, 2026

Copy link
Copy Markdown
Contributor Author

recheck

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/03dc6f1629564d1dba2bfe632cda3754

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 56m 24s
✔️ watcher-s2i-content-provider SUCCESS in 11h 39m 24s
❌ watcher-operator-validation-master FAILURE in 38m 17s
✔️ noop SUCCESS in 0s
✔️ watcher-operator-kuttl SUCCESS in 1h 01m 07s

@morenod
morenod force-pushed the watcher-s2i-job branch 2 times, most recently from 18ba4b7 to d484cec Compare October 9, 2026 09:48
@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/3cd3fec7f8e54c6c88fd7d9a4c743f18

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 39m 37s
✔️ watcher-s2i-content-provider SUCCESS in 2h 20m 03s
❌ watcher-operator-validation-master FAILURE in 32m 48s
✔️ noop SUCCESS in 0s
✔️ watcher-operator-kuttl SUCCESS in 59m 00s

The old CentOS 9 EDPM validation jobs are superseded by the s2i-based
job, which provides the same test coverage but uses a regular EDPM
parent instead of the s2i parent, allowing depends-on across repos.

.zuul.yaml:
- Drop the CentOS 9 base jobs and rename the CentOS Stream 10 ones to
  the canonical watcher-operator-base / watcher-operator-validation-base.
- Remove the old CentOS 9 watcher-operator-validation-master (meta
  content) and periodic-watcher-operator-validation-master.
- Rename watcher-s2i-tempest to watcher-operator-validation-master and
  add a matching periodic-watcher-operator-validation-master, both on
  CentOS Stream 10 consuming s2i-built images.
- Consolidate the content-provider and validation jobs into the
  opendev-master-watcher-operator-pipeline github-check template.
- Make the jobs voting.

Enable the full tempest coverage on the NFS configuration: boot-from-
volume, skipped action and delete/shelve action tests, and register the
nfs/nfs-2 cinder volume backends during edpm prepare.

Pin the nfs/nfs-2 cinder volume backends to the master s2i cinder-volume
image via an explicit cinderVolumeImages entry in
cifmw_set_containers_images (set_containers backends format). Otherwise
the edpm_prepare role auto-generates those backends from the default
registry/org/tag, which resolve to the antelope-centos9 cinder-volume
image. That mismatches the master cinder api/scheduler/backup services
and breaks AMQP exchange durability (cinder-scheduler_fanout), crash-
looping cinder-volume and failing every volume test.

Assisted-by: Claude Opus 4.8 <noreply@anthropic.com>
@centosinfra-prod-github-app

Copy link
Copy Markdown

Build succeeded (check pipeline).
https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/ffb4f6a5b54040eeb59c1685b6fe30a9

✔️ openstack-k8s-operators-content-provider SUCCESS in 5h 57m 57s
✔️ watcher-s2i-content-provider SUCCESS in 5h 32m 18s
✔️ watcher-operator-validation-master SUCCESS in 2h 12m 12s
✔️ noop SUCCESS in 0s
✔️ watcher-operator-kuttl SUCCESS in 55m 33s

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants