Arm backend: Enable Vulkan validation layer - #22841
Merged
Merged
Conversation
Signed-off-by: Elena Zhelezina <elena.zhelezina@arm.com> Change-Id: Ia693cf38b8a5ccf97fd6c4afed6e23611a0df556
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22841
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New Failure, 1 Unrelated Failure, 1 Unclassified FailureAs of commit d4c017a with merge base d93e1f7 ( NEW FAILURE - The following job has failed:
UNCLASSIFIED FAILURE - DrCI could not classify the following job because the workflow did not run on the merge base. The failure may be pre-existing on trunk or introduced by this PR:
BROKEN TRUNK - The following job failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
mansnils
approved these changes
Sep 15, 2026
zingo
approved these changes
Sep 15, 2026
huydhn
added a commit
that referenced
this pull request
Sep 30, 2026
This job was added after pull.yml was migrated, so it came in against the old template: it was the last `runner: linux.*` and the last `linux_job_v2` caller left in the file. Matches its sibling test-arm-backend-no-driver: v3, `needs: docker-image`, and the hash-pinned image tag rather than the floating `ci-image:<name>` that only v2 resolves. Keeps the 24.04 arm-sdk image, which docker-builds.yml does build. linux.2xlarge.memory is an r5.2xlarge, 8 vCPU / 64 GiB, so mt-l-x86iavx512-8-64 is an exact match rather than a judgement call. Also drops `sudo sysctl fs.inotify.max_user_watches`. Nothing in this job watches files: the three legs run pytest, a cmake/ctest build, and a wrapper that scans a `tee`'d log, and there is no inotify reference anywhere in backends/arm or .ci. It arrived with the job in #22841, which does not mention it in the description or review. It could not have taken effect here in any case -- it is a node-level knob and these pods are not privileged -- so leaving it would only have meant a guaranteed-failing command or a `|| true` implying a dependency that does not exist. Authored with Claude Code.
huydhn
added a commit
that referenced
this pull request
Sep 30, 2026
Last `runner: linux.*` and last `linux_job_v2` caller in `pull.yml`. The job was added after the file was migrated, so it came in against the old template. - v2 → v3, `needs: docker-image`, hash-pinned image tag instead of the floating `ci-image:<name>` that only v2 resolves. Matches its sibling `test-arm-backend-no-driver`. Keeps the 24.04 arm-sdk image, which `docker-builds.yml` builds. - `linux.2xlarge.memory` is an r5.2xlarge (8 vCPU / 64 GiB), so `mt-l-x86iavx512-8-64` is an exact match. - Drops `sudo sysctl fs.inotify.max_user_watches=1048576`. Nothing in this job watches files — the three legs run pytest, a cmake/ctest build, and a wrapper that scans a `tee`'d log — and there is no `inotify` reference anywhere in `backends/arm` or `.ci`. It arrived with the job in #22841, which does not mention it in the description or review. It could not have taken effect here regardless: it is a node-level knob and these pods are not privileged. ## Testing `pull_request` runs all three matrix legs (`test_pytest_ops_vkml`, `test_ootb_tests_vgf`, `test_runtime_vgf`) here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Enable the Khronos Vulkan Validation Layer for the Arm VGF/VKML test paths.
The goal of this change is to detect invalid Vulkan API usage during VGF execution and make validation errors visible as CI failures, while keeping the production VGF runtime unchanged.
What this change does
Enables
VK_LAYER_KHRONOS_validationfor VGF/VKML tests.Configures the validation layer through the Vulkan loader environment rather than modifying
VkInstanceCreateInfoin the backend.Ensures the validation layer is placed before the Arm ML emulation layers.
Makes the validation-layer manifest visible when the ML SDK overrides
VK_LAYER_PATH.Restricts validation output to errors so warnings do not fail otherwise valid tests.
Adds validation to both:
runner_utils.py.examples/arm/run.shand the VGF OOTB tests.Adds a wrapper for direct commands that converts Vulkan validation errors into a non-zero exit status.
Preserves validation errors produced by earlier Vulkan subprocesses when a test launches multiple Vulkan processes.
Keeps validation opt-in for generic pytest/Buck runs so environments without the Vulkan SDK or validation layer can still run the normal Arm test suite.
Adds integration tests verifying that the validation layer is enabled and that validation failures are propagated correctly.
Validation can be controlled using:
The dedicated VGF/VKML test entry points enable it, while explicitly setting it to
0,false,off, ornodisables validation for debugging.Known validation errors
There are currently several validation errors produced by known issues in the VKML / Vulkan emulation-layer dependencies.
These VUIDs are temporarily filtered so that enabling Vulkan validation does not make the VGF CI permanently fail on already-known dependency bugs.
The filtered errors currently include issues related to:
The corresponding VUID filters are intentionally narrow so that any new or unexpected Vulkan validation error still fails the test.
These filters are temporary and should be removed individually once the corresponding bugs are fixed in the emulation-layer dependencies.
Why
Without the validation layer, invalid Vulkan API usage can remain unnoticed when running against the emulation layers. This change makes Vulkan API correctness part of the VGF/VKML CI signal and should help catch regressions closer to where they are introduced.
cc @SS-JIA @manuelcandales @digantdesai @cbilgin @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell @rascani