Repository navigation
[Vulkan] Add aten.any.dim - #23253
Merged
Merged
Conversation
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
Contributor
Author
|
Stack from ghstack (oldest at bottom): |
mergennachin
added
the
module: vulkan
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/23253
Note: Links to docs will display an error until the docs builds have been completed. ⏳ No Failures, 9 PendingAs of commit 33a663f with merge base 9e47127 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This was referenced Sep 29, 2026
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
mergennachin
added a commit
that referenced
this pull request
Oct 7, 2026
…ormer blocks The dynamic transformer reproductions now lower to one Vulkan delegate across changing sequence lengths. This final PR makes `expand_copy` resizable and adds the eager-attention and SDPA integration tests, including fully masked rows. Fixes #23156. This is part 15 of the original ghstack series. Parts 1–7 (#23240 through #23246) have landed in main; the remaining eight PRs run from #23247 through #23254. Review each PR against its selected base branch. This PR contains the final expansion and transformer changes; part 10 owns the scalar-tensor issue #23158. | Part | PR | Change | | --- | --- | --- | | 1 | #23240 (landed) | Hardware Vulkan CI | | 2 | #23241 (landed) | Scalar cache type and signed-zero keys | | 3 | #23242 (landed) | Vulkan-local signed-zero serialization | | 4 | #23243 (landed) | GELU modes and view kwargs | | 5 | #23244 (landed) | Reduction clamp, NaN, and FP16 rounding | | 6 | #23245 (landed) | Reduction and arg-reduction dimension/storage guards | | 7 | #23246 (landed) | Bool staging and logical_not | | 8 | #23247 | Scalar representability and symbolic guards | | 9 | #23248 | 64-bit dtype and fusion policy | | 10 | #23249 | scalar_tensor with exact integer values | | 11 | #23250 | Typed, resizable full | | 12 | #23251 | Power special values and logical FP16 dtype | | 13 | #23252 | mul.Scalar | | 14 | #23253 | any.dim textures with either keepdim setting | | 15 | #23254 | Dynamic expand and transformer integration | The stack retains the FACTO and ATen conformance fixes and their regression tests: typed integer fills, NaN and signed-zero behavior, reduction range and accumulation fixes, safe scalar fallbacks, and the FP16 power contract. On native FP16 devices, the scalar-tensor, full texture, single-dimension texture reduction, and binary scalar shaders round their FP16 outputs to nearest-even. The binary scalar shaders also preserve the requested FP16 dtype when storage is emulated with FP32. Other operators retain their existing FP32 intermediate behavior on those devices. Coverage includes scalar exponents 2.0001 and 2049 and a two-node FP16 mul.Scalar chain in both texture and buffer storage. PR4 introduces `test_vulkan_dynamic.py` and its CI/Buck references. In #23253, `any.dim` now supports both `keepdim` settings on textures. With `keepdim=False`, a reduction into a temporary texture is followed by a GPU view/repack into the squeezed output shape. Both nodes resize dynamically, and empty reduced axes produce false. This adds one temporary texture and one GPU dispatch without introducing a bool-buffer requirement for texture models. Unsupported scalar inputs and 4D batch/channel axes use CPU fallback. Current rebase: Rebased onto main at `5f72739cf26e` after #23246 landed. All eight remaining code patches are unchanged, and the rebase had no conflicts. All 1153 shader variants compile with glslc, the 15 graph-builder/serialization tests pass, and lintrunner and `git diff --check` pass. Native GPU execution was validated on an earlier base; hardware and SwiftShader CI will rerun on the updated ghstack heads. Prior texture-path validation on main at `903cef063774`: the Release Vulkan/portable runtime built and eleven focused native tests passed on Apple M1 Pro / MoltenVK. Coverage included dynamic and chained any reductions, actual texture storage, scalar outputs, singleton and empty dimensions, growth after a zero-length dimension, transformer integration, supported and unsupported 4D reductions, special values, and FP16 rounding. An additional 27 FACTO-generated bool any.dim cases and six boundary cases matched ATen and portable kernels exactly through the shared `backends/test/` harness: 23 ran on Vulkan textures and 10 exercised expected CPU fallback. The subsequent review of #23245 with #23253 verified 784 support/storage combinations and eight mixed dynamic models; all nine review tests passed, with execution results matching ATen and portable kernels. These GPU execution results precede the latest upstream shader-indexing change. Earlier validation of the operator changes passed 116 FACTO cases against both ATen and portable kernels, plus 144 boundary cases against ATen, with texture and buffer preferences: all 520 Vulkan-configuration executions matched ATen, including CPU fallback where unsupported. The additional 239-case sweep covered all 91 available FACTO specifications for registered ATen overloads and 57 targeted examples. Exact-input replays of every candidate Vulkan failure on a separately built main at `a31838280f9309f04af1b375147375b702d29339` found no new regressions or unresolved comparisons. The prior full native run passed 51 tests with one expected SwiftShader-only skip. Subsequent review coverage added exact FP16 rounding boundaries, int32 buffer amax/amin range checks against both ATen and portable kernels, and 28 argmax/argmin execution and fallback cases. These are prior validation results, not a repeat of the full suite for this update. Validation uses Release portable CPU kernels and Vulkan; optimized kernels and XNNPACK are disabled. Buffer preference uses `texture_limits=(1,1,1)`, with actual storage and fallback recorded separately. Single-texel tensors can still select textures. FACTO revision: `3b8c778c99766a8b4d0d04563ae0b16cbb276829`, seed 0. Recreates #23162 through ghstack. Prior review discussion remains on that PR. Authored with OpenAI Codex; split planned with Claude Code. cc @SS-JIA @manuelcandales @digantdesai @cbilgin ghstack-source-id: 6b879b2 ghstack-comment-id: 5892899249 Pull-Request: #23254
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
mergennachin
added a commit
that referenced
this pull request
Oct 7, 2026
…ormer blocks The dynamic transformer reproductions now lower to one Vulkan delegate across changing sequence lengths. This final PR makes `expand_copy` resizable and adds the eager-attention and SDPA integration tests, including fully masked rows. Fixes #23156. This is part 15 of the original ghstack series. Parts 1–7 (#23240 through #23246) have landed in main; the remaining eight PRs run from #23247 through #23254. Review each PR against its selected base branch. This PR contains the final expansion and transformer changes; part 10 owns the scalar-tensor issue #23158. | Part | PR | Change | | --- | --- | --- | | 1 | #23240 (landed) | Hardware Vulkan CI | | 2 | #23241 (landed) | Scalar cache type and signed-zero keys | | 3 | #23242 (landed) | Vulkan-local signed-zero serialization | | 4 | #23243 (landed) | GELU modes and view kwargs | | 5 | #23244 (landed) | Reduction clamp, NaN, and FP16 rounding | | 6 | #23245 (landed) | Reduction and arg-reduction dimension/storage guards | | 7 | #23246 (landed) | Bool staging and logical_not | | 8 | #23247 | Scalar representability and symbolic guards | | 9 | #23248 | 64-bit dtype and fusion policy | | 10 | #23249 | scalar_tensor with exact integer values | | 11 | #23250 | Typed, resizable full | | 12 | #23251 | Power special values and logical FP16 dtype | | 13 | #23252 | mul.Scalar | | 14 | #23253 | any.dim textures with either keepdim setting | | 15 | #23254 | Dynamic expand and transformer integration | The stack retains the FACTO and ATen conformance fixes and their regression tests: typed integer fills, NaN and signed-zero behavior, reduction range and accumulation fixes, safe scalar fallbacks, and the FP16 power contract. On native FP16 devices, the scalar-tensor, full texture, single-dimension texture reduction, and binary scalar shaders round their FP16 outputs to nearest-even. The binary scalar shaders also preserve the requested FP16 dtype when storage is emulated with FP32. Other operators retain their existing FP32 intermediate behavior on those devices. Coverage includes scalar exponents 2.0001 and 2049 and a two-node FP16 mul.Scalar chain in both texture and buffer storage. PR4 introduces `test_vulkan_dynamic.py` and its CI/Buck references. In #23253, `any.dim` now supports both `keepdim` settings on textures. With `keepdim=False`, a reduction into a temporary texture is followed by a GPU view/repack into the squeezed output shape. Both nodes resize dynamically, and empty reduced axes produce false. This adds one temporary texture and one GPU dispatch without introducing a bool-buffer requirement for texture models. Unsupported scalar inputs and 4D batch/channel axes use CPU fallback. Current rebase: Rebased onto main at `5f72739cf26e` after #23246 landed. All eight remaining code patches are unchanged, and the rebase had no conflicts. All 1153 shader variants compile with glslc, the 15 graph-builder/serialization tests pass, and lintrunner and `git diff --check` pass. Native GPU execution was validated on an earlier base; hardware and SwiftShader CI will rerun on the updated ghstack heads. Prior texture-path validation on main at `903cef063774`: the Release Vulkan/portable runtime built and eleven focused native tests passed on Apple M1 Pro / MoltenVK. Coverage included dynamic and chained any reductions, actual texture storage, scalar outputs, singleton and empty dimensions, growth after a zero-length dimension, transformer integration, supported and unsupported 4D reductions, special values, and FP16 rounding. An additional 27 FACTO-generated bool any.dim cases and six boundary cases matched ATen and portable kernels exactly through the shared `backends/test/` harness: 23 ran on Vulkan textures and 10 exercised expected CPU fallback. The subsequent review of #23245 with #23253 verified 784 support/storage combinations and eight mixed dynamic models; all nine review tests passed, with execution results matching ATen and portable kernels. These GPU execution results precede the latest upstream shader-indexing change. Earlier validation of the operator changes passed 116 FACTO cases against both ATen and portable kernels, plus 144 boundary cases against ATen, with texture and buffer preferences: all 520 Vulkan-configuration executions matched ATen, including CPU fallback where unsupported. The additional 239-case sweep covered all 91 available FACTO specifications for registered ATen overloads and 57 targeted examples. Exact-input replays of every candidate Vulkan failure on a separately built main at `a31838280f9309f04af1b375147375b702d29339` found no new regressions or unresolved comparisons. The prior full native run passed 51 tests with one expected SwiftShader-only skip. Subsequent review coverage added exact FP16 rounding boundaries, int32 buffer amax/amin range checks against both ATen and portable kernels, and 28 argmax/argmin execution and fallback cases. These are prior validation results, not a repeat of the full suite for this update. Validation uses Release portable CPU kernels and Vulkan; optimized kernels and XNNPACK are disabled. Buffer preference uses `texture_limits=(1,1,1)`, with actual storage and fallback recorded separately. Single-texel tensors can still select textures. FACTO revision: `3b8c778c99766a8b4d0d04563ae0b16cbb276829`, seed 0. Recreates #23162 through ghstack. Prior review discussion remains on that PR. Authored with OpenAI Codex; split planned with Claude Code. cc @SS-JIA @manuelcandales @digantdesai @cbilgin ghstack-source-id: 0705e5f ghstack-comment-id: 5892899249 Pull-Request: #23254
…oid nested SwiftShader buffer skips and validate eight focused tests on SwiftShader and MoltenVK. [ghstack-poisoned]
mergennachin
added a commit
that referenced
this pull request
Oct 8, 2026
Boolean any(dim) reductions in attention masks fell back to CPU. This adds uint8 any variants to the texture and per-row buffer reduction shaders and supports both keepdim settings on textures. For keepdim=False, including the default when keepdim is omitted, the runtime reduces into a temporary texture with the reduced axis retained, then uses the existing texture view shader to repack into the requested output shape. Both nodes resize with the input. This adds one temporary texture and one GPU repack dispatch. Texture execution requires no 8-bit storage buffers, so surrounding texture operators can remain in the same delegate. The reduction shader writes the false identity when the reduced axis is empty. Delegation is limited to bool tensors and supported reduction axes. Rank-zero inputs and unsupported 4D batch/channel reductions fall back to CPU. Buffer execution is available for the last axis on devices with 8-bit storage support. Part 14/15 of the Vulkan transformer and operator-conformance stack. Depends on #23252; review against the selected base branch. Integration PR: #23254. Validation: The Release Vulkan/portable runtime builds and 11 focused native tests pass on Apple M1 Pro / MoltenVK. Tests assert actual texture storage and exact ATen results for omitted and explicit keepdim=False, chained reductions, scalar outputs, singleton and empty dimensions, supported 4D shapes, and dynamic dimensions shrinking to zero and growing again. Existing keepdim=True texture/buffer execution, transformer integration, reduction special values, and FP16 rounding tests pass. Through the shared backends/test harness, 27 FACTO-generated bool any.dim cases and six boundary cases match both ATen and portable kernels exactly: 23 execute on Vulkan textures and 10 use expected CPU fallback. Lintrunner and git diff --check pass. SwiftShader and hardware CI are pending on the updated heads. Recreates #23214 through ghstack. Prior review discussion remains on that PR. Authored with OpenAI Codex; split planned with Claude Code. cc @SS-JIA @manuelcandales @digantdesai @cbilgin ghstack-source-id: 148791b ghstack-comment-id: 5892898410 Pull-Request: #23253
This PR needs a
|
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Boolean any(dim) reductions in attention masks fell back to CPU. This adds uint8 any variants to the texture and per-row buffer reduction shaders and supports both keepdim settings on textures.
For keepdim=False, including the default when keepdim is omitted, the runtime reduces into a temporary texture with the reduced axis retained, then uses the existing texture view shader to repack into the requested output shape. Both nodes resize with the input. This adds one temporary texture and one GPU repack dispatch. Texture execution requires no 8-bit storage buffers, so surrounding texture operators can remain in the same delegate. The reduction shader writes the false identity when the reduced axis is empty.
Delegation is limited to bool tensors and supported reduction axes. Rank-zero inputs and unsupported 4D batch/channel reductions fall back to CPU. Buffer execution is available for the last axis on devices with 8-bit storage support.
Part 14/15 of the Vulkan transformer and operator-conformance stack. #23252 has landed; this PR is rebased directly onto
origin/mainat9e47127d28. Integration PR: #23254.Test Plan: Rebuilt ExecuTorch with Vulkan enabled from the rebased source on macOS arm64. Ran eight focused tests from
test_vulkan_dynamic.pythrough the public Buck Python test adapter, explicitly selecting each driver ICD. SwiftShader: 8/8 passed in 145.877s. MoltenVK: 8/8 passed in 151.952s. Both runs had zero failures, errors, or skips. Coverage includes all five added or modified tests plus reduction special values and FP16 halfway/overflow rounding: unsupported-input partitioning, omitted/false keepdim, chained texture reductions, scalar and empty outputs, 4D axes, and dynamic dimensions shrinking to zero and growing again. Lintrunner on all six changed files,git diff --check, andpip checkpassed.The unchanged
test_dynamic_any_dimreproduced a Buck adapter assertion when skipping an unsupported SwiftShader buffer subtest. The test now selects supported storages before entering subtests, matching the existing SwiftShader guards. SwiftShader retains all texture cases; MoltenVK retains both texture and buffer cases.Recreates #23214 through ghstack. Prior review discussion remains on that PR.
Authored with OpenAI Codex; split planned with Claude Code.
cc @SS-JIA @manuelcandales @digantdesai @cbilgin