Repository navigation
[Vulkan] Match ATen special values in pow.Tensor_Scalar - #23251
Merged
Merged
Conversation
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
…ghstack [ghstack-poisoned]
Contributor
Author
mergennachin
added
the
module: vulkan
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/23251
Note: Links to docs will display an error until the docs builds have been completed. ⏳ No Failures, 39 PendingAs of commit 1230e03 with merge base 265ed38 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This was referenced Sep 29, 2026
[ghstack-poisoned]
mergennachin
added a commit
that referenced
this pull request
Oct 7, 2026
…ormer blocks The dynamic transformer reproductions now lower to one Vulkan delegate across changing sequence lengths. This final PR makes `expand_copy` resizable and adds the eager-attention and SDPA integration tests, including fully masked rows. Fixes #23156. This is part 15 of the original ghstack series. Parts 1–7 (#23240 through #23246) have landed in main; the remaining eight PRs run from #23247 through #23254. Review each PR against its selected base branch. This PR contains the final expansion and transformer changes; part 10 owns the scalar-tensor issue #23158. | Part | PR | Change | | --- | --- | --- | | 1 | #23240 (landed) | Hardware Vulkan CI | | 2 | #23241 (landed) | Scalar cache type and signed-zero keys | | 3 | #23242 (landed) | Vulkan-local signed-zero serialization | | 4 | #23243 (landed) | GELU modes and view kwargs | | 5 | #23244 (landed) | Reduction clamp, NaN, and FP16 rounding | | 6 | #23245 (landed) | Reduction and arg-reduction dimension/storage guards | | 7 | #23246 (landed) | Bool staging and logical_not | | 8 | #23247 | Scalar representability and symbolic guards | | 9 | #23248 | 64-bit dtype and fusion policy | | 10 | #23249 | scalar_tensor with exact integer values | | 11 | #23250 | Typed, resizable full | | 12 | #23251 | Power special values and logical FP16 dtype | | 13 | #23252 | mul.Scalar | | 14 | #23253 | any.dim textures with either keepdim setting | | 15 | #23254 | Dynamic expand and transformer integration | The stack retains the FACTO and ATen conformance fixes and their regression tests: typed integer fills, NaN and signed-zero behavior, reduction range and accumulation fixes, safe scalar fallbacks, and the FP16 power contract. On native FP16 devices, the scalar-tensor, full texture, single-dimension texture reduction, and binary scalar shaders round their FP16 outputs to nearest-even. The binary scalar shaders also preserve the requested FP16 dtype when storage is emulated with FP32. Other operators retain their existing FP32 intermediate behavior on those devices. Coverage includes scalar exponents 2.0001 and 2049 and a two-node FP16 mul.Scalar chain in both texture and buffer storage. PR4 introduces `test_vulkan_dynamic.py` and its CI/Buck references. In #23253, `any.dim` now supports both `keepdim` settings on textures. With `keepdim=False`, a reduction into a temporary texture is followed by a GPU view/repack into the squeezed output shape. Both nodes resize dynamically, and empty reduced axes produce false. This adds one temporary texture and one GPU dispatch without introducing a bool-buffer requirement for texture models. Unsupported scalar inputs and 4D batch/channel axes use CPU fallback. Current rebase: Rebased onto main at `5f72739cf26e` after #23246 landed. All eight remaining code patches are unchanged, and the rebase had no conflicts. All 1153 shader variants compile with glslc, the 15 graph-builder/serialization tests pass, and lintrunner and `git diff --check` pass. Native GPU execution was validated on an earlier base; hardware and SwiftShader CI will rerun on the updated ghstack heads. Prior texture-path validation on main at `903cef063774`: the Release Vulkan/portable runtime built and eleven focused native tests passed on Apple M1 Pro / MoltenVK. Coverage included dynamic and chained any reductions, actual texture storage, scalar outputs, singleton and empty dimensions, growth after a zero-length dimension, transformer integration, supported and unsupported 4D reductions, special values, and FP16 rounding. An additional 27 FACTO-generated bool any.dim cases and six boundary cases matched ATen and portable kernels exactly through the shared `backends/test/` harness: 23 ran on Vulkan textures and 10 exercised expected CPU fallback. The subsequent review of #23245 with #23253 verified 784 support/storage combinations and eight mixed dynamic models; all nine review tests passed, with execution results matching ATen and portable kernels. These GPU execution results precede the latest upstream shader-indexing change. Earlier validation of the operator changes passed 116 FACTO cases against both ATen and portable kernels, plus 144 boundary cases against ATen, with texture and buffer preferences: all 520 Vulkan-configuration executions matched ATen, including CPU fallback where unsupported. The additional 239-case sweep covered all 91 available FACTO specifications for registered ATen overloads and 57 targeted examples. Exact-input replays of every candidate Vulkan failure on a separately built main at `a31838280f9309f04af1b375147375b702d29339` found no new regressions or unresolved comparisons. The prior full native run passed 51 tests with one expected SwiftShader-only skip. Subsequent review coverage added exact FP16 rounding boundaries, int32 buffer amax/amin range checks against both ATen and portable kernels, and 28 argmax/argmin execution and fallback cases. These are prior validation results, not a repeat of the full suite for this update. Validation uses Release portable CPU kernels and Vulkan; optimized kernels and XNNPACK are disabled. Buffer preference uses `texture_limits=(1,1,1)`, with actual storage and fallback recorded separately. Single-texel tensors can still select textures. FACTO revision: `3b8c778c99766a8b4d0d04563ae0b16cbb276829`, seed 0. Recreates #23162 through ghstack. Prior review discussion remains on that PR. Authored with OpenAI Codex; split planned with Claude Code. cc @SS-JIA @manuelcandales @digantdesai @cbilgin ghstack-source-id: a7c2766 ghstack-comment-id: 5892899249 Pull-Request: #23254
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
mergennachin
added a commit
that referenced
this pull request
Oct 7, 2026
…ormer blocks The dynamic transformer reproductions now lower to one Vulkan delegate across changing sequence lengths. This final PR makes `expand_copy` resizable and adds the eager-attention and SDPA integration tests, including fully masked rows. Fixes #23156. This is part 15 of the original ghstack series. Parts 1–7 (#23240 through #23246) have landed in main; the remaining eight PRs run from #23247 through #23254. Review each PR against its selected base branch. This PR contains the final expansion and transformer changes; part 10 owns the scalar-tensor issue #23158. | Part | PR | Change | | --- | --- | --- | | 1 | #23240 (landed) | Hardware Vulkan CI | | 2 | #23241 (landed) | Scalar cache type and signed-zero keys | | 3 | #23242 (landed) | Vulkan-local signed-zero serialization | | 4 | #23243 (landed) | GELU modes and view kwargs | | 5 | #23244 (landed) | Reduction clamp, NaN, and FP16 rounding | | 6 | #23245 (landed) | Reduction and arg-reduction dimension/storage guards | | 7 | #23246 (landed) | Bool staging and logical_not | | 8 | #23247 | Scalar representability and symbolic guards | | 9 | #23248 | 64-bit dtype and fusion policy | | 10 | #23249 | scalar_tensor with exact integer values | | 11 | #23250 | Typed, resizable full | | 12 | #23251 | Power special values and logical FP16 dtype | | 13 | #23252 | mul.Scalar | | 14 | #23253 | any.dim textures with either keepdim setting | | 15 | #23254 | Dynamic expand and transformer integration | The stack retains the FACTO and ATen conformance fixes and their regression tests: typed integer fills, NaN and signed-zero behavior, reduction range and accumulation fixes, safe scalar fallbacks, and the FP16 power contract. On native FP16 devices, the scalar-tensor, full texture, single-dimension texture reduction, and binary scalar shaders round their FP16 outputs to nearest-even. The binary scalar shaders also preserve the requested FP16 dtype when storage is emulated with FP32. Other operators retain their existing FP32 intermediate behavior on those devices. Coverage includes scalar exponents 2.0001 and 2049 and a two-node FP16 mul.Scalar chain in both texture and buffer storage. PR4 introduces `test_vulkan_dynamic.py` and its CI/Buck references. In #23253, `any.dim` now supports both `keepdim` settings on textures. With `keepdim=False`, a reduction into a temporary texture is followed by a GPU view/repack into the squeezed output shape. Both nodes resize dynamically, and empty reduced axes produce false. This adds one temporary texture and one GPU dispatch without introducing a bool-buffer requirement for texture models. Unsupported scalar inputs and 4D batch/channel axes use CPU fallback. Current rebase: Rebased onto main at `5f72739cf26e` after #23246 landed. All eight remaining code patches are unchanged, and the rebase had no conflicts. All 1153 shader variants compile with glslc, the 15 graph-builder/serialization tests pass, and lintrunner and `git diff --check` pass. Native GPU execution was validated on an earlier base; hardware and SwiftShader CI will rerun on the updated ghstack heads. Prior texture-path validation on main at `903cef063774`: the Release Vulkan/portable runtime built and eleven focused native tests passed on Apple M1 Pro / MoltenVK. Coverage included dynamic and chained any reductions, actual texture storage, scalar outputs, singleton and empty dimensions, growth after a zero-length dimension, transformer integration, supported and unsupported 4D reductions, special values, and FP16 rounding. An additional 27 FACTO-generated bool any.dim cases and six boundary cases matched ATen and portable kernels exactly through the shared `backends/test/` harness: 23 ran on Vulkan textures and 10 exercised expected CPU fallback. The subsequent review of #23245 with #23253 verified 784 support/storage combinations and eight mixed dynamic models; all nine review tests passed, with execution results matching ATen and portable kernels. These GPU execution results precede the latest upstream shader-indexing change. Earlier validation of the operator changes passed 116 FACTO cases against both ATen and portable kernels, plus 144 boundary cases against ATen, with texture and buffer preferences: all 520 Vulkan-configuration executions matched ATen, including CPU fallback where unsupported. The additional 239-case sweep covered all 91 available FACTO specifications for registered ATen overloads and 57 targeted examples. Exact-input replays of every candidate Vulkan failure on a separately built main at `a31838280f9309f04af1b375147375b702d29339` found no new regressions or unresolved comparisons. The prior full native run passed 51 tests with one expected SwiftShader-only skip. Subsequent review coverage added exact FP16 rounding boundaries, int32 buffer amax/amin range checks against both ATen and portable kernels, and 28 argmax/argmin execution and fallback cases. These are prior validation results, not a repeat of the full suite for this update. Validation uses Release portable CPU kernels and Vulkan; optimized kernels and XNNPACK are disabled. Buffer preference uses `texture_limits=(1,1,1)`, with actual storage and fallback recorded separately. Single-texel tensors can still select textures. FACTO revision: `3b8c778c99766a8b4d0d04563ae0b16cbb276829`, seed 0. Recreates #23162 through ghstack. Prior review discussion remains on that PR. Authored with OpenAI Codex; split planned with Claude Code. cc @SS-JIA @manuelcandales @digantdesai @cbilgin ghstack-source-id: 6b879b2 ghstack-comment-id: 5892899249 Pull-Request: #23254
[ghstack-poisoned]
[ghstack-poisoned]
mergennachin
added a commit
that referenced
this pull request
Oct 7, 2026
…ormer blocks The dynamic transformer reproductions now lower to one Vulkan delegate across changing sequence lengths. This final PR makes `expand_copy` resizable and adds the eager-attention and SDPA integration tests, including fully masked rows. Fixes #23156. This is part 15 of the original ghstack series. Parts 1–7 (#23240 through #23246) have landed in main; the remaining eight PRs run from #23247 through #23254. Review each PR against its selected base branch. This PR contains the final expansion and transformer changes; part 10 owns the scalar-tensor issue #23158. | Part | PR | Change | | --- | --- | --- | | 1 | #23240 (landed) | Hardware Vulkan CI | | 2 | #23241 (landed) | Scalar cache type and signed-zero keys | | 3 | #23242 (landed) | Vulkan-local signed-zero serialization | | 4 | #23243 (landed) | GELU modes and view kwargs | | 5 | #23244 (landed) | Reduction clamp, NaN, and FP16 rounding | | 6 | #23245 (landed) | Reduction and arg-reduction dimension/storage guards | | 7 | #23246 (landed) | Bool staging and logical_not | | 8 | #23247 | Scalar representability and symbolic guards | | 9 | #23248 | 64-bit dtype and fusion policy | | 10 | #23249 | scalar_tensor with exact integer values | | 11 | #23250 | Typed, resizable full | | 12 | #23251 | Power special values and logical FP16 dtype | | 13 | #23252 | mul.Scalar | | 14 | #23253 | any.dim textures with either keepdim setting | | 15 | #23254 | Dynamic expand and transformer integration | The stack retains the FACTO and ATen conformance fixes and their regression tests: typed integer fills, NaN and signed-zero behavior, reduction range and accumulation fixes, safe scalar fallbacks, and the FP16 power contract. On native FP16 devices, the scalar-tensor, full texture, single-dimension texture reduction, and binary scalar shaders round their FP16 outputs to nearest-even. The binary scalar shaders also preserve the requested FP16 dtype when storage is emulated with FP32. Other operators retain their existing FP32 intermediate behavior on those devices. Coverage includes scalar exponents 2.0001 and 2049 and a two-node FP16 mul.Scalar chain in both texture and buffer storage. PR4 introduces `test_vulkan_dynamic.py` and its CI/Buck references. In #23253, `any.dim` now supports both `keepdim` settings on textures. With `keepdim=False`, a reduction into a temporary texture is followed by a GPU view/repack into the squeezed output shape. Both nodes resize dynamically, and empty reduced axes produce false. This adds one temporary texture and one GPU dispatch without introducing a bool-buffer requirement for texture models. Unsupported scalar inputs and 4D batch/channel axes use CPU fallback. Current rebase: Rebased onto main at `5f72739cf26e` after #23246 landed. All eight remaining code patches are unchanged, and the rebase had no conflicts. All 1153 shader variants compile with glslc, the 15 graph-builder/serialization tests pass, and lintrunner and `git diff --check` pass. Native GPU execution was validated on an earlier base; hardware and SwiftShader CI will rerun on the updated ghstack heads. Prior texture-path validation on main at `903cef063774`: the Release Vulkan/portable runtime built and eleven focused native tests passed on Apple M1 Pro / MoltenVK. Coverage included dynamic and chained any reductions, actual texture storage, scalar outputs, singleton and empty dimensions, growth after a zero-length dimension, transformer integration, supported and unsupported 4D reductions, special values, and FP16 rounding. An additional 27 FACTO-generated bool any.dim cases and six boundary cases matched ATen and portable kernels exactly through the shared `backends/test/` harness: 23 ran on Vulkan textures and 10 exercised expected CPU fallback. The subsequent review of #23245 with #23253 verified 784 support/storage combinations and eight mixed dynamic models; all nine review tests passed, with execution results matching ATen and portable kernels. These GPU execution results precede the latest upstream shader-indexing change. Earlier validation of the operator changes passed 116 FACTO cases against both ATen and portable kernels, plus 144 boundary cases against ATen, with texture and buffer preferences: all 520 Vulkan-configuration executions matched ATen, including CPU fallback where unsupported. The additional 239-case sweep covered all 91 available FACTO specifications for registered ATen overloads and 57 targeted examples. Exact-input replays of every candidate Vulkan failure on a separately built main at `a31838280f9309f04af1b375147375b702d29339` found no new regressions or unresolved comparisons. The prior full native run passed 51 tests with one expected SwiftShader-only skip. Subsequent review coverage added exact FP16 rounding boundaries, int32 buffer amax/amin range checks against both ATen and portable kernels, and 28 argmax/argmin execution and fallback cases. These are prior validation results, not a repeat of the full suite for this update. Validation uses Release portable CPU kernels and Vulkan; optimized kernels and XNNPACK are disabled. Buffer preference uses `texture_limits=(1,1,1)`, with actual storage and fallback recorded separately. Single-texel tensors can still select textures. FACTO revision: `3b8c778c99766a8b4d0d04563ae0b16cbb276829`, seed 0. Recreates #23162 through ghstack. Prior review discussion remains on that PR. Authored with OpenAI Codex; split planned with Claude Code. cc @SS-JIA @manuelcandales @digantdesai @cbilgin ghstack-source-id: 0705e5f ghstack-comment-id: 5892899249 Pull-Request: #23254
…0 pow combinations on SwiftShader and MoltenVK. [ghstack-poisoned]
mergennachin
added a commit
that referenced
this pull request
Oct 8, 2026
power_of only special-cased zero bases and used a 1e-5 tolerance to detect odd integer exponents, so NaN, infinities, signed zeros and negative bases with fractional exponents gave wrong results. It now follows ATen case by case: FP32 uses sqrt/rsqrt for exponents of plus or minus 0.5, while FP16 rounds the scalar exponent to half and uses generic pow. Tests cover exponents 2.0001 and 2049, whose half conversion changes the result category or sign. Devices without full FP16 support store half tensors as float. vTensor now records the requested dtype, and the binary scalar shaders receive it as a specialization constant so pow rounds the exponent and result to half with round-to-nearest-even. Explicit conversion also handles native FP16 stores whose rounding mode differs. These rules follow the graph's requested dtype, including force_fp16. Other operators retain their existing FP32 intermediate behavior on emulating devices. Authored with OpenAI Codex; split planned with Claude Code. ghstack-source-id: 87d3eb3 ghstack-comment-id: 5892896803 Pull-Request: #23251
This PR needs a
|
mergennachin
added a commit
that referenced
this pull request
Oct 8, 2026
Tensor-times-scalar products appear in attention scaling and were kept on CPU, splitting otherwise fully delegated blocks. Add a mul variant to the binary scalar shaders and register it with pow.Tensor_Scalar under the shared scalar-value guard. FP16 mul.Scalar rounds each result to half on every device, including devices that emulate FP16 storage with FP32. A regression test keeps two mul.Scalar nodes inside one delegate and checks intermediate rounding with exact ATen output comparison for texture and buffer storage. Part 13/15 of the Vulkan transformer and operator-conformance stack. #23251 has landed; this PR is rebased directly onto `origin/main` at `286643dbb2`. Integration PR: #23254. Test Plan: Rebuilt ExecuTorch with Vulkan enabled from the rebased source on macOS arm64. Ran `TestVulkanDynamic.test_signed_zero_scalars` and `TestVulkanDynamic.test_fp16_chained_mul_scalar` through the public Buck Python test adapter with each driver ICD explicitly selected. SwiftShader passed both tests and all 6 subcases in 15.542s; MoltenVK passed both tests and all 6 subcases in 15.620s. Both runs had zero failures, errors, or skips. Coverage includes signed-zero preservation for FP32/FP16 and exact intermediate FP16 rounding with two mul.Scalar nodes in one delegate, using both texture and buffer storage. SwiftShader exercises FP16 storage emulation. Lintrunner on all 6 changed files, `git diff --check`, and `pip check` passed. Recreates #23213 through ghstack. Prior review discussion remains on that PR. Authored with OpenAI Codex; split planned with Claude Code. cc @SS-JIA @manuelcandales @digantdesai @cbilgin
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
power_of only special-cased zero bases and used a 1e-5 tolerance to detect odd integer exponents, so NaN, infinities, signed zeros and negative bases with fractional exponents gave wrong results. It now follows ATen case by case: FP32 uses sqrt/rsqrt for exponents of plus or minus 0.5, while FP16 rounds the scalar exponent to half and uses generic pow. Tests cover exponents 2.0001 and 2049, whose half conversion changes the result category or sign.
Devices without full FP16 support store half tensors as float. vTensor now records the requested dtype, and the binary scalar shaders receive it as a specialization constant so pow rounds the exponent and result to half with round-to-nearest-even. Explicit conversion also handles native FP16 stores whose rounding mode differs. These rules follow the graph's requested dtype, including force_fp16. Other operators retain their existing FP32 intermediate behavior on emulating devices.
Part 12/15 of the Vulkan transformer and operator-conformance stack. The dependency #23250 has landed. Rebased onto
origin/mainat265ed38b340180af63b1feafed2671e623da19ef; review against main. Integration PR: #23254.Validation after rebase: Rebuilt ExecuTorch 1.6.0+87402fb from source with Vulkan enabled and PyTorch 2.14.0. Ran
TestVulkanDynamic.test_power_special_valuesthrough the public Buck Python adapter on both drivers. SwiftShader passed all 40 combinations in 98.598 seconds; MoltenVK 1.4.1 on Apple M1 Pro passed all 40 in 96.458 seconds. Both runs had zero failures, errors, or skips. Each run covers FP32 and FP16, TEXTURE_3D and BUFFER, and all ten exponents, with NaN and signed-zero checks against PyTorch. Native macOS arm64 SwiftShader at1e80438d2b93ef36a7c05f8d2b81233bac0e3d16reportsshaderFloat16 = falseandstorageBuffer16BitAccess = false, so that run exercises FP16 storage emulation. Lintrunner passed for all eight changed files;git diff --checkandpip checkpassed.git range-diffconfirms the rebase preserved the original patch.Recreates #23212 through ghstack. Prior review discussion remains on that PR.
Authored with OpenAI Codex; split planned with Claude Code.
cc @SS-JIA @manuelcandales @digantdesai @cbilgin