Pull 13.4.1 changes into main - #2789
Open
mdboom wants to merge 30 commits into
Open
Conversation
* run_cybind_cython_gen 13.4.0 ../ctk-next-public-main-staging-2026-07-15+1105 (NO MANUAL CHANGES) * run_cybind_native 13.4.0 ../ctk-next-public-main-staging-2026-07-15+1105 (NO MANUAL CHANGES) * git merge --squash ctk-next-public-main-2026-07-15+1105-merge && git rm -r -f qa/ (NO MANUAL CHANGES) * Regenerate cuda-core stubs after transfer preview (NO MANUAL CHANGES) * Support CUDA prerelease packages in fetch-ctk Resolve the public preview package set and extract it into the existing mini-toolkit layout while preserving stable redistributable behavior. * Add focused CUDA 13.4 prerelease wheel build Exercise one Linux x86_64 Python build through the existing wheel workflow without expanding the regular CI matrix. * Prepare for CTK 13.4 * Update tests to handle hidden reserved fields * Build CUDA 13.4 prerelease wheels in ordinary CI * Fix dangling pointer issues * Fix readonly and ownership bugs * CTK 13.4 fixes * Fix for CTK cache problem * Fix union types * Fix enum handling * Fix enums * Fix enum usage * Extend CUDA 13.4 prerelease CI to all four build platforms Windows preview toolkits ship as local installers rather than redistributable archives, so teach fetch-ctk to download and extract them and re-enable Windows wheel builds in the ordinary CI matrix, including a win-arm64 canary. * Assemble Windows CUDA 13.4 preview mini-CTK from installer components The local preview installer bundles per-component archives rather than a flat CUDAToolkit tree, so extract the needed component directories with 7-Zip and merge them into the layout fetch_ctk expects. * Streamline Windows preview CTK extraction in fetch_ctk Use a single 7-Zip pass, merge directly into the cache staging dir via a separate work directory, and drop prerelease-only unit tests that CI already exercises end to end. * Fix Windows preview mini-CTK lib layout for linker. Merge component libs into lib/x64 or lib/arm64 so cuda_bindings can find cudart_static.lib, and bump the prerelease cache key to drop stale flat-lib entries. * Drop layout-v2 cache key bump. Stale Windows prerelease caches were purged manually; the lib/x64 merge fix remains in fetch_ctk_redistrib.py. * Build Windows ARM64 wheels across supported Python versions Promote the successful canary to the full Python matrix and describe its CUDA 13-only wheel path as a single-major build. * Fix installation of cuda compute sanitizer * Fix for sanitizer-only install * Remove build smoke test --------- Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com> Co-authored-by: Andy Jost <ajost@nvidia.com>
* Add beta release notes * Remove documented things that don't exist in the code * Add all new content to release notes * Fix NVML docs * Revert "Bypass the doc check" This reverts commit 9f0e5d5. * Check for release notes on the tagged commit
…ve excluded paths (NO MANUAL CHANGES)
Contributor
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
github-actions
Bot
added
CI/CD
leofang
reviewed
Sep 9, 2026
- Medium: [arch_check.py:84](/wrk/forked/pr2788/cuda_python_test_helpers/cuda_python_test_helpers/arch_check.py:84) converts the raw NVML architecture integer to DeviceArch while handling an expected unsupported call. An unknown future architecture raises ValueError instead of skipping. Restore cont7’s try/except and UNKNOWN(<id>) fallback.
…se None instead of specific DeviceArch values. - Medium/low: [test_device.py:162](/wrk/forked/pr2788/cuda_bindings/tests/nvml/test_device.py:162) changed unsupported_before(device, None) to KEPLER. Mike’s pre-merge release branch and cont7 both use None; the adjacent getter does too. The current version propagates NotSupportedError on modern devices where the setter is unavailable.
Signed-off-by: Rui Luo <ruluo@nvidia.com>
… 13.3 (NVIDIA#2383)" This reverts commit 3df6115.
* Refactor: tidy cdef declarations and struct init in .pyx files (#2434)
Declare cdef locals at their point of initialization rather than at the
top of the function, and replace field-by-field struct setup with
Cython's struct-initializer syntax.
Only complete initializers are converted. Cython does not zero-fill
omitted members, so a partial initializer would leave them holding stack
garbage; sites that depend on a preceding memset are left unchanged.
Where every member is now supplied, the redundant memset is dropped.
No behavior change.
* Add pre-commit check to keep pixi cuda-version pins in sync with ci/versions.yml (#2306)
* Got initial version to address issue 2183. Let pre-commit check covers pixi cuda version pins to ci/versions.yml
* rename to be more accurate
* make error message more readable and accurate
* put cuda_bindings / cuda_core to error message to be best accurate
* add docstring
* rename cuda_feature from cu13 to cu{major} to support bumping major version, e.g. 13.x.x to 14.x.x
* add extracted line from pixi files to shown when check OK
* add concrete build version alon side with expected version
* polish to fix cosmetic
* address pre commit check
* Pin pyyaml in check-pixi-cuda-version pre-commit hook
* coverage: add cuda.core tests for system device, checkpoint, memory, stream, green context, and tensor map (#2404)
Signed-off-by: Rui Luo <ruluo@nvidia.com>
* test(core): add cuda.core.__all__ vs public docs consistency check (#2347)
* test(core): add cuda.core.__all__ vs public docs consistency check
Closes #2326.
Parses docs/source/api.rst (autosummary entries and data directives while
cuda.core is the active module) and compares the flat public names against
cuda.core.__all__ in both directions. Dotted entries such as graph.Graph or
checkpoint.Process are submodule namespaces and are excluded. Symbols
documented in api_private.rst are accepted as documented so returned-helper
docs do not fail the check.
The tests skip when cuda.core.__all__ is not defined, so this lands
independently of #2300 and activates once #2300 merges. Also adds the
__all__-names-resolve guard suggested in the #2300 review.
Signed-off-by: Aryan <aryansputta@gmail.com>
* test(core): land cuda.core.__all__ and extend docs check to public subpackages
Addresses review feedback that the consistency check was too narrow:
- Define cuda.core.__all__ (flat public namespace) so the check runs instead
of skipping, and add an aggregated __all__ to cuda.core.graph derived from
its star-imported submodules.
- Auto-discover public subpackages from cuda.core.__path__ (graph, system,
texture, utils, and any added later; the internal cuNN wheel shims are
excluded) and assert each defines a fully resolvable __all__.
- Cross-check each documented subpackage's __all__ against api.rst, handling
both the dotted (graph.Graph) and flat (currentmodule) doc conventions.
system is documented in api_nvml.rst, so its doc cross-check is skipped.
* test(core): parse API docs with docutils
* Update content to pass current tests
* Add docutils dependency to pyproject.toml
* Simplify checks. No longer make sure that everything documented is public.
* test(core): document intentional scope limits of api docs consistency check
* Reorganize __all__
* Fix doc reference
* Address findings in PR
* Fix tests and make __all__ construction consistent
* test(core): drop IPC types from _memory package contents expectation
_ipc.__all__ is now empty, so `from cuda.core._memory import *` no longer
binds IPCAllocationHandle or IPCBufferDescriptor. Update the expected list
in test_package_contents to match.
Signed-off-by: Aryan <aryansputta@gmail.com>
---------
Signed-off-by: Aryan <aryansputta@gmail.com>
Co-authored-by: Michael Droettboom <mdroettboom@nvidia.com>
Co-authored-by: Michael Droettboom <mdboom@gmail.com>
* Check for release notes on the tagged commit (#2446)
* Fix version parsing in cuda_core; Allow new enums to be added to cuda_bindings (#2451)
* Fix version parsing in cuda_core
* Fix enum checks
* chore: cython warnings as errors in cuda.core (#2441)
* chore: -Werror for cythonization in cuda.core
* chore: -Werror for cythonization in cuda.bindings
* address review feedback
* fix other cython warnings missed locally
* [doc-only] docs: document setuptools-scm clone requirements for source builds (#2424)
* docs: document setuptools-scm clone requirements for source builds
Signed-off-by: Bharat Raghunathan <bharatrgatech@gmail.com>
* Applied review suggestions from @mdboom
Signed-off-by: Bharat Raghunathan <bharatrgatech@gmail.com>
---------
Signed-off-by: Bharat Raghunathan <bharatrgatech@gmail.com>
* Add cuda-core 1.1.1 release notes (#2452)
* Add griffe to CI (#2300)
* cuda.core: add graph definition node updates (#2395)
* feat(cuda.core): add event record node updates
Use the generic node setter with failure-atomic attachment replacement, establishing the shared path for definition-level parameter mutation.
* test(cuda.core): remove unused graph update import
* feat(cuda.core): add event wait and host node updates
Extend definition-level mutation to event waits and both Python and ctypes host callbacks while preserving old executable state and attachment ownership.
* feat(cuda.core): require CUDA 12.2 for node updates
Report unsupported driver or binding versions before preparing mutation attachments or calling the generic node setter.
* feat(cuda.core): add memset node updates
Allow partial memset parameter replacement while preserving graph-owned destination lifetimes and previously instantiated graph behavior.
* feat(cuda.core): add memcpy node updates
Support partial copy parameter replacement while preserving independent source and destination ownership across graph instantiations.
* feat(cuda.core): add kernel node updates
Support independent launch configuration and argument replacement while requiring explicit arguments when changing kernels.
* feat(cuda.core): add child graph node updates
Replace embedded child hierarchies while preserving attachment metadata, invalidating stale views, and keeping existing executables independent.
* docs(cuda.core): document graph node updates
Describe supported mutation methods, CUDA 12.2 requirements, and executable graph behavior in the API and release notes.
* fix(cuda.core): avoid cross-extension deleter symbol
Use type-erased shared ownership for prepared child updates so extension loading does not depend on a hidden C++ deleter symbol.
* fix(cuda.core): align prepared child update stub
Reflect shared ownership for the opaque child update transaction in the generated stub.
* api(cuda.core): make partial node updates keyword-only
Make memcpy and memset mutation calls explicit and unambiguous before the public API freezes.
* fix(cuda.core): harden graph node updates
Preserve memory-node contexts, reject unsupported node forms, and fail clearly when child graph metadata cannot be updated.
* fix(cuda.core): support older bindings in node updates
Resolve the CUDA 13.2 graph parameter getter dynamically so CUDA 12 binding builds remain compilable.
* fix(cuda.core): harden memory node updates
Clarify parameter handling and cover host/device memory transitions while exposing context-sensitive test teardown for follow-up.
* test(cuda.core): reject updates to destroyed nodes
Cover the public invalid-node state to ensure parameter updates fail cleanly without restoring graph membership.
* docs(core): drop incorrect handle_type requirement for cuMemRetainAllocationHandle (#2418)
Defect 4 of #2388.
Signed-off-by: Aryan <aryansputta@gmail.com>
* Add identity preserving pattern to critical_sections (#2453)
* Add identity preserving pattern to critical_sections
this is annoying, but at least the library and probably kernel
attributes should be idempotent.
But critical sections *can and will be* released (similar to the GIL
although not sure what is more likely).
The important thing to note here is that the final attribute setting
section is self-contained and holds the lock (even if another thread
may have already set the attribute or still be executing the code
above).
* Use call-once pattern for module loading as double-load is problematic
As per review by Keith
* Minimal thread-unsafe initialization order fixes
* Make Windows pathfinder dynamic library searches architecture-aware (#2393)
* Make Windows pathfinder searches architecture-aware
* Avoid Windows architecture detection on Linux
* Rename unsupported architecture error
* Skip CUDA 12 wheel paths on Windows ARM64
* Restore ARM64 cudla CTK search path
* Group Windows search paths by architecture
* Add architecture-specific Windows path tables
* Make Windows CTK libnames architecture-aware
* Add Windows architecture availability helper
* Correct cuSPARSELt Windows ARM64 wheel path
* Correct Windows CTK NVVM and CUPTI paths
* Validate Windows NVVM binary architecture
* Remove Windows architecture availability helper
* Remove site-package catalog generation tool
* Move pathfinder changes to 1.6.1 release notes
* Remove pathfinder catalog generation tools
* Restore site-packages collection scripts
* Use platform-specific supported library names
* Regenerate pathfinder 1.6.1 release notes
* Remove redundant Windows libname consistency test
* Mark agent-authored pathfinder tests
* Clarify Windows binary architecture validation
* Use all available dynamic library names
* Test Windows site-package libraries by architecture
* Require exactly one Windows architecture flag
---------
Co-authored-by: Michael Wang <isVoid@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
* fix(cuda.core): surface real CUDA error in Device.set_current() (#2461)
* cuda.core: fix host memcpy source pointer cast (#2476)
* CI: avoid SIGPIPE when selecting core release tag (#2475)
* Make pre-commit work on Windows (#2327)
* Make pre-commit work on Windows
* Update .pre-commit-config.yaml
* Address some of the comments in the PR
* Simplify type-checking
* Address comments in PR
* Simplifications
* Fix simplifications
* Fix type check
* Add comment about stubgen-pyx issues
* Update CONTRIBUTING.md
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com>
* Update cuda_pathfinder/cuda/pathfinder/_dynamic_libs/load_dl_windows.py
---------
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
* Bumpy cython dependency to 2.3.5 (#2469)
As Keith noted, this is needed for using the `py_safe_call_once`
definitions, Cython 3.2.5 changelog:
https://cython.readthedocs.io/en/latest/src/changes.html
(I guess the bump in the pre-commit is likely not strictly needed, but
there also were no stub changes.)
* ci: drop custom NumPy and use beta 4 for Python 3.15 (#2411)
* ci: drop custom NumPy builds for Python 3.15
Signed-off-by: tirthpatel90 <tirthpatel5393@gmail.com>
* ci: enable scientific-python-nightly-wheels index for Python 3.15
Signed-off-by: tirthpatel90 <tirthpatel5393@gmail.com>
* ci: set PIP_ONLY_BINARY and relax numpy version pin for Python 3.15
Signed-off-by: tirthpatel90 <tirthpatel5393@gmail.com>
* Fix NumPy version in pyproject.toml and try re-adding windows python 3.15
* Bump cibuildwheel to 4.1.1 (which uses containers with 3.15 b4)
* Revert "Use Python 3.15b2 for now until cibuildwheel is updated (#2433)"
This reverts commit 3ef82d6892d27cd81b9f3f675a00cfc34b9b90d8.
* Add allow-prereleases to windows CI to try and run 3.15
* Exclude ml-dtypes from windows (builds in 1 minute on linux so kept it)
* Drop windows 3.15t again as psutil doesn't have free-threaded wheels
---------
Signed-off-by: tirthpatel90 <tirthpatel5393@gmail.com>
Co-authored-by: Sebastian Berg <sebastianb@nvidia.com>
* Remove tests involving NVLINK_MAX_LINKS (#2483)
* Make temperature threshold checks forward-compatible (#2488)
Compare raw NVML device architecture values so architectures newer than the generated DeviceArch enum do not raise ValueError before the threshold query. Add regression coverage for an unrecognized architecture value.
* fix(cuda.core): fall back to driver when nvJitLink < 12.3 is installed (#2409)
* fix(cuda.core): fall back to driver when nvJitLink < 12.3 is installed
Stop probing nvJitLink availability via module.version(), which calls
the unversioned nvJitLinkVersion symbol missing in nvJitLink 12.0-12.2.
Use symbol pointer inspection via _nvjitlink_has_version_symbol()
instead, restoring cuda-core 0.6.0 fallback behavior.
Fixes #2408
Signed-off-by: Omar Atie <atiaomar1978-hub@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(cuda.core): add coverage for nvJitLink <12.3 driver fallback
Add regression tests for Linker.which_backend() and
_decide_nvjitlink_or_driver() when the nvJitLinkVersion symbol is
missing (nvJitLink 12.0-12.2).
Related to #2408
Signed-off-by: Omar Atie <atiaomar1978-hub@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* docs(cuda.core): add 1.2.0 release note for nvJitLink <12.3 fallback fix
Document the #2408 regression fix in the cuda.core 1.2.0 release notes.
Related to #2408
Signed-off-by: Omar Atie <atiaomar1978-hub@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(cuda.core): probe nvJitLink version under DynamicLibNotFoundError guard
Address review feedback: keep the >=12.3 version-symbol check inside
_optional_cuda_import's probe so a missing nvJitLink dylib still falls
back to cuLink. Continue avoiding module.version(), which raises
FunctionNotFoundError on nvJitLink 12.0-12.2 (#2408).
Add coverage for missing-dylib fallback and a guard that the probe does
not call module.version().
Signed-off-by: Omar Atie <atiaomar1978-hub@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(cuda.core): use explicit try/except for nvJitLink version probe
Address review feedback: drop the probe side-effect and catch
DynamicLibNotFoundError around _nvjitlink_has_version_symbol so missing
dylibs still fall back to cuLink. Keep avoiding module.version() for
nvJitLink <12.3 (#2408).
Mark newly added tests with agent_authored authorship markers.
Signed-off-by: Omar Atie <atiaomar1978-hub@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(cuda.core): drop obsolete nvJitLink probe comments
Signed-off-by: Omar Atie <atiaomar1978-hub@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Signed-off-by: Omar Atie <atiaomar1978-hub@users.noreply.github.com>
Co-authored-by: Omar Atie <atiaomar1978-hub@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Michael Wang <13521008+isVoid@users.noreply.github.com>
* Fix #2377: Reorganize the test helpers (where appropriate) to cuda_python_test_helpers (#2384)
* Experiment: Install test_helpers as a package
* Try something different in CI
* Reorganize all the tests
* Update a few more imports
* docs: update nvshmem4py link to /api/latest/ path (#2492)
NVSHMEM docs now live under /nvshmem/api/latest/; the old unversioned
deep link 404s and breaks lychee on rendered docs.
* Add mitigations for intermittent test failures with CUDA OOM (#2484)
* Add context sync to teardown in init_cuda fixture
* Cap memory pool size in some tests
* cuda.core: return CUmodule via as_py in ObjectCode.get_module (#2481)
* cuda.core: return CUmodule via as_py in ObjectCode.get_module
Use the shared handle export path for legacy CUmodule interop instead of
constructing driver.CUmodule directly.
Signed-off-by: Jinfeng <jinfengl@nvidia.com>
* cuda.core: drop unused intptr_t import in _module.pyx
Satisfy cython-lint after switching get_module() to as_py().
Signed-off-by: Jinfeng <jinfengl@nvidia.com>
* cuda.core: add as_intptr overload for CUmodule
Route as_py(CUmodule) through as_intptr for consistency with other handle exports.
* let as_cu supports CUModule
---------
Signed-off-by: Jinfeng <jinfengl@nvidia.com>
* cuda.core: resolve default-stream context per call instead of caching it (#2490)
* cuda.core: resolve default-stream context per call instead of caching it
LEGACY_DEFAULT_STREAM and PER_THREAD_DEFAULT_STREAM wrap default-stream
tokens, which denote whatever context is current. Both are module-level
singletons, and Stream_ensure_ctx / Stream_ensure_ctx_device stored the
first context and device they observed on the object and never cleared
them, so a process-wide object became permanently bound to one context.
Replace the two helpers with resolvers that return the context and device
through out-parameters and cache on the object only when the stream is not
a default-stream token. Stream.context, .device, .resources, .record(), and
__repr__ now follow the current context, a query no longer pins a context
reference for the lifetime of the process, and the shared singletons are no
longer written to from multiple threads.
Object identity is preserved, so __eq__ and __hash__ keying off the handle
are unaffected.
Fixes #2485
* fix(cuda.core): harden default-stream context resolution for #2485
Skip sticky context reuse on default-stream tokens, document ambient
context behavior on device/resources/record, and cover resources in the
multi-GPU regression test.
---------
Co-authored-by: Andy Jost <ajost@nvidia.com>
* build: report internal build dependency provenance (#2509)
* Fix nvbug6550424: Don't check NVML init behavior on CTK >= 13.4 (#2512)
* cuda.core: add executable graph node updates (#2473)
* Add executable graph attachment ownership
Install a private CUDA user object per graph executable so later node updates can retain replacement resources safely.
* Add executable graph node updates
Expose ephemeral graph-node views that update complete executable parameters while retaining every replacement resource CUDA may still use.
* test(core): cover executable graph node updates
Exercise public mutators, rollback, source reclamation, independent ownership, whole updates, and in-flight cleanup end to end.
* refactor(core): own exec graph creation behind one handle function
Instantiation and whole-graph update each went through a prepare/commit
pair. That exposed an opaque transaction type over the internal C++
interface and split the exec ownership contract between C++ and Cython,
unlike every other resource handle, which a single create_* function
owns end to end.
Replace the pairs with create_graph_exec_handle and graph_exec_update.
Each stages a fresh attachment accumulator on the source graph, makes
the CUDA call with the GIL released, and adopts or publishes the result,
so the staging transaction becomes a stack guard in the anonymous
namespace instead of a header type. Cython keeps only what belongs to
it: filling the instantiation params and decoding the failure reasons.
The two driver entry points move into the C++ loader table with the
calls.
Convert the attachment append transaction to the unique_ptr plus
rollback deleter pattern that node attachments already use, which
retires the committed flag in favor of the same release-and-delete
mechanism. Drop GraphExecBox::attachment_object, which nothing reads.
* test(core): cover executable attachment accumulator lifetimes
Three gaps remained around owners attached to an executable graph.
Sequential updates to the same node must keep the superseded owner
reachable, because CUDA cannot detach user objects from an executable;
verified by breaking the append into a replace, which fails the new
test on exactly that assertion.
Closing an executable while a launch is in flight must not retire the
accumulator, since the launch still writes through the buffer that an
individual node update attached.
A child-graph update attaches no owner of its own and relies on CUDA
cloning the replacement graph's user object references into the
executable. Assert that contract directly: the callback outlives the
definition that supplied it and is released with the executable.
* docs(core): describe the executable attachment accumulator
CUDA accepts user objects on a CUgraph only, so an executable graph can
never receive an owner after it exists. Document the consequence: one
accumulator is retained on the source graph, propagated by instantiation
or whole-graph update, and then released from the source so the
executable becomes its only owner.
Record why an owner is never removed once appended, and correct the two
Scope entries that still described executable graphs as untracked.
State the retention limit in the release notes as well. The API
reference already documents it, but the note is what a reader sees when
adopting the feature, and retention that looks unbounded deserves the
warning there.
* docs(core): focus executable attachment docs on cuda.core behavior
Describe retention and complete-replacement rules without framing the
notes around CUDA limitations, and shorten the executable attachment
design section to problem, solution, and append-vs-replace limits.
* docs(core): clarify Graph.__getitem__ for executable node updates
Describe how callers use the view rather than how the binding retains
handles or when CUDA validates the node association.
* Address review feedback on executable graph attachments
Clarify attachment ownership and deferred-cleanup docs, trim the
invariants list to the cross-cutting rules, initialize instantiate
params with Cython struct syntax, and simplify the executable update
test helper.
* test(core): restore explicit executable update kwargs
The shared replacement fixture carries extra fields that are not update
parameters, so spreading it as kwargs breaks memset and kernel cases.
* Add check and agent guidance about uncapped mempools (#2514)
* Add check and agent guidance about uncapped mempools
* Address review: move mempool check to pre-commit, share POOL_SIZE
Review feedback on #2514:
- Move the uncapped-pool check out of the live test suite into a
check-mempool-hygiene pre-commit hook. The rule is about source text and
needs no GPU, so a hook catches it earlier and for free. Its tests move to
ci/tools/tests, alongside the other check scripts'.
- Add helpers/constants.py and route the eleven ad-hoc POOL_SIZE definitions
through it.
- Qualify "device memory" as installed/physical where the doc explains what an
uncapped pool reserves.
* build: report cuda-bindings provenance in PEP 517 builds (#2520)
Resolve the CUDA path before importing cuda.bindings so the existing pathfinder import repairs PEP 517 namespace shadowing first. Reuse the resolved path for the CUDA include directory.
* Make Windows static library searches architecture-aware (#2491)
* Make static library discovery architecture-aware
* Use architecture-specific static library paths
---------
Co-authored-by: Michael Wang <isVoid@users.noreply.github.com>
* cuda.core: add LaunchConfig.programmatic_stream_serialization for programmatic dependent launch (#2456)
* feat(cuda.core): expose PDL via LaunchConfig.programmatic_stream_serialization
Allow users to set CU_LAUNCH_ATTRIBUTE_PROGRAMMATIC_STREAM_SERIALIZATION
through LaunchConfig, matching the is_cooperative attribute pattern (#1334).
* test(cuda.core): verify PDL overlap for primary/secondary launch
Add an end-to-end Hopper+ test that launches primary and secondary kernels
on the same stream with programmatic_stream_serialization, and asserts
overlap only when the PDL attribute is enabled (#1334).
* test(cuda.core): simplify PDL secondary kernel and log success
Drop unused secondary sync/sleep from the overlap test, clarify the
primary clock window comment, and print a short success line for CI.
* add pre-commit passed
* revise to xfail
* Remove last remnants of Python 3.9 support (#2394)
* Match private extension modules against the in-package path only (#2504)
check_cython_abi's private-module filter tested `so_path.parts` on the
absolute path, so any ancestor directory starting with an underscore made
every module look private. That is the normal layout under manylinux
(/opt/_internal/cpython-*/) and in GitHub Actions containers (/__w/), where
`generate` then writes zero ABI files and exits 0 -- a green run with no
coverage at all.
`check`'s new-module scan had no filter, while `generate` skipped private
modules. Since `generate` never wrote an .abi.json for them, `check` reported
every private module as "New module added" on every run and set
has_allowed_changes, so it could not print "No changes found" for a package
shipping private submodules (cuda.bindings has _bindings/, _internal/, _lib/).
Extract the predicate into iter_public_extension_modules() so both paths use
it, and match only on the path relative to the package root.
* Migrate cuda_pathfinder tests from os.path to pathlib (#2495)
Part 4 of the series proposed in #2410.
Filesystem predicates and path joining in the pathfinder tests now go through
pathlib: os.path.isfile/isdir become Path.is_file()/is_dir(), os.path.basename
becomes Path.name, os.path.join becomes Path joining, and the site-packages
check uses Path.parts instead of splitting on os.path.sep.
site_pkg_rel.replace("/", os.sep) is dropped in test_find_static_lib.py: Path
already accepts forward slashes on Windows.
Two files are left out on purpose. test_find_nvidia_binaries.py moves with
part 3, whose signature changes it depends on. test_search_steps.py is being
edited by #2489 (part 1), so converting it here would only create a conflict.
Left on the stdlib modules: glob.glob in test_find_nvidia_headers.py, which
expands an absolute pattern from the header catalog (Path.glob needs a base
dir, and the wildcard is not pinned to the last component); os.pathsep in
test_ctk_root_discovery.py, which builds PYTHONPATH, not a path; and os.sep in
test_utils_env_vars.py, which builds a trailing separator on purpose.
Signed-off-by: LeSingh1 <sshaurya914@gmail.com>
* coverage: repair the Windows coverage wheels (#2508)
Windows coverage has not collected a test since 2026-03-17. The job builds
its wheels with a plain `pip wheel`, which never reads [tool.cibuildwheel],
so the delvewheel repair every other Windows build performs never ran here.
Those wheels import a bare "MSVCP140.dll" and resolve it against whatever the
test machine has in System32, which on the coverage runner is 14.00.24215.1,
built in 2015.
_resource_handles.pyd is compiled by MSVC 14.44 and imports exactly _Mtx_lock
and _Mtx_unlock from that DLL -- never _Mtx_init_in_situ, because std::mutex
has had a constexpr constructor since VS 2022 17.10. The 2015 runtime still
expects that initialisation and dereferences a null handle on the first lock,
which _stream.pyx takes while cuda.core is still importing. It is the only
extension module in either package that locks a mutex, which is why
cuda.bindings and cuda.pathfinder have always passed on the same machine.
Repairing the wheels vendors msvcp140 14.44 into cuda_core.libs and rewrites
the import tables to match, so the process no longer depends on what the test
machine carries. Verified on the coverage runner: 18 failed, 2929 passed,
918 skipped in 346s, against three to seven seconds of dying beforehand, and
the first Windows coverage data since March.
The same commit pins cuda-bindings to the wheel built one step earlier.
PIP_PRE is set so pip will consider that wheel at all -- it carries a .devN
version -- but it also admits PyPI's pre-releases, and cuda-bindings 13.4.0b1,
published 2026-07-29, outranks the local build. Its cydriver.pxd comes from
CTK 13.4 headers where CUmemLocation has a `localized` field, while cuda.core
compiles against the 13.3.0 mini-CTK where it does not, so the build has been
failing on `error C2039` ever since.
Signed-off-by: Rui Luo <ruluo@nvidia.com>
* cuda.core: accept ProgramOptions(name=None) (#2517)
* cuda.core: accept ProgramOptions(name=None)
ProgramOptions.name is annotated str | None, but __post_init__ called
.encode() on it unconditionally, so passing None raised AttributeError
before any CUDA call was reached.
Normalize None to the documented default, matching how arch is handled
in the same method. The encoded value is identical to the existing
default path, so the bytes passed to nvrtcCreateProgram are unchanged.
Signed-off-by: Aryan <aryansputta@gmail.com>
* cuda.core: cover name=None through compile and add a release note
Extend coverage past ProgramOptions construction to assert the
normalized name reaches ObjectCode.name, matching the shape of
test_program_compile_valid_target_type.
Signed-off-by: Aryan <aryansputta@gmail.com>
* cuda.core: drop the redundant compile-level test and the name constant
ObjectCode.name receives an already-normalized options.name, so the
compile-level assertion could not fail independently of the options
test. Inline the default literal instead of a module constant, which
kept a private symbol out of the generated stub.
Signed-off-by: Aryan <aryansputta@gmail.com>
---------
Signed-off-by: Aryan <aryansputta@gmail.com>
Co-authored-by: Michael Droettboom <mdboom@gmail.com>
* cuda.bindings: Fix API status handling (#2530)
* cuda.bindings tests: enable BAR lookup test on GH200
* cuda_bindings fixes
---------
Co-authored-by: Ralf Juengling <rjuengling@utskinnyjoe-dvt-65.ipp2u1.colossus.nvidia.com>
* cuda.core: validate pinned host memory pool support (#2487)
* cuda.core: validate pinned pool support
Reject unsupported host memory pools during allocation instead of allowing a later copy to fail with CUDA_ERROR_INVALID_VALUE.
Signed-off-by: Uday Arora <udaya@nvidia.com>
* cuda.core: tighten pinned host pool capability check
Drop the unnecessary CUDA 12 fence around host_memory_pools_supported,
raise RuntimeError instead of a synthetic CUDAError, and keep the
regression test hardware-gated for devices without host memory pools.
---------
Signed-off-by: Uday Arora <udaya@nvidia.com>
Co-authored-by: Andy Jost <ajost@nvidia.com>
* cuda.core: report host accessibility for NUMA-located VMM resources (#2503)
`VirtualMemoryResource.__init__` classifies "host", "host_numa" and
"host_numa_current" all as host-located (it clears `self.device` for
each), but `is_host_accessible` compared with `== "host"`. A resource
configured with `location_type="host_numa"` or `"host_numa_current"`
therefore reported `is_host_accessible is False` *and*
`is_device_accessible is False` -- an impossible answer that propagates
to `Buffer.is_host_accessible`, which forwards to the memory resource.
Share a single `_HOST_LOCATION_TYPES` set between the constructor and
the property so the two classifications cannot drift again.
* cuda.core: validate ctypes host callback signatures against CUhostFn (#2525)
* cuda.core: validate ctypes host callback signatures against CUhostFn
Reject incompatible ctypes prototypes before CUDA sees them, document
the required ABI, and note the stronger checking in the 1.2.0 release notes.
* cuda.core: make ctypes flag lookups stubgen/mypy-friendly
Use getattr for private ctypes calling-convention constants so the
regenerated _host_callback.pyi type-checks cleanly.
* cuda.core: check host callback prototypes via public ctypes attributes
The previous check inspected ctypes' private _flags_ bits to identify the
calling convention. That is wrong on Windows: CPython defines
FUNCFLAG_STDCALL as 0, so a bitwise test can never match WINFUNCTYPE, and
every win-64 test job rejected a valid callback. The 0x2 fallback used when
_ctypes.FUNCFLAG_STDCALL is absent is FUNCFLAG_HRESULT, not stdcall.
Drop the calling-convention check rather than repair the bit arithmetic.
ctypes only honors stdcall when building a callback on 32-bit x86 Windows,
which cuda.core does not support, and FUNCFLAG_PYTHONAPI is never consulted
on the callback path, so CFUNCTYPE, WINFUNCTYPE, and PYFUNCTYPE all yield the
same FFI_DEFAULT_ABI thunk. That leaves the declared result and argument
types, which are reachable through the public restype/argtypes attributes.
Reading those public attributes also lets a function pointer taken from a
shared library be accepted once its restype and argtypes are declared, which
the class-level lookup could never see.
* chore: avoid some warnings when running cuda.core tests (#2515)
* ci: constrain internal builds to exact local wheels (#2510)
* ci: constrain internal builds to exact local wheels
* ci: keep CI tool tests in nightly workflow
* ci: generate local wheel constraints in workflows
* fix(pathfinder): place Windows arm64 cudart test fixtures under bin/arm64 (#2528)
* fix(pixi): restore conda test deps and relock after #2384 (#2532)
#2384 inserted a pypi-dependencies header mid-table, moving conda test
deps to PyPI without updating lockfiles. Fresh CI installs then dropped
the local cuda-bindings/cuda-core source packages, causing ModuleNotFoundError.
* ci: limit pytest duration reports to the slowest 20 tests (#2523)
* ci: limit pytest duration reports to the slowest 20 tests
--durations=0 prints every test phase and floods CI logs. Report only
the slowest 20 instead.
* ci: set pytest --durations=20 via package config defaults
Move the duration limit into pytest addopts so CI and local runs
share one default, instead of repeating --durations on every command.
* Fix SM resource alignment discovery test (#2389)
* Fix SM resource alignment discovery test
* Refine SM discovery alignment coverage
* Document CUDA 13.4 SM discovery behavior
* feat(security): onboard security-suite (secret + CodeQL) scanning. (#2589)
Call the centrally maintained NVIDIA/security-workflows security suite rather
than wiring each scan separately: one pinned reference runs the Pulse secret
scan and CodeQL SAST, both explicitly enabled.
Replace .github/workflows/codeql.yml with the suite's SAST scan. Both publish
code scanning results under the category /language:python, so keeping the local
workflow would put two analyses on every commit that overwrite each other's
alerts. The suite performs the same analysis: python, build-mode none,
security-extended queries, on ubuntu-latest.
* fix(cuda.core): avoid truncating graph queries (#2587)
* fix(cuda.core): avoid truncating graph queries
* perf(cuda.core): retain adjacency stack buffer
* test(cuda.core): cover large predecessor graph queries
Verify exact edge identities so graph query regressions cannot pass through count-only checks.
---------
Co-authored-by: Andy Jost <ajost@nvidia.com>
* ci: add selective wheel build plumbing (#2464)
* Fix Windows binary utility discovery on Arm64 (#2586)
* Fix Windows binary utility discovery on Arm64
* Clarify binary utility search order
* Expand standalone installation documentation
* Align standalone search step comments
* Preserve literal Nsight launcher lookup
* Cover Windows binary discovery fallbacks
* Document Windows architecture selection
* Harden Windows Arm64 utility discovery
* Fix Windows pre-commit checks
* Fix CUDA path precedence documentation
* Document Windows binary utility discovery
---------
Co-authored-by: Michael Wang <isVoid@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
* Use pathlib in cuda.pathfinder._static_libs (part 2 of #2410) (#2493)
* Migrate _static_libs finders from os.path to pathlib
Part 2 of the series proposed in #2410, following the same conversion
style as part 1 (#2489).
Path construction, joining, and filesystem predicates in
find_static_lib.py and find_bitcode_lib.py now go through pathlib.Path
instead of os.path string manipulation. Both modules keep importing os
solely for os.environ.get("CONDA_PREFIX").
Compatibility is preserved: every entry point still accepts str, and
every function that documents or returns str still returns str. Path is
used strictly as the internal representation and converted back with
str() at each return, so LocatedStaticLib.abs_path, LocatedBitcodeLib
.abs_path, find_static_lib() and find_bitcode_lib() are unchanged in
both type and value. No signature changes.
Signed-off-by: LeSingh1 <sshaurya914@gmail.com>
* Return Path from the _static_libs internals
Follow-up to the review feedback on #2489: the str-compatibility constraint
applies only to the public API.
The try_* methods and _no_such_file_in_dir now work in Path throughout. str()
is applied once, where abs_path is stored on the public LocatedStaticLib and
LocatedBitcodeLib. The relative-path constants go from os.path.join(...) to
forward-slash literals, matching how site_packages_dirs is already written in
the same dicts; Path normalizes the separator on Windows.
One behavior change: a CUDA_PATH or CONDA_PREFIX containing redundant
separators ("//", "/.") now produces a normalized abs_path, because Path
collapses them. Differential fuzzing against the pre-revision code (16k lookups
over randomized trees, comparing located paths and full error text) shows no
other difference, and none at all when those variables are free of redundant
separators.
Signed-off-by: LeSingh1 <sshaurya914@gmail.com>
---------
Signed-off-by: LeSingh1 <sshaurya914@gmail.com>
Co-authored-by: Michael Droettboom <mdboom@gmail.com>
* chore: fix Apache-2.0 license notice and attribution gaps (#2605)
* chore: fix Apache-2.0 license notice and attribution gaps
An open-source license review flagged several Apache-2.0 compliance gaps.
This addresses three of them, plus the guard that let one class of them
through. Licensing metadata only; no logic changes.
Copyright notices (15 files)
Two different defects that happened to share a symptom:
- 14 files under cuda_bindings/examples/ carried a non-standard notice
("Copyright 2021-2026 NVIDIA Corporation. All rights reserved.") with
no (c), no SPDX-FileCopyrightText prefix, and the wrong entity casing.
- toolshed/conda_create_for_pathfinder_testing.ps1 had the correct prefix
and casing but was truncated before "& AFFILIATES. All rights reserved.".
All now carry the canonical string. Years are preserved as found.
Header guard (toolshed/check_spdx.py)
COPYRIGHT_REGEX made "& AFFILIATES. All rights reserved." optional, so a
bare "NVIDIA CORPORATION" satisfied pre-commit. The suffix is now
required. (The 14 example files were passing for a different reason:
.spdx-ignore excludes cuda_bindings/examples/ entirely. That exclusion is
left alone here, but the files now conform, so it can be dropped in a
follow-up if desired.)
Tightening the regex surfaced two pre-existing files whose notice was
split or truncated -- cuda_core/cuda/core/_include/layout.hpp and
toolshed/build_static_bitcode_input.py. Both are corrected so the
mandated sentence appears verbatim on one line.
Third-party attribution (cuda_core/NOTICE)
cuda/core/_include/aoti_shim.h is a vendored subset of PyTorch's AOT
Inductor stable C ABI, BSD-3-Clause, carrying the upstream Facebook,
Idiap, Deepmind, NEC and NYU copyright lines, but NOTICE listed only
DLPack. A PyTorch entry is added with the full copyright block. The
accompanying aoti_shim.def carries no copyright line of its own and is
covered explicitly by that entry rather than given an NVIDIA header,
since it declares the same upstream symbol names. The DLPack entry now
also records where it is vendored.
LICENSE files (all five)
Every LICENSE ended at "END OF TERMS AND CONDITIONS", omitting the
required "APPENDIX: How to apply the Apache License to your work" and
its boilerplate. Appended to all five. The text is verified identical
to the canonical Apache 2.0 appendix.
Verified: 0 files with a non-conforming copyright string; check_spdx.py
passes over all 868 in-scope tracked files with the tightened regex.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Rob Parolin <rparolin@nvidia.com>
* docs: document per-subproject license files in root README
OSRB (NVBUG 4707569, comment #22) flagged the four sub-component LICENSE
files as redundant with the root LICENSE and asked for either their removal
or a root README Licensing section naming each subproject, its license and
its license path.
Each subproject builds an independent wheel and resolves its license file
relative to its own root, so the copies are kept and documented instead of
removed. Verified that the copies reach the built wheels: building
cuda_pathfinder produces dist-info/licenses/LICENSE even though its
pyproject.toml declares no explicit license-files (setuptools' default
LICEN[CS]E* glob covers it), as is also the case for cuda_core.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Signed-off-by: Rob Parolin <rparolin@nvidia.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Use pathlib in toolshed and ci helper scripts (part 7 of #2410) (#2496)
* Migrate toolshed and ci helper scripts from os.path to pathlib
Part 7 of the series proposed in #2410.
Path joining and filesystem predicates in the toolshed and ci/tools helper
scripts now go through pathlib. glob.glob in dump_cutile_b64.py becomes
Path.glob, with the mtime key reading Path.stat().
Kept on os.path, with a comment where it is not obvious:
- os.path.abspath in build_static_bitcode_input.py, since sys.path wants a str
and Path.absolute() does not normalize.
- os.path.isfile in check_generated_file_seals.py. That guard exists to skip
anything that is not a readable regular file, and Path.is_file() is not a
drop-in: it propagates OSError for errnos outside pathlib's ignore list
(EACCES, ENAMETOOLONG) where os.path.isfile returns False.
- os.path.normpath in check_spdx.py, which already carries its own comment.
The plan on #2410 also listed a root conftest.py; there is no such file. The
three conftest.py files live under cuda_pathfinder, cuda_core and
cuda_bindings, and none of them use os.path.
Verified locally: ci/tools/tests/test_check_release_notes.py passes (42
tests), and check_spdx.py and check_generated_file_seals.py produce output
identical to the pre-change scripts when run over every tracked .py file.
Signed-off-by: LeSingh1 <sshaurya914@gmail.com>
* Return Path from notes_path; use Path.is_file in seal checker
Per review: treat these helper scripts as private, so notes_path can return
Path and drop the str/Path round-trip at its call site. Accept the behavioral
change from os.path.isfile to Path.is_file in check_generated_file_seals.
* Review: thread Path through check_release_notes, drop remaining os.path
Follow-up to mdboom's review.
- repo_root is now a Path end to end: load_backport_branch, check_release_notes
and validate_backport_decision take Path, and --repo-root parses with
type=Path. That removes the Path(repo_root) re-wrap inside the functions and
the 19 str(tmp_path) conversions the tests needed to call them. The five
main() argv lists keep str(): those are command-line strings, which argparse
then turns back into a Path.
- build_static_bitcode_input: the last os.path use (os.path.abspath) becomes
Path.resolve(); the os import is now unused and is dropped.
---------
Signed-off-by: LeSingh1 <sshaurya914@gmail.com>
Co-authored-by: Michael Droettboom <mdboom@gmail.com>
* Fix operator precedence in test_cudart.supportsCudaAPI (#2554)
def supportsCudaAPI(name):
return name in dir(cuda) or dir(cudart)
parses as `(name in dir(cuda)) or dir(cudart)`. `dir(cudart)` is a non-empty
list for any module, so it is unconditionally truthy and the function returns
a truthy value for every input, including names that exist nowhere.
The left operand is dead too: `cuda` is cuda.bindings.driver and every name
passed in is a cudaXxx runtime symbol. cudaGraphGetId, cudaGreenCtxCreate,
cudaDeviceGetExecutionCtx and cudaGraphConditionalHandleCreate are all defined
in runtime.pyx and appear nowhere in driver.pyx, so `name in dir(cuda)` is
always False and the result is always the `dir(cudart)` list.
Consequence: `not supportsCudaAPI(...)` is always False, so the API-presence
half of all 17 skipif guards that use it (lines 1443-1954) never fires. On a
build whose bindings genuinely lack the API, the test runs and dies with
AttributeError instead of skipping; only the driver_version_less_than() half
of each guard does any work.
Adds test_supportsCudaAPI, pinning all three cases: a runtime-only name, a
driver-only name, and a name that exists in neither. The last two fail before
this change.
* Catch up to current generator main (#2603)
* Catch up to current cybind main
* Bugfix for get_buffer_pointer
* Fix the field-id enum name in the nvml test helper supports_nvlink (#2560)
def supports_nvlink(device):
fields = nvml.FieldValue(1)
fields[0].field_id = nvml.FI.DEV_NVLINK_GET_STATE
There is no `FI` attribute on cuda.bindings.nvml. The enum is `FieldId`
(nvml.pyx:1229), with DEV_NVLINK_GET_STATE at nvml.pyx:1454, and the sibling
test uses the correct spelling: test_nvlink.py:19 does
`fields[0].field_id = nvml.FieldId.DEV_NVLINK_LINK_COUNT`.
So the helper raises AttributeError on its first line of real work. Nobody
has noticed because it has no callers -- a repo-wide grep for
`supports_nvlink` finds only its own definition. Contrast util.supports_ecc,
which is called from test_page_retirement.py.
Adds tests/nvml/test_util.py, which stubs nvml.device_get_field_values so the
helper can be exercised without an NVLink-capable device, and asserts both
that it returns True and that it queried FieldId.DEV_NVLINK_GET_STATE. It
fails with AttributeError before this change.
* fix(core): don't crash `import cuda.core` on a non-integer opt-out value (#2535)
`cuda/core/__init__.py` reads `CUDA_CORE_DONT_FIX_TAB_COMPLETION` with a
bare `int(os.environ.get(..., "0"))` at import time. `int()` raises for any
value that is not a base-10 integer, and `os.environ.get` returns the empty
string (not the `"0"` default) when the variable is set but empty, so:
export CUDA_CORE_DONT_FIX_TAB_COMPLETION=
python -c "import cuda.core"
ValueError: invalid literal for int() with base 10: ''
Clearing a variable with `export VAR=` is the usual way to neutralize it in
a shell profile, a Dockerfile, or a CI job spec, and `=true` / `=yes` are
the obvious guesses for a boolean-looking opt-out. All of them make the
whole package unimportable, which is a hard failure for a knob whose only
purpose is to skip an optional `rlcompleter` patch.
Parse the value leniently instead. Integer values keep their existing
meaning (non-zero opts out, so `0` and `00` still install the patch), while
a non-integer, non-empty value is honored as an opt-out rather than being
silently ignored. Unset and empty/whitespace-only both mean "not set".
Also document the variable, which was not listed on the environment
variables page, and drop the stale "only installed in interactive mode"
comment: the interactivity gate was intentionally removed in #2055 ("Always
install the monkeypatch"), so the patch has been unconditional since then.
The new parametrized test asserts the resulting behavior for eight values;
four of them ("", " ", "true", "yes") fail on main because the subprocess
exits non-zero with the ValueError above.
Co-authored-by: Michael Droettboom <mdboom@gmail.com>
* PERF: Inline return code checks in cuda_core (#2608)
* test: skip LatchKernel event test without __nanosleep (#2611)
* docs(cuda.core): use PEP 604 unions in docstrings (#2601)
Docstrings across cuda_core still spelled parameter types with the
pre-3.10 typing generics. Replace Union[...] and Optional[...] with the
| form the rest of the package already uses, e.g. `stream : Stream |
None, optional` in _memoryview.pyx.
Docstrings only, so the .pyi changes are the stubgen-pyx output for the
edited .pyx files and no runtime behavior moves. In _module.pyx this also
realigns the max_potential_block_size docstring with its signature, which
already reads int | driver.CUoccupancyB2DSize.
Two code-level spellings stay as they are:
- LinkerHandleT in _linker.pyx is a runtime value, not an annotation.
_program.pyx builds ProgramHandleT from it with `nvrtc.nvrtcProgram |
int | LinkerHandleT`, and PEP 604 `|` on the forward-reference strings
it holds raises TypeError.
- The union_type literal in _process_define_macro is error-message text
rather than a docstring.
Sequence[...] and Iterable[...] elsewhere in cuda_core are
collections.abc generics and are unaffected.
Signed-off-by: Aryan <aryansputta@gmail.com>
Co-authored-by: Michael Droettboom <mdboom@gmail.com>
* Use subtests where appropriate everywhere (#2391)
* Use subtests where appropriate everywhere
* Fix test
* Test every fan in a separate subtest
* Recognize skipped pytest subtests in CI logs
* Isolate independent inner test cases
* Narrow the cooler unsupported-call scope
* Contain fan-count failures per device
* Use stable identifiers for device subtests
* Fold nested subtest context managers
* Preserve the existing power-limit getter guard
* Guard memory affinity on pre-Kepler devices
* Keep invalid subtest results contained
---------
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
* [no-ci] Support organization-owned forks in AGENTS.md (#2378)
* Update PR guidance for organization-owned forks
* Clarify agent remote-write policy
* Add fork-aware pull request skill
* Do not report a registered type as "Unknown type" in get_cuda_native_handle (#2551)
get_cuda_native_handle() wraps both the registry lookup and the getter call
in one try:
try:
return _handle_getters[obj_type](obj)
except KeyError:
raise TypeError("Unknown type: " + str(obj_type)) from None
The except clause is meant for "this type has no registered getter", but it
also fires for a KeyError raised *inside* the getter. When that happens the
diagnosis is wrong twice over: the reported type is registered, and
`from None` suppresses the context so the traceback that would show the real
failure is gone.
>>> _add_cuda_native_handle_getter(Registered, getter_that_raises_keyerror)
>>> get_cuda_native_handle(Registered())
TypeError: Unknown type: <class 'Registered'>
Move the getter call out of the try. The unregistered-type path is
unchanged, which the existing test_get_handle_error still covers.
* Resolve BENCH_DIR at call time in the benchmark runner's main() (#2563)
discover_benchmarks() goes out of its way to avoid def-time binding, and
says so:
# Resolve the default inside the call so tests (and embedders) can
# monkeypatch ``BENCH_DIR`` at the module level - Python binds default
# args at def-time, so a literal default would ignore later patches.
if bench_dir is None:
bench_dir = BENCH_DIR
main() then reintroduces exactly that binding:
def main(
*,
bench_dir: Path = BENCH_DIR,
default_output: Path = DEFAULT_OUTPUT,
...
registry = discover_benchmarks(bench_dir=bench_dir, ...)
Because main() always passes a non-None bench_dir down, the sentinel branch
in discover_benchmarks() can never be taken on this path, and patching
runner.main.BENCH_DIR - the documented mechanism - has no effect on main().
Same for DEFAULT_OUTPUT.
run_pyperf.py calls main() with no arguments, so this is the production
path. The existing tests patch BENCH_DIR and call discover_benchmarks()
directly, which is why the gap is invisible today.
Apply the same sentinel to both parameters. Explicit arguments keep working
unchanged, so the embedder API is unaffected.
Adds test_main_honors_a_monkeypatched_bench_dir, which patches BENCH_DIR to a
tmp dir holding one bench_*.py and drives main() with --list. It fails before
this change (main() lists the repo's real benchmarks instead).
* cuda.core: Add copy_batch to cuda.core.utils (#2593)
* cuda.core: Add copy_batch to cuda.core.utils
* fallback for CUDA 12 and type annotations
* be more precise about CUDA requirements
* skip tests on Windows that require managed memory
* rework some tests
* Deduplicate _to_cumemlocation
* add missing file
* address review feedback
* review feedback: don't assume NUMA capabilities
* review feedback: clarify buffer requirements for async batched copies
* review feedback: explicitly reject special default streams
* review feedback: explicitly reject capturing streams
* review feedback: drop warning about unsupported PREFER_OVERLAP_WITH_COMPUTE hint
* review feedback: add missing descriptions for copy options values
* review feedback: align CopyOptions validation with existing practice
* review feedback: drop conditional imports for type checking
* account for CUDA 12/13 driver differences
* CUDA 12: drop rejection of unsupported copy options
* simplify tests
* fix(cuda.bindings): make cythonization warning-clean and enable -Werror (#2463)
* fix(cuda.bindings): make cythonization warning-clean and enable -Werror
Clear the Cython warnings that blocked matching cuda.core's warning_errors
setting (#2450): drop ignored except clauses on Python-returning cudla
cpdefs, declare LOAD_LIBRARY_SEARCH_SYSTEM32 as const in windll.pxd, and
enable Cython Options.warning_errors in build_hooks.
Add source-level regression tests so these patterns do not return.
Signed-off-by: Omar Atie <atiaomar1978-hub@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* style: ruff-format cython warning cleanliness tests
Signed-off-by: Omar Atie <atiaomar1978-hub@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(cuda.bindings): drop cython warning cleanliness tests
Address review feedback: warning_errors in build_hooks already
guards against Cython warning regressions, so the source-level
tests add unnecessary maintenance cost.
Signed-off-by: Omar Atie <atiaomar1978-hub@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Signed-off-by: Omar Atie <atiaomar1978-hub@users.noreply.github.com>
Co-authored-by: Omar Atie <atiaomar1978-hub@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(toolshed): accept `//` seals so C/C++ generated files can be sealed (#2539)
`check_generated_file_seals.py` declares three comment styles for the seal
line, one per generated-file family:
_COMMENT_CHARS = {".py": b"#", ..., ".rst": b"..", ".c": b"//",
".cpp": b"//", ".h": b"//"}
and `validate_generated_file_seal` compares the seal's captured prefix
against `expected_comment_prefix(filepath)` so a `.rst` file cannot be
sealed with a `#`, and so on. But the marker regex only ever accepts two of
the three:
rb"^(?P<prefix>#|\.\.) "
`//` can never be captured, so `fullmatch` returns None for any sealed
`.c` / `.cpp` / `.h` file and it is rejected as `MALFORMED generated-file
seal` before the prefix comparison runs at all. The `b"//"` entries in
`_COMMENT_CHARS` and the branch that would validate them are dead.
Add `//` to the alternation, with a note tying it to `_COMMENT_CHARS` so
the two do not drift again.
This also adds the first tests for the script, under `toolshed/tests/`, and
runs them alongside the existing `ci/tools/tests` in the nightly tooling
job. The parametrized case is driven from `_COMMENT_CHARS` itself, so a
future entry whose prefix the regex cannot match fails immediately instead
of silently becoming dead code.
* ci: add selective sdist build plumbing (#2465)
* cuda.core: capture complete Buffer deallocation recipe at creation (#2526)
* cuda.core: capture bound contexts for buffer deallocation streams
Record a DeallocationStream at device-pointer creation so default-stream tokens pin the allocation context (and PTDS the allocating thread) instead of relying on ambient state at free time.
* cuda.core: activate bound context during device-pointer teardown
Make the deallocation stream's context current around free/unmap/MR cleanup so destruction no longer depends on ambient CUDA context, and wire cuCtxSetCurrent into the resource-handles driver table.
* cuda.core: record from_handle deallocation streams at creation
Add keyword-only stream= on Buffer/ManagedBuffer.from_handle when mr owns the pointer, bind it at construction, and cover teardown with no or foreign current context.
* cuda.core: fail loudly on MemoryResource free errors
Stop treating CUDA_ERROR_INVALID_CONTEXT as a successful pool free, and let
explicit mr.deallocate() raise; destruction still contains errors in the
callback. Document PTDS deallocation ordering on the stream parameters and
note the context-safe Buffer teardown fix in the 1.2.0 release notes.
* cuda.core: reject incomplete buffer deallocation recipes
Require default deallocation streams to bind a current context at creation so teardown never relies on an ambiguous ambient token. Expand coverage and documentation for context-independent cleanup and failure reporting.
* cuda.core: initialize context when unpickling IPC buffers
Ensure spawned children can bind the imported buffer's default deallocation stream before their process target starts.
* test(cuda.core): set a current context in DLPack failure tests
Creating a Buffer with an owning memory resource now records a default
deallocation stream, which requires a current context. These two tests
never set one, so they passed or failed depending on whether the
preceding test left a context current under pytest-randomly.
* test(cuda.core): address review feedback on deallocation-stream PR
- Parametrize test_from_handle_mr_records_default_stream,
test_from_handle_mr_records_explicit_stream, and
test_from_handle_stream_requires_mr with [Buffer, ManagedBuffer] to
cover the ManagedBuffer.from_handle entry point directly.
- Add test_close_with_default_stream_requires_context covering the
_require_deallocation_stream_context guard in Buffer_close.
- Lift Stream_accept and default_stream to module-level imports.
- Replace _require_deallocation_stream_context (a pre-flight that
duplicated make_deallocation_stream's context check) with
_apply_deallocation_stream, which calls set_deallocation_stream once
and translates CUDA_ERROR_INVALID_CONTEXT into a descriptive
RuntimeError. Removes the redundant cuCtxGetCurrent call on the
default-stream success path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* cuda.core: minor refactoring to prepare for copy with options (#2618)
* cuda.core: minor refactoring to prepare for copy with options
* inline capability check helper
* ci: add selective wheel test plumbing (#2466)
* ci: add selective wheel test plumbing
* ci: update selective wheel test callers
* ci: enable nightly NumPy for metapackage tests
* ci: install exact local wheels in metapackage tests
* ci: simplify local wheel selection
* docs(cuda.core): don't document APIs accept dict for options (#2619)
* docs(cuda.core): don't document APIs accept dict for options
* test(cuda.core): use Options dataclasses instead of dicts in MR tests
* fix: remove unrelated Database classifier from cuda-bindings metadata (#2612)
Signed-off-by: andrewwhitecdw <andrewwhitecdw@users.noreply.github.com>
Co-authored-by: andrewwhitecdw <andrewwhitecdw@users.noreply.github.com>
* test(cuda.core): centralize memory resource instrumentation in test helpers (#2624)
* test(cuda.core): centralize memory resource instrumentation
Replace duplicated test resources with a configurable helper so allocation and deallocation telemetry stays consistent across buffer lifetime tests.
* style(cuda.core): format memory resource test refactor
* cuda.core: allow updating Buffer deallocation streams (#2602)
* cuda.core: allow updating Buffer deallocation streams
* docs(cuda.core): add deallocation stream transfer example
Show how to move a Buffer between streams with an event so its eventual deallocation remains correctly ordered.
* test(cuda.core): use shared memory resource instrumentation
Keep the deallocation-stream tests aligned with the centralized test helper merged in #2624.
* [no-ci] ci: restrict PR metadata check token to pull-request read access (#2081)
* chore: declare minimum scope on pr-metadata-check workflow
Signed-off-by: Arpit Jain <arpitjain099@gmail.com>
* chore: remove unused contents permission
---------
Signed-off-by: Arpit Jain <arpitjain099@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
* Remove unstable test_pynvml.py:test_device_get_total_energy_consumption (see cuda-python-private issue 509) (#2644)
* Guard unsupported NVML clock-domain queries (#2651)
Treat per-domain clock queries as independently optional. Individual domains may reject minimum, maximum, or current clock queries even on newer devices.
* fix(cuda.core): emit -numba-debug with a single dash for NVVM (#2639)
* fix(cuda.core): emit -numba-debug with a single dash for NVVM
ProgramOptions(numba_debug=True) always failed on the NVVM backend.
The option was emitted as --numba-debug, but libNVVM's parser accepts
only single-dashed options, so every such compile raised
NVVM_ERROR_INVALID_OPTION. NVRTC tolerates both spellings and was
unaffected. This was the only double-dashed option in the NVVM path,
which otherwise emits -arch=, -g and -ftz=1.
The defect hid itself: test_nvvm_program_numba_debug was gated on a
probe that asked libNVVM about the same wrong spelling, so the test
skipped everywhere and had never executed. Fixing the probe makes it
run, and it passes. Its skip reason also blamed CTK 13.2, which is not
the cause.
libNVVM from CUDA 12.x rejects both spellings, so the option remains
unavailable there; the release note says so.
Closes #2570
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(cuda.core): reject double-dashed NVVM options at the source
Add a guard at the single exit point of _prepare_nvvm_options_impl that
raises if any option is double-dashed. libNVVM accepts only single-dashed
options, and every option on this path is generated from typed fields, so
a double dash can only mean a bug in cuda.core rather than bad user input.
Raising here names the offending option instead of leaving the user with
libNVVM's opaque NVVM_ERROR_INVALID_OPTION.
The guard is a separate cpdef helper so a test can exercise it directly;
an inline check would be unreachable and therefore unverifiable. A second
test sets every NVVM-supported field of ProgramOptions and asserts no
emitted option is double-dashed, so the invariant covers options added
later.
Verified by mutation: restoring --numba-debug turns three tests red, and
the compile test then fails with this guard's error rather than libNVVM's.
The NVRTC path keeps --numba-debug, which it accepts; the guard is scoped
to the NVVM emitter.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Fix Windows Cython test build paths (#2650)
Run the Python build driver from the batch wrapper and feed Cython relative source names after switching to the tests/cython directory. This avoids duplicating the absolute checkout path under build/temp and preserves the build result.
* Clarify policy for explicitly requested upstream pushes (#2645)
* Refactor standalone Windows Nsight discovery (#2614)
* docs(pathfinder): finalize 1.6.1 release notes (#2656)
Co-authored-by: Michael Wang <isVoid@users.noreply.github.com>
* test(cuda.core): cover PDL same-stream overlap via GraphBuilder capture (#2524)
* test(cuda.core): verify PDL GraphBuilder capture and overlap
Cover GraphBuilder stream-capture for programmatic_stream_serialization:
functional launch, programmatic dependency edge mapping, and Hopper+
overlap (xfail if opportunistic), plus 1.2.0 release notes.
* docs(cuda.core): use runtime PDL attribute name in 1.2.0 notes
Refer to cudaLaunchAttributeProgrammaticStreamSerialization instead of
the driver-style CU_LAUNCH_ATTRIBUTE_* spelling.
* resolve pre-commit errors
* test(cuda.core): skip PDL overlap graph capture on NumPy < 2.2.5
The test writes host buffers from np.from_dlpack, which are read-only before NumPy 2.2.5 (GH #28632).
* test(cuda.core): use init_cuda in PDL overlap graph capture test
Align with other graph builder tests so context setup and teardown stay consistent.
* test(cuda.core): clarify PDL graph capture doc references
Drop fragile Programming Guide section numbers and note Driver vs Runtime enum name equivalence at the edge asserts.
* test(cuda.core): share PDL overlap protocol via helper runner
Extract kernels and the same-stream / graph-capture overlap check into
run_pdl_overlap_check so launcher and GraphBuilder tests stay in sync.
* test(cuda.core): rename PDL overlap tests to emphasize same-stream
Both direct and graph-capture paths are same-stream; update names and docs accordingly.
* test(cuda.core): tidy PDL overlap helper import order and docstring
* test(cuda.core): drop shared PDL overlap helper for clearer per-path tests
Keep the stream overlap case self-contained in test_launcher, and inline the
graph capture/overlap checks in test_graph_builder so each path stays simple.
* test(cuda.core): r…
|
leofang
added a commit
to leofang/cupy
that referenced
this pull request
Sep 10, 2026
CUDA gained Windows-on-Arm host support in CTK 13.4, so wire the GHA build lane to also produce cupy-cuda13x win_arm64 wheels alongside the existing linux-64 / linux-aarch64 / win-64 lanes. Follows the NVIDIA/cuda-python#2789 pattern: - New `build-win-arm64` job in ci.yml, gated on `startsWith(CUDA_BUILD_VER, '13.')` so pre-13.4 backport branches cleanly skip it. - Uses only `CUDA_BUILD_VER` (there is no CUDA 12 x arm64 pairing). - Python 3.10 is excluded on `win-arm64` -- CPython upstream has no official 3.10 Windows ARM64 build (nuget-cpython and actions/setup-python's manifest both drop it), so the matrix entry is unbuildable. - Runner: `windows-11-arm` (GitHub-hosted). - `fetch_ctk` action learns the `windows-arm64` redist subdir; supporting Python tools (`env-vars`, `prepare_wheel_build.py`, `wheel_configs.py`, `fetch_ctk_redistrib.py`) get the matching platform entries. - cuTENSOR preload and rapidsai/sccache are skipped for `win-arm64` because NVIDIA has not yet shipped WoA cuTENSOR binaries and rapidsai/sccache has no WoA build. Both are marked in-code as temporary; re-enable when the upstreams catch up. - No test/import step for `win-arm64` -- no GHA runner has an ARM64 GPU. This lane is build-only for now.
leofang
reviewed
Sep 10, 2026
Comment on lines
+52
to
+77
| # Windows ARM64 is available starting with CUDA 13.4. | ||
| if [[ "$cuda_build_ver" =~ ^([0-9]+)\.([0-9]+)(\.|$) ]]; then | ||
| cuda_build_major="${BASH_REMATCH[1]}" | ||
| cuda_build_minor="${BASH_REMATCH[2]}" | ||
| else | ||
| echo "Invalid CUDA build version: $cuda_build_ver" >&2 | ||
| exit 1 | ||
| fi | ||
| if (( cuda_build_major > 13 || (cuda_build_major == 13 && cuda_build_minor >= 4) )); then | ||
| windows_arm64_supported=true | ||
| else | ||
| windows_arm64_supported=false | ||
| fi | ||
| echo "windows_arm64_supported=$windows_arm64_supported" >> $GITHUB_OUTPUT | ||
|
|
||
| # No CUDA 13 windows-arm64 toolkit exists for a major other than the | ||
| # current one (windows-arm64 support started mid-way through the 13.x | ||
| # series), so cuda.core can only be built against a single CUDA major | ||
| # while the build major is still 13. Once the build major advances to | ||
| # 14, a CUDA 13 windows-arm64 toolkit will exist as the prior major. | ||
| if [[ "$cuda_build_major" == "13" ]]; then | ||
| windows_arm64_single_cuda_major=true | ||
| else | ||
| windows_arm64_single_cuda_major=false | ||
| fi | ||
| echo "windows_arm64_single_cuda_major=$windows_arm64_single_cuda_major" >> $GITHUB_OUTPUT |
Member
There was a problem hiding this comment.
We only use 1 CUDA major.minor to build in the main branch. Now that we bump the build floor I think this can be dropped and simplified.
Contributor
Author
There was a problem hiding this comment.
main builds cuda-core twice and then combines them.
13.x build
12.x build
merge wheel
Looking at this linux-x64 build on a PR, for example, you can see all three steps running.
This is needed so it doesn't build a 12.9 for Windows on ARM (which wouldn't compile) and merge it into the megawheel.
leofang
reviewed
Sep 10, 2026
Comment on lines
+45
to
+50
| if [[ "${HOST_PLATFORM}" == "win-arm64" ]]; then | ||
| # cibuildwheel's `auto` architecture detection resolves to AMD64 on the | ||
| # windows-11-arm hosted runner (the Actions runner process itself reports | ||
| # AMD64 via emulation), so the target arch must be forced explicitly. | ||
| echo "CIBW_ARCHS=ARM64" >> $GITHUB_ENV | ||
| fi |
Member
There was a problem hiding this comment.
I am very confused why this is needed. I would think cibuildwheels can figure it out. I don't have this in CuPy but it still works? cupy/cupy#10294
Contributor
Author
There was a problem hiding this comment.
I will investigate. This was done in response to it failing without it, so I'll try again.
Contributor
Author
There was a problem hiding this comment.
It might be something related to the outer actions runner, but without this, we get undefined symbols: https://github.com/NVIDIA/cuda-python/actions/runs/34491398338/job/102918906318
In that run, it's running on ARM hardware, but running the X86-hosted compiler (in emulation) that emits ARM code, which is also a bit strange. In any event, it's clear this snippet (or something like it) is required in our context.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This shows just the changes necessary to support CTK 13.4.1, compared against main. Should make #2788 easier to review.