From e712e632cf4e5a2f5a2f05e2a94ea333f9de53c7 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Mon, 7 Sep 2026 12:45:27 +0800 Subject: [PATCH 1/3] 0.2.4: every rule brings its own environment A project that wanted a device island wrote two things: the rule edge, and the rule's own package list repeated with exact pins. The second is a copy that goes stale silently -- the rule moves and the projects do not. Each rule now declares the payloads it drives, under the feature that selects it and the accelerator it serves, so a project writes one edge: [build-dependencies.mcpp] plugins = { version = "0.2.4", features = ["rules-cuda"], host-module = true } Two gates, and both must open before a byte is downloaded: the feature says whether the rule is wanted, the selector says whether this build compiles for the device. A CPU-only build opens neither. The shape of each default is a judgement about coupling, not a style. An exact version where the payload's version is coupled to something the rule cannot see -- a CUDA runtime must not be newer than the driver it will meet, so the 12.9 line is offered and a project with newer machines names 13.x itself. A floor (`>=`) where no such coupling exists: glslang, dpcpp, the CANN toolkit. mcpp reads the difference: a bare version is a choice and a project pinning another wins; a `>=` is a requirement and a pin below it is refused naming both sides. Not moved: anything the produced PROGRAM chooses to run on. A Vulkan ICD is a device, and a rule declaring one would force a software renderer onto consumers that have a GPU. The runtime adapters stay in the project for that reason and because this package is a [build-dependencies] edge. Two fixtures declared fewer accelerators than their globs name: on the NVIDIA platform HIP and SYCL both compile THROUGH the CUDA back end, so `cuda` belongs in their `[package] accelerators`. mcpp 2026.9.6.5 refuses that mismatch; the CI pin was older, so only raising it surfaced them. The floor for every rule is now 2026.9.6.6, the release in which a payload a dependency declared is both installed and answerable. --- .github/workflows/ci.yml | 55 ++++++++++++++- README.md | 78 +++++++++++++++++---- mcpp.toml | 102 +++++++++++++++++++++++++++- rules/ascendc.cppm | 10 ++- rules/cuda.cppm | 18 ++++- rules/hip.cppm | 9 ++- rules/spirv.cppm | 10 ++- rules/sycl.cppm | 9 ++- tests/cuda-consumer/mcpp.toml | 27 +++----- tests/hip-consumer/mcpp.toml | 31 +++++---- tests/multi-rule-consumer/mcpp.toml | 23 +++---- tests/spirv-consumer/mcpp.toml | 6 +- tests/sycl-consumer/mcpp.toml | 38 ++++------- 13 files changed, 320 insertions(+), 96 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 8740b75..27efad5 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -9,7 +9,7 @@ on: env: # The mcpp release the consumers build with. Raising it is what admits a # member that relies on a newer engine; the README states each member's floor. - MCPP_VERSION: 2026.9.6.1 + MCPP_VERSION: 2026.9.6.6 jobs: consumers: @@ -147,6 +147,59 @@ jobs: grep -q '^magic=(no shader in this build)' run.log grep -q '^12 24 36 48' run.log + # THE PAYLOADS CAME FROM THE RULE, NOT FROM THE FIXTURE. + # + # Every fixture above builds, and that alone does not distinguish "the + # rule declared its toolkit" from "the fixture still declares it": a + # re-added `[xlings.workspace]` block would keep all of them green while + # the claim this package makes stopped being true. + # + # The object is mcpp's own report of WHERE the entries came from. It + # names the pass -- the graph's, not the root's -- so it separates the + # two cases directly rather than by reading the manifests back. + - name: the rules declare their own payloads + run: | + set -e + # THE PROVISIONING LINE ONLY PRINTS ON A STAMP MISS, and this job + # restores ~/.mcpp from a cache. Without clearing the record, the + # criterion's "no" and "this ran on a warm cache" are the same + # reading. Removing the stamp costs nothing: the packages are still + # installed, so the re-provision is one xlings round trip. + rm -rf ~/.mcpp/provisioned + for d in tests/cuda-consumer tests/hip-consumer \ + tests/sycl-consumer tests/spirv-consumer; do + # No project-side declaration is the first half of the claim, and + # it is a property of a file this repository owns. + if grep -qE '^\[(target\..*\.)?xlings\.workspace\]' "$d/mcpp.toml"; then + echo "FAIL: $d declares payloads itself, so this step proves nothing" + exit 1 + fi + ( cd "$d" && rm -rf target && "$MCPP" build > prov.log 2>&1 ) || { + echo "build failed in $d"; tail -20 "$d/prov.log"; exit 1; } + grep -q 'entries declared by dependencies' "$d/prov.log" || { + echo "FAIL: $d installed no payload the rule declared" + grep -i 'provisioning' "$d/prov.log" || echo "(no provisioning line at all)" + exit 1 + } + echo "ok: $d — $(grep -m1 'entries declared by dependencies' "$d/prov.log")" + rm -f "$d/prov.log" + done + # And the one fixture that DOES override, so the override path is on + # this graph too: it declares one entry, and one version is installed. + d=tests/multi-rule-consumer + grep -q 'xlings.workspace' "$d/mcpp.toml" || { + echo "FAIL: $d no longer demonstrates the override"; exit 1; } + ( cd "$d" && rm -rf target && "$MCPP" build > prov.log 2>&1 ) || { + echo "build failed in $d"; tail -20 "$d/prov.log"; exit 1; } + n=$(ls -1 ~/.mcpp/registry/data/xpkgs/xim-x-glslang | wc -l | tr -d ' ') + [ "$n" = "1" ] || { + echo "FAIL: the override installed $n versions of glslang:" + ls -1 ~/.mcpp/registry/data/xpkgs/xim-x-glslang + exit 1 + } + echo "ok: $d — one glslang, $(ls -1 ~/.mcpp/registry/data/xpkgs/xim-x-glslang)" + rm -f "$d/prov.log" + # THE HOST-LEAK ASSERTION, and what it measures was corrected once. # # Its first form grepped a verbose build for `/usr`. That reported diff --git a/README.md b/README.md index 20671fc..06c56af 100644 --- a/README.md +++ b/README.md @@ -46,13 +46,59 @@ engine's own module family and is not used here. | feature | module | since mcpp | what it needs | |---|---|---|---| -| `rules-ascendc` | `mcpp.rules.ascendc` | 2026.9.6.5 | `xim:cann-toolkit` in `[xlings.workspace]`, `[build] accel = "ascend8.5+{dav-c220}"`, a constrained glob for `*.asc`. Compiles with BiSheng in MIXED mode, so the object carries the device binary and a host-callable launcher and joins the ordinary link -- no registration file and no device-link step. The floor is the release whose device-source table carries `.asc` and whose `mcpp::link_flag` can emit the `-rpath-link` the toolkit's own shared libraries need | -| `rules-cuda` | `mcpp.rules.cuda` | 2026.9.5.2 | the toolkit named in `[xlings.workspace]` (`xim:cuda-nvcc`, `xim:cuda-cudart`, and `xim:libcurand` for the clang route, whose wrapper includes a cuRAND header unconditionally), `[build] accel = "cuda…"`, a constrained glob for `*.cu`; the clang route with an LLVM toolchain, the nvcc route with a GCC one | -| `rules-hip` | `mcpp.rules.hip` | 2026.9.5.2 | `xim:hip-nvidia` plus the CUDA back end it compiles through (`xim:cuda-nvcc`, `xim:cuda-cudart`, `xim:libcurand`, `xim:cuda-cccl`), `[build] accel = "hip, cuda12.9+{sm_89}"`, a constrained glob for `*.hip`. On the NVIDIA platform HIP is a header layer over the CUDA runtime, so the compiler is the project's own clang and there is no ROCm on the machine | -| `rules-spirv` | `mcpp.rules.spirv` | 2026.9.5.3 | `xim:glslang` or `xim:shaderc` in `[xlings.workspace]`, `[build] accel = "vulkan1.2"`, a constrained glob for the shader stages; emits one header per shader through a `role = "source"` action, and states which of the two compilers produced it | -| `rules-sycl` | `mcpp.rules.sycl` | 2026.9.6.1 | `xim:dpcpp` (the compiler), `xim:gcc` (the C++ standard library the unit compiles against, not a second toolchain) and `xim:cuda-nvcc` for an NVIDIA target; `[build] accel = "sycl"` or `"sycl, cuda12.9+{sm_89}"`, a constrained glob for `*.sycl`, and `compat:sycl-runtime` so the artifact can reach `libsycl.so.9` at run time. The floor is the release whose device-source table carries `.sycl` | +| `rules-ascendc` | `mcpp.rules.ascendc` | 2026.9.6.6 | `[build] accel = "ascend8.5+{dav-c220}"`, a constrained glob for `*.asc`. Compiles with BiSheng in MIXED mode, so the object carries the device binary and a host-callable launcher and joins the ordinary link -- no registration file and no device-link step. Its own engine needs are `.asc` in the device-source table and `mcpp::link_flag` for the `-rpath-link` the toolkit's shared libraries require, both 2026.9.6.5 | +| `rules-cuda` | `mcpp.rules.cuda` | 2026.9.6.6 | `[build] accel = "cuda…"`, a constrained glob for `*.cu`; the clang route with an LLVM toolchain, the nvcc route with a GCC one | +| `rules-hip` | `mcpp.rules.hip` | 2026.9.6.6 | `[build] accel = "hip, cuda12.9+{sm_89}"`, a constrained glob for `*.hip`. On the NVIDIA platform HIP is a header layer over the CUDA runtime, so the compiler is the project's own clang and there is no ROCm on the machine | +| `rules-spirv` | `mcpp.rules.spirv` | 2026.9.6.6 | `[build] accel = "vulkan1.2"`, a constrained glob for the shader stages; emits one header per shader through a `role = "source"` action, and states which of the two compilers produced it | +| `rules-sycl` | `mcpp.rules.sycl` | 2026.9.6.6 | `[build] accel = "sycl"` or `"sycl, cuda12.9+{sm_89}"`, a constrained glob for `*.sycl`, and `compat:sycl-runtime` so the artifact can reach `libsycl.so.9` at run time. Its own engine need is `.sycl` in the device-source table, 2026.9.6.1 | | `tools-embed` | `mcpp.tools.embed` | 2026.9.5.4 | nothing beyond mcpp: it reads a file and writes a header while the build program runs. The floor is the release whose fast path compares a declared file input, without which an edit to the data does not reach the binary | +### Each rule brings its own environment + +A project names the rule and nothing else: + +```toml +[build-dependencies.mcpp] +plugins = { version = "0.2.4", features = ["rules-cuda"], host-module = true } +``` + +The payloads each rule drives are declared **here**, under the feature that +selects the rule and the accelerator it serves: + +```toml +[target.'cfg(accelerator = "cuda")'.feature-xlings.rules-cuda] +"xim:cuda-nvcc" = "12.9.86" +"xim:cuda-cudart" = "12.9.79" +``` + +Two gates, and both must open before a byte is downloaded. The feature says +whether the rule is wanted; the selector says whether this build compiles for +the device. A CPU-only build opens neither. + +**The shape of each default is a judgement about coupling.** An exact version +where the payload is coupled to something the rule cannot see -- a CUDA runtime +must not be newer than the driver it will meet, so the 12.9 line is offered and +a project with newer machines names 13.x itself. A floor (`>=`) where no such +coupling exists: a shader compiler, a SYCL compiler, a CANN toolkit. + +mcpp reads the difference. A bare version is a **choice**, so a project pinning +a different one wins and the override is reported; a `>=` is a **requirement**, +so a project pinning below it is refused naming both sides. Either way one +version is installed. To override: + +```toml +[target.'cfg(accelerator = "cuda")'.xlings.workspace] +"xim:cuda-nvcc" = "13.3.33" +``` + +**What is not here:** anything the produced program chooses to run *on*. A +Vulkan ICD (`xim:mesa-lavapipe`) is a device, and a rule that declared one would +force a software renderer onto consumers that have a GPU. The runtime adapters +(`compat:cuda-runtime`, `compat:sycl-runtime`, `compat:vulkan-runtime`) stay in +the project for that reason and for a second one: this package is reached +through a `[build-dependencies]` edge, so its own `[dependencies]` deliberately +do not reach the consumer's target. + ### Each rule takes the extensions it claims `mcpp::device_sources()` is the package's WHOLE device set, not one rule's @@ -77,14 +123,20 @@ A device source that NO rule claims is not silently dropped: mcpp refuses a device source that reached no action, naming the file. That is the engine's half of this rule and it needs 2026.9.6.5. -The floor is the mcpp release whose engine carries what the member relies on: -`rules-spirv` needs the device-source table that classifies shader extensions, -which 2026.9.5.3 introduced; `tools-embed` needs the fast path to compare a -declared file input, which 2026.9.5.4 introduced; and `rules-sycl` needs `.sycl` -in that same device-source table, which 2026.9.6.1 introduced. The index -descriptor states the highest floor among the members, so it is the floor of the -collection rather than of any one feature; a project on an older mcpp is refused -at resolution rather than at the first shader. +The floor is the mcpp release whose engine carries what the member relies on. +From 0.2.4 every rule shares one: **2026.9.6.6**, the release in which a payload +a DEPENDENCY declared is both installed and answerable. Before it a rule could +declare `>=8.5.0`, have it installed, and still be told by `xpkg_dir` that +nothing was there -- which is why each rule's list used to be repeated in every +project that used it. The earlier per-member floors are still the floors of the +rules themselves (`rules-spirv` needs the shader extensions in the device-source +table, 2026.9.5.3; `tools-embed` needs the fast path to compare a declared file +input, 2026.9.5.4; `rules-sycl` needs `.sycl` in that table, 2026.9.6.1), and +they are all below the shared one. + +The index descriptor states the highest floor among the members, so it is the +floor of the collection rather than of any one feature; a project on an older +mcpp is refused at resolution rather than at the first shader. ## How the engine sees this package diff --git a/mcpp.toml b/mcpp.toml index 3970a18..5ad0d38 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,7 +1,7 @@ [package] name = "plugins" namespace = "mcpp" -version = "0.2.3" +version = "0.2.4" description = "Official mcpp build plugins: rule packages under mcpp.rules.*, build-time utilities under mcpp.tools.*, each member selected by a feature" license = "Apache-2.0" authors = ["mcpp-community"] @@ -28,5 +28,105 @@ rules-spirv = { sources = ["rules/spirv.cppm"] } rules-sycl = { sources = ["rules/sycl.cppm"] } tools-embed = { sources = ["tools/embed.cppm"] } +# ── The environment each rule needs (mcpp 2026.9.6.6+) ────────────────────── +# +# A RULE OWNS THE LIST OF PACKAGES IT DRIVES. It is the code that runs the +# compiler, reads the headers and puts the library directory on the link line, +# so "which package, and which version of it" is its knowledge. Before this, a +# project repeated the list in its own `[xlings.workspace]` -- a copy that goes +# stale silently, because the rule moves and the projects do not. +# +# TWO GATES, AND BOTH MUST OPEN BEFORE A BYTE IS DOWNLOADED. The feature says +# whether this rule is wanted at all; `cfg(accelerator = ...)` says whether this +# particular build compiles for the device. A CPU-only build of a project that +# depends on this package with `features = ["rules-cuda"]` opens the first and +# not the second, and installs nothing -- which is the build CI runs, and the +# one a multi-gigabyte vendor toolkit would otherwise make the most expensive. +# +# THE SHAPE OF EACH DEFAULT IS A JUDGEMENT ABOUT COUPLING, not a style. +# +# - An EXACT version where the payload's version is coupled to something the +# rule cannot see. A CUDA runtime must not be newer than the driver it will +# meet: the 12.9 line reaches every driver from r525 onward, and the 13.x +# line raises that to r580. That is a property of the consumer's machines, +# so the rule offers a line it has verified and the project moves it. +# - A FLOOR (`>=`) where no such coupling exists and newer is simply newer. +# +# mcpp reads the difference: a bare version is a CHOICE, so a project pinning a +# different one wins and is reported; a `>=` is a REQUIREMENT, so a project +# pinning below it is refused naming both sides. See "One package, one version" +# in mcpp's docs/05. +# +# WHAT IS NOT HERE: anything the produced PROGRAM chooses to run on. A Vulkan +# ICD (`xim:mesa-lavapipe`) is a device, not a build tool, and a rule that +# declared one would force a software renderer onto consumers that have a GPU. +# The rule declares what it needs to COMPILE; the project declares what it needs +# to RUN. The runtime adapters (`compat:cuda-runtime`, `compat:sycl-runtime`, +# `compat:vulkan-runtime`) stay in the project for the same reason and for a +# second one: this package is a `[build-dependencies]` edge, so its own +# `[dependencies]` deliberately do not reach the consumer's target. + +[target.'cfg(accelerator = "cuda")'.feature-xlings.rules-cuda] +"xim:cuda-nvcc" = "12.9.86" +"xim:cuda-cudart" = "12.9.79" +# clang's CUDA wrapper includes a cuRAND header for every device unit, and that +# header includes from CCCL. Neither is called by an ordinary +# kernel; on a developer machine the host's /usr/include supplied them silently. +"xim:libcurand" = "10.3.10.19" +"xim:cuda-cccl" = "12.9.27" + +# The driver sentinel is Linux-only, and the OS goes in the PREDICATE rather +# than in the value: a value carrying platform keys under an already-predicated +# table would state the condition twice and let the two disagree. +[target.'cfg(all(accelerator = "cuda", linux))'.feature-xlings.rules-cuda] +"xim:libcuda-host-link" = "0.0.1" + +# HIP on the NVIDIA platform is a header layer over the CUDA runtime, so this +# list is the CUDA one plus the header package HIP includes at its second line. +[target.'cfg(accelerator = "hip")'.feature-xlings.rules-hip] +"xim:hip-nvidia" = "7.2.4" +"xim:cuda-nvcc" = "12.9.86" +"xim:cuda-cudart" = "12.9.79" +"xim:libcurand" = "10.3.10.19" +"xim:cuda-cccl" = "12.9.27" +"xim:cuda-profiler-api" = "12.9.79" + +[target.'cfg(all(accelerator = "hip", linux))'.feature-xlings.rules-hip] +"xim:libcuda-host-link" = "0.0.1" + +# A floor: the compiler's version is not coupled to any driver, and the rule +# needs `-fsycl-link` to behave as it does from 7.1.0 onward. +[target.'cfg(accelerator = "sycl")'.feature-xlings.rules-sycl] +"xim:dpcpp" = ">=7.1.0" + +# THE SECOND COMPILER MUST BE TOLD WHERE THE C LIBRARY IS. dpcpp's clang is not +# configured with this ecosystem's glibc, so without these its device units +# compile against the host's -- which no command line shows, because an implicit +# include search never appears on one. +[target.'cfg(all(accelerator = "sycl", linux))'.feature-xlings.rules-sycl] +"xim:gcc" = "15.1.0" +"xim:glibc" = "" +"xim:linux-headers" = "" + +# Only when the SYCL build actually names an NVIDIA target. `accelerator` is a +# SET, so this is a conjunction of two membership tests rather than a +# contradiction -- and without it every SYCL build would download a CUDA +# toolkit it may have no target for. +[target.'cfg(all(accelerator = "sycl", accelerator = "cuda"))'.feature-xlings.rules-sycl] +"xim:cuda-nvcc" = "12.9.86" + +# glslang alone: it is the rule's first choice, and `xim:shaderc` is the +# fallback a project names when it wants glslc instead. Declaring both would +# install both and use one. +[target.'cfg(accelerator = "vulkan")'.feature-xlings.rules-spirv] +"xim:glslang" = ">=15.1.0" + +# 8.5.0 is a real floor rather than a preference: the mixed-mode object -- the +# one carrying both the device binary and a host-callable launcher, which is +# what lets it join an ordinary link -- and the 38 SoC simulators are what this +# rule drives, and they are present from that version onward. +[target.'cfg(accelerator = "ascend")'.feature-xlings.rules-ascendc] +"xim:cann-toolkit" = ">=8.5.0" + [targets.plugins] kind = "lib" diff --git a/rules/ascendc.cppm b/rules/ascendc.cppm index afe0c38..43bef15 100644 --- a/rules/ascendc.cppm +++ b/rules/ascendc.cppm @@ -210,8 +210,14 @@ inline std::optional find_toolkit() { const auto pkg = xpkg("cann-toolkit"); if (pkg.empty()) { std::println(std::cerr, - "mcpp.rules.ascendc: the CANN toolkit is not declared.\n" - " Name it under [xlings.workspace] and mcpp provisions it on first use:\n" + "mcpp.rules.ascendc: the CANN toolkit is not installed.\n" + " This rule DECLARES it, so a project normally writes nothing. Check, in " + "order:\n" + " mcpp older than 2026.9.6.6; `features = [\"rules-ascendc\"]` missing from " + "the\n" + " [build-dependencies] edge; or a build that names no Ascend accelerator.\n" + " To pin a different version, name it in your own project and it wins:\n" + " [target.'cfg(accelerator = \"ascend\")'.xlings.workspace]\n" " \"xim:cann-toolkit\" = \"8.5.0\"\n" " It carries both halves this rule needs: the device compiler and,\n" " for a machine with no NPU, the per-SoC simulators."); diff --git a/rules/cuda.cppm b/rules/cuda.cppm index 4bbacf1..4fc7318 100644 --- a/rules/cuda.cppm +++ b/rules/cuda.cppm @@ -226,8 +226,18 @@ inline std::optional find_toolkit() { t.driver_dir = xpkg("libcuda-host-link"); if (t.nvcc_root.empty() || t.cudart_root.empty()) { std::println(std::cerr, - "mcpp.rules.cuda: the toolkit is not declared.\n" - " Name it under [xlings.workspace] and mcpp provisions it on first use:\n" + "mcpp.rules.cuda: the toolkit is not installed.\n" + " This rule DECLARES it, so a project normally writes nothing. Three " + "things stop\n" + " that from reaching the build, in the order worth checking:\n" + " - mcpp older than 2026.9.6.6, which cannot answer a payload a " + "dependency declared;\n" + " - `features = [\"rules-cuda\"]` missing from the " + "[build-dependencies] edge;\n" + " - the build names no CUDA accelerator (`--accel \"cuda12.9+{{sm_89}}\"` " + "or [build] accel).\n" + " To pin a different line, name it in your own project and it wins:\n" + " [target.'cfg(accelerator = \"cuda\")'.xlings.workspace]\n" " \"xim:cuda-nvcc\" = \"12.9.86\"\n" " \"xim:cuda-cudart\" = \"12.9.79\"\n" " (found nvcc: '{}', cudart: '{}')", t.nvcc_root, t.cudart_root); @@ -488,7 +498,9 @@ inline std::vector plan(std::span sources, options opt std::println(std::cerr, "mcpp.rules.cuda: the clang route needs cuRAND's headers, which clang's CUDA " "wrapper includes unconditionally, and CCCL's, which they include in turn.\n" - " Name the payloads under [xlings.workspace] and mcpp provisions them on first use:\n" + " This rule declares both; see the note above for why they may not have " + "arrived.\n" + " To pin a different line, name it in your own project and it wins:\n" " \"xim:cuda-cccl\" = \"12.9.27\" (the 12.9 line; 13.x pairs with 13.x)\n" " \"xim:libcurand\" = \"10.3.10.19\" (the 12.9 line; 10.4.x pairs with 13.x)\n" " (found cccl: '{}', curand: '{}')", tk->cccl_root, tk->curand_root); diff --git a/rules/hip.cppm b/rules/hip.cppm index bdde306..f4388ac 100644 --- a/rules/hip.cppm +++ b/rules/hip.cppm @@ -290,8 +290,13 @@ inline std::vector plan(std::span sources, options opt missing += std::format(" \"xim:{}\" = \"{}\"\n", n.pkg, n.version); if (!missing.empty()) { std::println(std::cerr, - "mcpp.rules.hip: the HIP island needs payloads this project has not declared.\n" - " Add to mcpp.toml:\n\n [xlings.workspace]\n{}\n" + "mcpp.rules.hip: the HIP island needs payloads that are not installed.\n" + " This rule DECLARES them, so a project normally writes nothing. Check, in " + "order:\n" + " mcpp older than 2026.9.6.6; `features = [\"rules-hip\"]` missing from the\n" + " [build-dependencies] edge; or a build that names no HIP accelerator.\n" + " To pin different versions, name them in your own project and they win:\n\n" + " [target.'cfg(accelerator = \"hip\")'.xlings.workspace]\n{}\n" " They are PAYLOADS: the version is the project's choice, not the machine's.", missing); return out; diff --git a/rules/spirv.cppm b/rules/spirv.cppm index cd905de..78b3b3b 100644 --- a/rules/spirv.cppm +++ b/rules/spirv.cppm @@ -401,8 +401,14 @@ inline bool compile(std::span shaders, options opt = {}) { if (!cc) { if (cc.reported) return false; std::println(stderr, - "mcpp.rules.spirv: no shader compiler found. Install one into the workspace\n" - " [xlings.workspace]\n" + "mcpp.rules.spirv: no shader compiler found.\n" + " This rule DECLARES glslang, so a project normally writes nothing. Check, in " + "order:\n" + " mcpp older than 2026.9.6.6; `features = [\"rules-spirv\"]` missing from the\n" + " [build-dependencies] edge; or a build that names no Vulkan accelerator.\n" + " To use glslc instead, or to pin a different version, name it in your own\n" + " project and it wins:\n" + " [target.'cfg(accelerator = \"vulkan\")'.xlings.workspace]\n" " \"xim:glslang\" = \"15.1.0\" # glslangValidator\n" " \"xim:shaderc\" = \"2026.3\" # glslc\n" "or name it: MCPP_GLSLANG=/path/to/glslangValidator, MCPP_GLSLC=/path/to/glslc,\n" diff --git a/rules/sycl.cppm b/rules/sycl.cppm index 8c598ca..422a32a 100644 --- a/rules/sycl.cppm +++ b/rules/sycl.cppm @@ -298,8 +298,13 @@ inline std::vector plan(std::span sources, options opt missing += " \"xim:cuda-nvcc\" = \"12.9.86\"\n"; if (!missing.empty()) { std::println(std::cerr, - "mcpp.rules.sycl: the SYCL island needs payloads this project has not declared.\n" - " Add to mcpp.toml:\n\n [xlings.workspace]\n{}\n" + "mcpp.rules.sycl: the SYCL island needs payloads that are not installed.\n" + " This rule DECLARES them, so a project normally writes nothing. Check, in " + "order:\n" + " mcpp older than 2026.9.6.6; `features = [\"rules-sycl\"]` missing from the\n" + " [build-dependencies] edge; or a build that names no SYCL accelerator.\n" + " To pin different versions, name them in your own project and they win:\n\n" + " [target.'cfg(accelerator = \"sycl\")'.xlings.workspace]\n{}\n" " `xim:gcc` is not a second toolchain: it is the C++ standard library the SYCL\n" " unit compiles against, and `xim:glibc` with `xim:linux-headers` is the C\n" " library underneath it. Without them dpcpp's clang reads the HOST's headers,\n" diff --git a/tests/cuda-consumer/mcpp.toml b/tests/cuda-consumer/mcpp.toml index 4e75d14..6da6026 100644 --- a/tests/cuda-consumer/mcpp.toml +++ b/tests/cuda-consumer/mcpp.toml @@ -33,23 +33,16 @@ plugins = { path = "../..", features = ["rules-cuda"], host-module = true } [dependencies.compat] cuda-runtime = "2026.09.05" -# The toolkit this project builds with, named rather than discovered. These -# are PAYLOADS, so the version is the project's choice and not the machine's. -# The 12.9 line is named on purpose: a runtime must not be newer than the -# driver it will meet, and 12.x reaches every driver from r525 onward. The -# rule package states the driver relation and mcpp compares it before the -# first compile. -[xlings.workspace] -"xim:cuda-nvcc" = "12.9.86" -"xim:cuda-cudart" = "12.9.79" -# cuRAND's headers: clang's CUDA wrapper includes curand_mtgp32_kernel.h for -# every device unit, so the clang route needs them even when nothing calls -# cuRAND. The 10.3.x line pairs with CUDA 12.x. -"xim:libcurand" = "10.3.10.19" -# CCCL (libcu++, cub, thrust): cuRAND's header includes from it. -# The 12.x toolkits ship it as a separate package; 12.9.27 is the 12.9 line. -"xim:cuda-cccl" = "12.9.27" -"xim:libcuda-host-link" = { linux = "0.0.1" } +# NO [xlings.workspace]. The toolkit -- nvcc, cudart, cuRAND, CCCL and the +# driver sentinel -- is declared by `mcpp.rules.cuda` itself, under +# `cfg(accelerator = "cuda")` and the `rules-cuda` feature. The 12.9 line is +# the rule's default and it is an EXACT version rather than a floor, because a +# runtime must not be newer than the driver it will meet: 12.x reaches every +# driver from r525 onward and 13.x raises that to r580. A project whose +# machines are newer names 13.x in its own [xlings.workspace] and wins. +# +# Needs mcpp 2026.9.6.6; before it, a payload a dependency declared was +# installed and then answered as absent. [build] # What this build compiles device code FOR. Written once, here: the rule diff --git a/tests/hip-consumer/mcpp.toml b/tests/hip-consumer/mcpp.toml index 7cc3939..d6c8efc 100644 --- a/tests/hip-consumer/mcpp.toml +++ b/tests/hip-consumer/mcpp.toml @@ -3,7 +3,12 @@ name = "hip-consumer" namespace = "example" version = "0.1.0" description = "A HIP kernel behind a seam module, reaching an NVIDIA device, with a CPU fallback" -accelerators = ["hip"] +# BOTH, because the constrained glob below names both. On the NVIDIA platform +# HIP compiles THROUGH the CUDA back end, so `cuda` is a backend this package's +# sources are written for even though nothing here says the word. mcpp refuses a +# glob naming a backend its package does not declare (2026.9.6.5+), which is +# what caught this list being short by one. +accelerators = ["hip", "cuda"] [language] standard = "c++23" @@ -31,20 +36,16 @@ plugins = { path = "../..", features = ["rules-hip"], host-module = true } [dependencies.compat] cuda-runtime = "2026.09.05" -# The payloads. `hip-nvidia` is headers only -- on this platform that is all -# HIP is -- and the four CUDA entries are the back end it compiles through. -# cuRAND and CCCL are on the list for the reason the CUDA consumer records: -# clang's CUDA wrapper includes curand_mtgp32_kernel.h for every device unit -# and that header includes , so a unit that calls neither still -# needs both. -[xlings.workspace] -"xim:hip-nvidia" = "7.2.4" -"xim:cuda-nvcc" = "12.9.86" -"xim:cuda-cudart" = "12.9.79" -"xim:libcurand" = "10.3.10.19" -"xim:cuda-cccl" = "12.9.27" -"xim:cuda-profiler-api" = "12.9.79" -"xim:libcuda-host-link" = { linux = "0.0.1" } +# NO [xlings.workspace]. The rule declares the payloads it drives, under the +# feature that selects it and the accelerator it serves, so this project names +# the rule and nothing else. Needs mcpp 2026.9.6.6; before it, a payload a +# dependency declared was installed and then answered as absent. +# +# The list `mcpp.rules.hip` carries is the CUDA one plus `cuda-profiler-api`: +# `nvidia_hip_runtime_api.h` includes at its second line, +# and a machine with a host CUDA installation finds that header in /usr/include +# without saying so -- which is how the entry came to be missing from a build +# that worked. [build] # Two chunks: the programming model, and the device. A device is spelled once diff --git a/tests/multi-rule-consumer/mcpp.toml b/tests/multi-rule-consumer/mcpp.toml index d14328e..8d3e92b 100644 --- a/tests/multi-rule-consumer/mcpp.toml +++ b/tests/multi-rule-consumer/mcpp.toml @@ -43,20 +43,17 @@ sources = [ ] include_dirs = ["include"] -# The payloads, pinned unconditionally. +# THE ONE FIXTURE THAT ALSO OVERRIDES, so the override path is on the CI graph +# rather than only in mcpp's own e2e. Both rules declare their payloads; this +# project additionally names one of them. # -# Gating them on the device each is for -- `[target.'cfg(accelerator = -# "cuda")'.xlings.workspace]` -- is what a project should do, and it needs mcpp -# 2026.9.6.5. This fixture is pinned to the floor its RULES need, not to the -# newest engine, so it writes the older spelling; mcpp's own -# examples/09-heterogeneous/multi-backend shows the gated one. -[xlings.workspace] -"xim:cuda-nvcc" = "12.9.86" -"xim:cuda-cudart" = "12.9.79" -"xim:libcurand" = "10.3.10.19" -"xim:cuda-cccl" = "12.9.27" -"xim:libcuda-host-link" = { linux = "0.0.1" } -"xim:glslang" = "15.1.0" +# `mcpp.rules.spirv` states `"xim:glslang" = ">=15.1.0"` -- a REQUIREMENT, +# because nothing about a shader compiler couples to this machine. The pin +# below satisfies it, so one version is installed and the build proceeds. A pin +# BELOW the floor would be refused naming both sides, which is the property +# that makes an override safe to offer at all. +[target.'cfg(accelerator = "vulkan")'.xlings.workspace] +"xim:glslang" = "15.1.0" [dependencies.compat] cuda-runtime = "2026.09.05" diff --git a/tests/spirv-consumer/mcpp.toml b/tests/spirv-consumer/mcpp.toml index 476d0af..3c68990 100644 --- a/tests/spirv-consumer/mcpp.toml +++ b/tests/spirv-consumer/mcpp.toml @@ -25,8 +25,10 @@ import_std = true [build-dependencies.mcpp] plugins = { path = "../..", features = ["rules-spirv"], host-module = true } -[xlings.workspace] -"xim:glslang" = "15.1.0" +# NO [xlings.workspace]. The rule declares the payloads it drives, under the +# feature that selects it and the accelerator it serves, so this project names +# the rule and nothing else. Needs mcpp 2026.9.6.6; before it, a payload a +# dependency declared was installed and then answered as absent. [build] accel = "vulkan1.2" diff --git a/tests/sycl-consumer/mcpp.toml b/tests/sycl-consumer/mcpp.toml index 6417fdf..2db4702 100644 --- a/tests/sycl-consumer/mcpp.toml +++ b/tests/sycl-consumer/mcpp.toml @@ -3,7 +3,10 @@ name = "sycl-consumer" namespace = "example" version = "0.1.0" description = "A SYCL kernel behind a seam module, reaching an NVIDIA device, with a CPU fallback" -accelerators = ["sycl"] +# BOTH, for the reason the HIP consumer records: the glob below names the +# NVIDIA target the SYCL unit is compiled for, so `cuda` is a backend this +# package's sources are written for. +accelerators = ["sycl", "cuda"] [language] standard = "c++23" @@ -34,29 +37,18 @@ plugins = { path = "../..", features = ["rules-sycl"], host-module = true } [dependencies.compat] sycl-runtime = "2026.09.07" -# Three payloads, each closing one hole the host would otherwise fill. +# NO [xlings.workspace]. The rule declares the payloads it drives, under the +# feature that selects it and the accelerator it serves, so this project names +# the rule and nothing else. Needs mcpp 2026.9.6.6; before it, a payload a +# dependency declared was installed and then answered as absent. # -# dpcpp the compiler. -# gcc NOT a second toolchain: the C++ standard library the SYCL unit -# compiles against. Left alone, dpcpp's clang reads the HOST's -# /usr/include/c++ -- measured, and invisible until a machine -# without it refuses the build. -# cuda-nvcc the NVIDIA back end's libdevice. Also measured: without -# `--cuda-path` clang finds the host's CUDA installation and says -# nothing about it. -[xlings.workspace] -"xim:dpcpp" = "7.1.0" -"xim:gcc" = "15.1.0" -# The C library the SYCL unit compiles against, and the kernel headers under -# it. Named for the same reason `xim:gcc` is: dpcpp's clang is not the clang -# mcpp resolved, and it is configured with neither. Left alone its search list -# reaches `/usr/include` with no ecosystem glibc anywhere on it -- measured. -# UNPINNED: the C library version is the runtime binding's choice, not this -# project's. Pinning it refuses on a machine whose binding chose another -- -# 2.44 here, 2.44.2 on a CI runner. `""` means "present, any version". -"xim:glibc" = "" -"xim:linux-headers" = "" -"xim:cuda-nvcc" = "12.9.86" +# `mcpp.rules.sycl` carries five, each closing one hole the host would +# otherwise fill: dpcpp (the compiler); gcc, glibc and linux-headers (the C++ +# and C libraries the SYCL unit compiles against -- dpcpp's clang is not the +# clang mcpp resolved and is configured with neither, so left alone its search +# list reaches /usr/include with no ecosystem glibc on it); and cuda-nvcc, +# whose libdevice the NVIDIA back end needs -- declared only when the accel +# actually names cuda, so a SYCL build for another target does not pay for it. [build] # Two chunks: the programming model, and the device. Written `sycl` alone, the From 1862ba9e10375e8bfc3ce2685937e672ebcf7dd5 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Mon, 7 Sep 2026 15:20:56 +0800 Subject: [PATCH 2/3] =?UTF-8?q?ci:=20=E5=88=A4=E6=8D=AE=E9=97=AE=20mcpp=20?= =?UTF-8?q?=E5=AE=83=E7=9A=84=20home=20=E5=9C=A8=E5=93=AA,=E8=80=8C?= =?UTF-8?q?=E4=B8=8D=E6=98=AF=E5=81=87=E5=AE=9A=20~/.mcpp?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 新加的那步在一次**构建成功**的运行上红了:`rm -rf ~/.mcpp/provisioned` 什么都 没删,于是前面几步写下的 stamp 还在,供给那一行不再打印。 已发布的 mcpp 是自包含的:没有 `MCPP_HOME` 时它的 home 就是解开的那个 tarball 目录。所以 `~/.mcpp` 从来不是它的 home——而这个 job 的 cache 正是 `path: ~/.mcpp`, 它保存和恢复的是一个 mcpp 从不碰的目录。**每一次运行都是冷的**,而下面那 90 分钟 的预算正是按冷的算的。 两处一起改: * `MCPP_HOME: /home/runner/.mcpp` 进 env。mcpp 自己的 ci-linux.yml 为同一个理由 钉了它并写明「so the cache keys restore into the same path」。 * 判据不再假定路径,改成问 `mcpp self env`。 ⚠️ 本地预演之所以通过,是因为**开发构建被排除在自包含模式之外**,它确实用 `~/.mcpp` —— 预演与 CI 清的是两个不同的目录。判据的对象在两个环境里不是同一个。 --- .github/workflows/ci.yml | 40 +++++++++++++++++++++++++++++++--------- 1 file changed, 31 insertions(+), 9 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 27efad5..b18549b 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -10,6 +10,15 @@ env: # The mcpp release the consumers build with. Raising it is what admits a # member that relies on a newer engine; the README states each member's floor. MCPP_VERSION: 2026.9.6.6 + # PINNED, AND WITHOUT IT THE CACHE BELOW CACHED NOTHING. + # + # A released mcpp is self-contained: with no `MCPP_HOME`, `mcpp self env` + # reports the extracted tarball's own directory as its home, so every install + # landed in `$GITHUB_WORKSPACE/mcpp--linux-x86_64/` while the cache + # saved and restored `~/.mcpp` -- a directory mcpp never touched. Every run + # was therefore the cold case, which is what the 90-minute budget below was + # sized for. mcpp's own ci-linux.yml pins it for this reason and says so. + MCPP_HOME: /home/runner/.mcpp jobs: consumers: @@ -160,12 +169,24 @@ jobs: - name: the rules declare their own payloads run: | set -e - # THE PROVISIONING LINE ONLY PRINTS ON A STAMP MISS, and this job - # restores ~/.mcpp from a cache. Without clearing the record, the - # criterion's "no" and "this ran on a warm cache" are the same - # reading. Removing the stamp costs nothing: the packages are still - # installed, so the re-provision is one xlings round trip. - rm -rf ~/.mcpp/provisioned + # THE PROVISIONING LINE ONLY PRINTS ON A STAMP MISS, and the earlier + # steps of this job have already written those stamps. Without + # clearing them, the criterion's "no" and "this ran after the + # fixtures" are the same reading. Removing them costs nothing: the + # packages are still installed, so the re-provision is one xlings + # round trip. + # + # THE PATH IS ASKED FOR, NOT ASSUMED. A released mcpp is + # self-contained and its home is the extracted tarball unless + # MCPP_HOME says otherwise -- so `rm -rf ~/.mcpp/provisioned` removed + # nothing and this step failed on a working build. Measured: a local + # rehearsal passed because a DEV build is disqualified from + # self-contained mode and does use ~/.mcpp, so the rehearsal and CI + # were clearing different directories. + HOME_DIR=$("$MCPP" self env | awk -F'= *' '/^MCPP_HOME/{print $2; exit}') + [ -n "$HOME_DIR" ] || { echo "FAIL: could not read MCPP_HOME"; exit 1; } + echo "clearing provisioning stamps under $HOME_DIR" + rm -rf "$HOME_DIR/provisioned" for d in tests/cuda-consumer tests/hip-consumer \ tests/sycl-consumer tests/spirv-consumer; do # No project-side declaration is the first half of the claim, and @@ -191,13 +212,14 @@ jobs: echo "FAIL: $d no longer demonstrates the override"; exit 1; } ( cd "$d" && rm -rf target && "$MCPP" build > prov.log 2>&1 ) || { echo "build failed in $d"; tail -20 "$d/prov.log"; exit 1; } - n=$(ls -1 ~/.mcpp/registry/data/xpkgs/xim-x-glslang | wc -l | tr -d ' ') + XPKGS="$HOME_DIR/registry/data/xpkgs" + n=$(ls -1 "$XPKGS/xim-x-glslang" | wc -l | tr -d ' ') [ "$n" = "1" ] || { echo "FAIL: the override installed $n versions of glslang:" - ls -1 ~/.mcpp/registry/data/xpkgs/xim-x-glslang + ls -1 "$XPKGS/xim-x-glslang" exit 1 } - echo "ok: $d — one glslang, $(ls -1 ~/.mcpp/registry/data/xpkgs/xim-x-glslang)" + echo "ok: $d — one glslang, $(ls -1 "$XPKGS/xim-x-glslang")" rm -f "$d/prov.log" # THE HOST-LEAK ASSERTION, and what it measures was corrected once. From 38857f17cae858236b20856e03612118e05f2e48 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Mon, 7 Sep 2026 15:23:57 +0800 Subject: [PATCH 3/3] =?UTF-8?q?ci:=20=E9=92=89=E4=BD=8F=20MCPP=5FHOME=20?= =?UTF-8?q?=E4=B9=8B=E5=90=8E=E8=A6=81=E6=98=BE=E5=BC=8F=E6=8C=87=E5=87=BA?= =?UTF-8?q?=E8=87=AA=E5=B8=A6=E7=9A=84=20xlings=20=E5=9C=A8=E5=93=AA?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .github/workflows/ci.yml | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b18549b..cca1bd3 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -48,8 +48,18 @@ jobs: tar -xzf mcpp.tar.gz MCPP="$PWD/mcpp-${MCPP_VERSION}-linux-x86_64/bin/mcpp" "$MCPP" --version + # THE BUNDLED xlings, NAMED EXPLICITLY BECAUSE `MCPP_HOME` IS PINNED. + # + # A released tarball carries its own xlings at `registry/bin/xlings` + # and finds it there when its home IS that tarball. Pinning the home + # elsewhere moves the place mcpp looks, so the first command fails + # with `xlings binary not found` -- measured. The override is needed + # only for that first call: mcpp copies the binary into the pinned + # home and answers from there afterwards. + export MCPP_VENDORED_XLINGS="$PWD/mcpp-${MCPP_VERSION}-linux-x86_64/registry/bin/xlings" "$MCPP" self config --mirror GLOBAL echo "MCPP=$MCPP" >> "$GITHUB_ENV" + echo "MCPP_VENDORED_XLINGS=$MCPP_VENDORED_XLINGS" >> "$GITHUB_ENV" # The rule's output is a header holding the SPIR-V module; the program # checks the magic number in its first word, so no Vulkan runtime is