From 7301bcefb8d1c0d617a5ffd2622169f67871c58a Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Tue, 8 Sep 2026 08:24:55 +0800 Subject: [PATCH 1/3] 0.4.0: the surface becomes pure, and generation becomes an action MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every payload a compile reads is now in the build graph. Two mechanisms carry that, and which one applies is decided by WHEN the thing is known. DISCOVERED WHILE THE COMPILER RUNS: a shader's `#include` -------------------------------------------------------- `a.input()` is fixed when `build.mcpp` runs, before the compiler has read a line, so a shader or kernel that includes another file had no edge to it: editing that file rebuilt nothing and the build stayed green over a stale artifact. `grep depfile rules/` returned nothing across all six rules. 每一种拼法都用真实输出验过,不是照着文档写的: glslangValidator --depfile -> out.spv: scale.comp ./common.glsl glslc -MD -MF -> out2.spv: scale.comp common.glsl slangc -depfile -> out.spv: .slang .slang nvcc -MMD -MF -> n.o : k.cu \ common.cuh clang++ -MMD -MF -> c.o: k.cpp common.cuh bisheng -MMD -MF -> k2.o: k.asc inc.h ascendc 是唯一没有 CI 消费者的 lane,「bisheng 是 clang 血统所以应该支持」这种理由 会把未验证的旗标发进唯一没人检查的规则里。工具包在本机,于是问了它。`-MMD` 而不是 `-MD`:后者在 BiSheng 上实测拉进五十个 `/usr/include` 宿主头,让目标文件依赖共享构建 目录不该携带的绝对路径。 DISCOVERED BY NOBODY: `.incbin` ------------------------------- 汇编器在汇编期打开被内嵌的文件,而生成的 `.S` 自己的文本并不随载荷改变,于是目标文件 只被汇编一次。在已发布的 0.3.0 上、在沙箱里、对着索引解析出来的包实测:改一个 shader, 程序打印的还是上一版载荷的字节数(1480 -> 1480),而 header 存储是 1480 -> 1776。 问汇编器不行,这是实测的:编译器驱动的 `-MD` 是预处理器通道看不见 `.incbin`;GNU as 的 `--MD` 报得出来;clang 的集成汇编器根本没有依赖输出(三种拼法全被拒)。采信它会让这条 依赖在 GCC 上被跟踪、在 Clang 上静默缺失 —— 比两边都缺失更坏。 **修法不是给引擎加一条通道,而是不要在错误的时刻写那个文件。** 一条 `mcpp::action` 声明它的输入;把生成变成 action、载荷作它的声明输入,这条边就是普通的边,用的是引擎 已有的唯一图原语。原型先证后写:`BYTES=64 -> 192`,零引擎改动。 机制逐文件追踪过:载荷 mtime 前移、`.S` 不动(内容没变)、`.o` 前移 —— ninja 因 action 的声明输入变化而重跑它,无 `restat` 于是其 output 视为新的,汇编边随之重跑。去掉载荷 输入,`.o` 不动、字节停住。两个方向都量过。 这要求三件事,而每一件本身都是对的 -------------------------------- 1. **`mcpp.plugins.surface` 只 import `std`。** 它原先读 `mcpp::target_os()`、 `mcpp::compiler()`、`mcpp::package_name()`;现在这些是参数。一个取输入而不读环境的 生成器,在构建程序里和在普通程序里是同一份代码 —— 而 action 的命令必须是程序。 实测:此前这个包**不能**普通构建,`mcpp: failed to read compiled module`。 2. **`mcpp-embed` 由 mcpp 从本包源码造**,走 `tools = ["mcpp-embed"]`,与规则同一条 依赖边。不单独发包:`docs/05` §2.14 写明代价 —— 「工具的版本**就是**依赖的版本, 所以 protoc 与它的 runtime 不匹配这件事不可表达」。生成器与它写下的声明是一个决定。 3. **`src/declare.cppm` 是第二个单元**,承载构建程序侧(`mcpp::action` / `dep_bin` / 按存储分派)。它需要 mcpp 2026.9.8.1:此前一个包的 host module 按**路径**排序, `rules/` 在 `src/` 之前,import 它会失败。 两条 action 而不是一条,这是关于输入的事实而非限制:接口是条目表的函数,实现体还是 载荷的函数。合成一条会让改一个载荷去重写接口、重建每个 import 它的 BMI。 (`mcpp::action::provides` 也是 action 级的,一条 action 三个 output 一个 provides 会让扫描器报 "already provided by" —— 实测。) 只有 `object` 需要这一切。`header` 的字节经由载荷编译器已经写出的数据头(本就是 action 的 output)到达产物,`sidecar` 根本不进编译 —— 都检查过而不是假定。所以默认路径不造 任何工具,不选 object 存储的消费者一个字都不用多写。 其它 ---- `options::store` 改变了形状,所以这是破坏性发布:0.3.x -> 0.4.0。 `storage::object` 要求 `item::payload_path`,现在它拒绝 —— 这条约束一直写在注释里而 没有任何东西执行它。今天只有 spirv 能走到这条路且只有它设这个字段,那是巧合不是保证。 `rules/spirv.cppm` 里 `options::module_name` 的注释说默认值取自「包目录」,而它调用的 函数开头就写着「THE PACKAGE'S NAME. NOT ITS DIRECTORY'S.」。 CI 四条步骤,判据都落在产物的字节上,不落退出码也不落日志行;第四条断言「没要工具就 用 object 存储」被拒绝且点名了要加的键 —— 消息是契约。 --- .github/workflows/ci.yml | 153 +++++++++++++++- README.md | 61 ++++++- mcpp.toml | 61 ++++++- rules/ascendc.cppm | 21 +++ rules/cuda.cppm | 23 +++ rules/hip.cppm | 23 +++ rules/slang.cppm | 28 ++- rules/spirv.cppm | 70 +++++-- rules/sycl.cppm | 23 +++ src/declare.cppm | 205 +++++++++++++++++++++ src/plugins.cppm | 263 +++++++++++++++++++-------- tests/spirv-object-storage/mcpp.toml | 8 +- tools/embed.cppm | 13 +- tools/embed_main.cpp | 132 ++++++++++++++ 14 files changed, 972 insertions(+), 112 deletions(-) create mode 100644 src/declare.cppm create mode 100644 tools/embed_main.cpp diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 254c727..3f15e6b 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -9,7 +9,7 @@ on: env: # The mcpp release the consumers build with. Raising it is what admits a # member that relies on a newer engine; the README states each member's floor. - MCPP_VERSION: 2026.9.7.1 + MCPP_VERSION: 2026.9.8.1 # PINNED, AND WITHOUT IT THE CACHE BELOW CACHED NOTHING. # # A released mcpp is self-contained: with no `MCPP_HOME`, `mcpp self env` @@ -472,6 +472,157 @@ jobs: || { echo "FAIL: it failed for some other reason"; cat /tmp/away.log; exit 1; } echo "ok: found from the package root, reported missing from elsewhere" + # AN EDITED SHADER REACHES THE ARTIFACT, UNDER EVERY STORAGE. + # + # The three storage steps above assert STRUCTURE: the `.S` names the + # payload, the section is aligned, no C initialiser was written. All three + # passed against 0.3.0, in which object storage served a STALE artifact -- + # the shader recompiled, the object did not, and the program reported the + # previous payload's byte count with a green build. Structure cannot see + # that. Only editing a shader and reading what comes out can. + # + # THE DENOMINATOR IS THE POINT. Header storage was already correct, so a + # run where only `object` fails names the mechanism, and a run where all + # three fail names the harness -- a distinction one fixture cannot make. + # + # The edit adds a statement guarded by a value the host never passes, so + # the compiled module must grow while the program's output stays + # comparable. + - name: an edited shader reaches the artifact under every storage + run: | + set -u + fail=0 + for fx in spirv-module-consumer:header spirv-object-storage:object spirv-sidecar:sidecar; do + dir=${fx%%:*}; store=${fx##*:} + cd "$GITHUB_WORKSPACE/tests/$dir" + shader=$(find shaders -name '*.comp' | sort | head -1) + cp "$shader" /tmp/shader.bak + # `|| { ...; continue; }` and not a bare command: a `run:` block + # runs under `bash -e`, so a bare failing build would end the STEP + # here and the other two storages would never run -- losing the + # denominator this check is built around. + "$MCPP" build > /dev/null \ + || { echo "FAIL[$store]: build"; fail=1; cp /tmp/shader.bak "$shader"; continue; } + before=$("$MCPP" run 2>&1 | sed -n 's/.*bytes=\([0-9]*\).*/\1/p' | awk '{s+=$1} END {print s+0}') + sed -i 's|void main() {|void main() {\n if (push.n == 0xdeadu) { v[0] = 1.0; v[1] = 2.0; v[2] = 3.0; }|' "$shader" + "$MCPP" build > /dev/null \ + || { echo "FAIL[$store]: rebuild"; fail=1; cp /tmp/shader.bak "$shader"; continue; } + after=$("$MCPP" run 2>&1 | sed -n 's/.*bytes=\([0-9]*\).*/\1/p' | awk '{s+=$1} END {print s+0}') + cp /tmp/shader.bak "$shader" + if [ -z "$before" ] || [ -z "$after" ]; then + echo "FAIL[$store]: no byte count printed (before='$before' after='$after')" + fail=1 + elif [ "$before" = "$after" ]; then + echo "FAIL[$store]: the shader changed and the artifact did not (bytes=$before both times)" + fail=1 + else + echo "ok[$store]: $before -> $after" + fi + done + [ "$fail" -eq 0 ] || exit 1 + + # AN EDITED `#include` REACHES THE ARTIFACT. + # + # The step above edits the shader itself, which every rule declares as an + # action input. This edits a file the shader INCLUDES, which no rule + # declares: `a.input()` is fixed when build.mcpp runs, before the compiler + # has read a line, so the only channel is the depfile the compiler writes + # afterwards. `grep depfile rules/` returned nothing across all six rules + # until 0.3.1, and editing an included `.glsl` rebuilt nothing. + # + # ON THE HEADER-STORAGE FIXTURE, AND THAT IS NOT ARBITRARY. Run against + # `spirv-object-storage` this step tests two mechanisms at once: measured, + # it fails when EITHER the depfile or the `.incbin` declaration is absent, + # so a red run would not say which. Header storage compiles the payload in + # as generated source, where the compile edge is dirty for the ordinary + # reason, and what remains under test is the depfile alone. + # + # Both shader compilers were measured writing one that names the include: + # glslangValidator --depfile -> out.spv: scale.comp ./common.glsl + # glslc -MD -MF -> out2.spv: scale.comp common.glsl + - name: an edited include reaches the artifact + working-directory: tests/spirv-module-consumer + run: | + set -u + shader=shaders/default/scale.comp + inc=shaders/default/common.glsl + cp "$shader" /tmp/scale.bak + printf 'const float kBias = 0.0;\n' > "$inc" + sed -i 's|#version 450|#version 450\n#extension GL_GOOGLE_include_directive : require\n#include "common.glsl"|' "$shader" + sed -i 's|+ v\[push.n + i\];|+ v[push.n + i] + kBias;|' "$shader" + "$MCPP" build > /dev/null + before=$("$MCPP" run 2>&1 | sed -n 's/.*bytes=\([0-9]*\).*/\1/p' | awk '{s+=$1} END {print s+0}') + # ONLY THE INCLUDED FILE CHANGES NOW. Nothing any rule declared as an + # input is touched, so a rebuild happens only if the depfile said so. + printf 'const float kBias = 1.0;\nconst float kPad0 = 2.0;\nconst float kPad1 = 3.0;\n' > "$inc" + "$MCPP" build > /dev/null + after=$("$MCPP" run 2>&1 | sed -n 's/.*bytes=\([0-9]*\).*/\1/p' | awk '{s+=$1} END {print s+0}') + cp /tmp/scale.bak "$shader" + rm -f "$inc" + if [ -z "$before" ] || [ -z "$after" ]; then + echo "FAIL: no byte count printed (before='$before' after='$after')" + exit 1 + fi + if [ "$before" = "$after" ]; then + echo "FAIL: the included file changed and the artifact did not (bytes=$before both times)" + exit 1 + fi + echo "ok: an included file reaches the artifact ($before -> $after)" + + # OBJECT STORAGE WITHOUT THE TOOL IS REFUSED, AND THE REFUSAL SAYS WHAT + # TO ADD. + # + # `tools = [...]` is default-off: nothing is built unless a consumer asks, + # which is what keeps the DEFAULT storage free of a tool build. The cost + # is that a project can ask for object storage and not for the tool, and + # what it gets then is a message -- so the message is a contract. Asserted + # on the three strings a reader needs: that the tool is what is missing, + # the key that supplies it, and the storage that needs none. + - name: object storage without the tool is refused, naming the fix + run: | + set -u + W="$GITHUB_WORKSPACE/tests/spirv-object-storage" + cp "$W/mcpp.toml" /tmp/objstore.toml + # Remove only the tools request, leaving everything else in place. + sed -i 's/, tools = \["mcpp-embed"\]//' "$W/mcpp.toml" + cd "$W" + rm -rf target + if "$MCPP" build > refusal.log 2>&1; then + cp /tmp/objstore.toml mcpp.toml + echo "FAIL: object storage built with no way to generate its assembly" + exit 1 + fi + cp /tmp/objstore.toml mcpp.toml + fail=0 + grep -q 'mcpp-embed' refusal.log || { echo "FAIL: the refusal does not name the tool"; fail=1; } + grep -q 'tools = ' refusal.log || { echo "FAIL: the refusal does not name the key that supplies it"; fail=1; } + grep -q 'storage::header' refusal.log || { echo "FAIL: the refusal does not name the way out"; fail=1; } + if [ "$fail" -ne 0 ]; then cat refusal.log; exit 1; fi + rm -f refusal.log + echo "ok: refused, naming the tool, the key and the alternative" + + # EVERY RULE PASSES A DEPFILE, AND THE DENOMINATOR IS THE RULE COUNT. + # + # The step above proves the channel works for the one lane CI can run. + # The other four compile for accelerators no runner has, so what can be + # asserted about them is that each one declares the field -- read from the + # code rather than from a list written here, so a seventh rule is counted + # the day it is added and not the day someone remembers this step. + - name: every rule declares a depfile + run: | + set -u + cd "$GITHUB_WORKSPACE" + rules=$(ls rules/*.cppm | wc -l) + withdep=$(grep -l 'depfile' rules/*.cppm | wc -l) + echo "rules with a depfile: $withdep of $rules" + if [ "$withdep" -ne "$rules" ]; then + echo "FAIL: these rules pass no depfile:" + for f in rules/*.cppm; do + grep -q 'depfile' "$f" || echo " $f" + done + exit 1 + fi + # SLANG: A DIFFERENT LANGUAGE, THE SAME SURFACE. # # The point of this step is not that Slang compiles -- it is that a diff --git a/README.md b/README.md index 0cebd4f..87fa346 100644 --- a/README.md +++ b/README.md @@ -127,12 +127,28 @@ device source that reached no action, naming the file. That is the engine's half of this rule and it needs 2026.9.6.5. The floor is the mcpp release whose engine carries what the member relies on. -From 0.3.0 every rule shares one: **2026.9.7.1**, the release that reads +From 0.4.0 every rule shares one: **2026.9.8.1**, the release in which a +package's host modules are ordered by their IMPORT GRAPH rather than by their +paths. This package needs that: `src/declare.cppm` is imported by every member, +and `rules/` sorts before `src/`, so before that release the members were +compiled first and failed with "failed to read compiled module". + +**The floor could have been avoided, and was not.** `src/declare.cppm` sorts +after `rules/`, which is exactly why it needs the ordering fix -- and naming it +`aa_declare.cppm` at the package root would make the old PATH order happen to be +correct, so 0.4.0 would run on 2026.9.7.1 with no floor move at all. That is +declined on purpose: it encodes a load-bearing constraint in a filename with +nothing enforcing it, which is the fragility the engine fix removes. A file +renamed for a reason nobody can see is a defect waiting for the rename that +looks harmless. + +The previous shared floor was 2026.9.7.1, the release that reads `device_extensions` and `rule_module`, reports `[language] modules` and the -package's own name to a build program, and writes the build program a declared -rule set describes. A client below it does not get a degraded surface; it gets a -build in which the rules never route -- the file falls through to the ordinary -source scan and mcpp says it has no role for the extension. +package's own name to a build program, writes the build program a declared rule +set describes, and gives `mcpp::action` its `depfile` field. A client below it +does not get a degraded surface; it gets a build in which the rules never route +-- the file falls through to the ordinary source scan and mcpp says it has no +role for the extension. The previous shared floor was 2026.9.6.6, the release in which a payload a DEPENDENCY declared is both installed and answerable. Before it a rule could @@ -224,6 +240,41 @@ identical under all three, so a project changes this and no consumer changes. | `object` | a section, through `.incbin` in a generated `.S` | total payload is large | | `sidecar` | a file beside the artifact, read at run time | hot reload, or a payload too large to link | +**Every payload a compile reads is in the build graph.** Two mechanisms carry +that, and which one applies is decided by WHEN the thing is known. + +A shader's `#include` is discovered by the compiler while it runs, so it arrives +afterwards, in a depfile. `mcpp::action::depfile` carries it and all six rules +pass one -- each spelling measured against the tool rather than read from its +help text. + +An `.incbin` is discovered by nobody. The assembler opens the file at assembly +time; the generated `.S`'s own text does not change when the payload does; the +object is assembled once. Measured on 0.3.0, in a sandbox against the published +packages: editing a shader left the program printing the previous payload's byte +count, with a green build. + +Asking the assembler does not fix it, and that was measured rather than assumed. +The compiler driver's `-MD` is a preprocessor channel that never sees `.incbin`; +GNU as names it in its own `--MD`; clang's integrated assembler has no +dependency output of any kind. Tracking it that way would work under GCC and +fail silently under Clang -- worse than failing under both. + +**So under `object` storage the generation is an ACTION and the payloads are its +declared inputs.** That is the one graph primitive the engine has, used for what +it is: a payload changes, the action reruns, its outputs count as new, and the +edge that assembles them reruns. The command is `mcpp-embed`, built from this +package through `tools = ["mcpp-embed"]` -- not published separately, because +docs/05 section 2.14 states what that costs: "the tool's version IS the +dependency's version, so a `protoc` that does not match its runtime is not +expressible." + +`header` and `sidecar` need none of it, and that was checked rather than +assumed. Under `header` the bytes reach the artifact through generated data +headers the payload's own compiler already writes as action outputs; under +`sidecar` they are never compiled at all. So the default path builds no tool, +and a consumer that never opts into object storage writes nothing extra. + **Which one is a measurement, not a preference.** With GCC 16.1 on 100 payloads of 16 KB each -- the size of an ordinary compute shader: diff --git a/mcpp.toml b/mcpp.toml index 85c431e..fa62ca2 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,7 +1,7 @@ [package] name = "plugins" namespace = "mcpp" -version = "0.3.0" +version = "0.4.0" description = "Official mcpp build plugins: rule packages under mcpp.rules.*, build-time utilities under mcpp.tools.*, each member selected by a feature" license = "Apache-2.0" authors = ["mcpp-community"] @@ -17,6 +17,26 @@ import_std = true # the source set; mcpp compiles every interface unit among the resolved # sources as a host module under the name the unit declares (2026.9.5.3+). [build] +# THE LIB ROOT ALONE, AND `src/declare.cppm` BEHIND A FEATURE. +# +# The two units divide by what they may import: +# +# src/plugins.cppm `mcpp.plugins.surface`, importing only `std`, so the +# same code compiles into `mcpp-embed` -- an ordinary +# program, which is what lets the generation be an ACTION +# and a payload be that action's declared input +# src/declare.cppm the build-program half: `mcpp::action`, `mcpp::generated` +# and the decision of how the generation reaches the graph +# +# The second one imports `mcpp`, which exists only inside a build program, so +# listing it here would break the ORDINARY build of this package -- the build +# that produces the tool. `[features.surface]` scopes it to the consumers that +# have a build program, and every member implies it. +# +# It also needs mcpp 2026.9.8.1: before that release a package's host modules +# were ordered by PATH, so `rules/spirv.cppm` was compiled before +# `src/declare.cppm` and importing it failed with "failed to read compiled +# module". They are ordered by their import graph now. sources = ["src/plugins.cppm"] # EACH RULE STATES WHAT IT COMPILES AND HOW TO REACH IT (mcpp 2026.9.7.1+). @@ -46,23 +66,33 @@ sources = ["src/plugins.cppm"] [features] default = [] +# The build-program half of the surface. Not a member a consumer names: every +# member that embeds a payload implies it, and a consumer that activates none +# of them compiles neither this nor anything that imports `mcpp`. +[features.surface] +sources = ["src/declare.cppm"] + [features.rules-ascendc] sources = ["rules/ascendc.cppm"] +implies = ["surface"] rule_module = "mcpp.rules.ascendc" device_extensions = [".asc", ".cce"] [features.rules-cuda] sources = ["rules/cuda.cppm"] +implies = ["surface"] rule_module = "mcpp.rules.cuda" device_extensions = [".cu"] [features.rules-hip] sources = ["rules/hip.cppm"] +implies = ["surface"] rule_module = "mcpp.rules.hip" device_extensions = [".hip"] [features.rules-slang] sources = ["rules/slang.cppm"] +implies = ["surface"] rule_module = "mcpp.rules.slang" device_extensions = [".slang"] @@ -71,6 +101,7 @@ device_extensions = [".slang"] # message than the engine's "no rule compiles it". [features.rules-spirv] sources = ["rules/spirv.cppm"] +implies = ["surface"] rule_module = "mcpp.rules.spirv" device_extensions = [".comp", ".vert", ".frag", ".geom", ".tesc", ".tese", ".mesh", ".task", ".rgen", ".rint", ".rahit", ".rchit", @@ -78,6 +109,7 @@ device_extensions = [".comp", ".vert", ".frag", ".geom", ".tesc", ".tese", [features.rules-sycl] sources = ["rules/sycl.cppm"] +implies = ["surface"] rule_module = "mcpp.rules.sycl" device_extensions = [".sycl"] @@ -86,9 +118,11 @@ device_extensions = [".sycl"] # `build.mcpp`. [features.tools-embed] sources = ["tools/embed.cppm"] +implies = ["surface"] [features.tools-island] sources = ["tools/island.cppm"] +implies = ["surface"] # ── The environment each rule needs (mcpp 2026.9.6.6+) ────────────────────── # @@ -257,3 +291,28 @@ sources = ["tools/island.cppm"] [targets.plugins] kind = "lib" + +# mcpp-embed -- the surface generator as a PROGRAM, so that the graph can invoke +# it and the payloads it embeds can be declared inputs of that invocation. +# +# WHY THE TOOL IS BUILT FROM THIS PACKAGE RATHER THAN PUBLISHED SEPARATELY. +# docs/05 section 2.14 states the reason, and it is not convenience: "The tool's +# version IS the dependency's version, so a `protoc` that does not match its +# runtime is not expressible. This is the problem with packaging the tool +# separately, and it is the failure mode that bites at run time rather than +# compile time." The generator and the declarations it writes are one decision; +# a separately published payload would let them drift by one release. +# +# A consumer asks for it on the same edge that asks for the rules: +# +# [build-dependencies.mcpp] +# plugins = { version = "0.4.0", features = ["rules-spirv"], +# host-module = true, tools = ["mcpp-embed"] } +# +# Only under `storage::object`. The default storage compiles the payload in as +# generated source and needs no program, so nothing is built unless someone +# asks -- and a rule that needs it and cannot find it refuses, naming the line +# to add. +[targets.mcpp-embed] +kind = "bin" +main = "tools/embed_main.cpp" diff --git a/rules/ascendc.cppm b/rules/ascendc.cppm index b7325d4..b3ba5f6 100644 --- a/rules/ascendc.cppm +++ b/rules/ascendc.cppm @@ -269,6 +269,20 @@ struct options { struct edge { std::string id, description; std::vector command, inputs, outputs; + // A depfile the COMMAND writes and ninja reads back. `inputs` is fixed + // when this program runs, before the compiler has seen the source, so a + // device unit that `#include`s a header had no edge to it: editing the + // header rebuilt nothing and the build stayed green over a stale object. + // The compiler already computes the answer while parsing. + // + // `-MMD` and not `-MD`, which is the choice mcpp makes for its own C and + // GAS units: user includes only. `-MD` was measured on BiSheng and pulled + // in fifty host headers under /usr/include -- correct, and useless, because + // it makes the object depend on absolute host paths that a shared build + // directory must not carry. A toolkit header is not missed by this: the + // toolkit's path carries its version, so a different toolkit is a different + // command line, which ninja already tracks. + std::string depfile; }; inline std::vector plan(std::span sources, options opt = {}) { @@ -344,6 +358,10 @@ inline std::vector plan(std::span sources, options opt e.command.push_back("-I" + (std::filesystem::path(d).is_absolute() ? d : root + "/" + d)); for (auto const& f : opt.flags) e.command.push_back(f); + e.depfile = obj + ".d"; + e.command.push_back("-MMD"); + e.command.push_back("-MF"); + e.command.push_back(e.depfile); e.command.push_back("-c"); e.command.push_back(std::filesystem::path(src).is_absolute() ? src : root + "/" + src); @@ -367,6 +385,9 @@ inline bool submit(std::span edges) { for (auto const& c : e.command) a.arg(c.c_str()); for (auto const& i : e.inputs) a.input(i.c_str()); for (auto const& o : e.outputs) a.output(o.c_str()); + // Empty for an edge that declares none, which serialises identically to + // an action from before the field existed -- see mcpp::action::depfile. + if (!e.depfile.empty()) a.depfile = e.depfile.c_str(); a.submit(); } return true; diff --git a/rules/cuda.cppm b/rules/cuda.cppm index 964d3dc..e0ccf1b 100644 --- a/rules/cuda.cppm +++ b/rules/cuda.cppm @@ -498,6 +498,20 @@ inline std::optional unreachable_stage(const toolkit& t, const std: struct edge { std::string id, description; std::vector command, inputs, outputs; + // A depfile the COMMAND writes and ninja reads back. `inputs` is fixed + // when this program runs, before the compiler has seen the source, so a + // device unit that `#include`s a header had no edge to it: editing the + // header rebuilt nothing and the build stayed green over a stale object. + // The compiler already computes the answer while parsing. + // + // `-MMD` and not `-MD`, which is the choice mcpp makes for its own C and + // GAS units: user includes only. `-MD` was measured on BiSheng and pulled + // in fifty host headers under /usr/include -- correct, and useless, because + // it makes the object depend on absolute host paths that a shared build + // directory must not carry. A toolkit header is not missed by this: the + // toolkit's path carries its version, so a different toolkit is a different + // command line, which ninja already tracks. + std::string depfile; }; // ON WINDOWS THE DEFAULT IS THE CLANG ROUTE WHATEVER THE PROJECT'S COMPILER IS. @@ -740,6 +754,12 @@ inline std::vector plan(std::span sources, options opt e.command.push_back("-I" + (std::filesystem::path(inc).is_absolute() ? inc : root + "/" + inc)); for (auto const& f : opt.flags) e.command.push_back(f); + // Where the compiler reports what it read. `.d`, so two + // architectures of one source do not share a file and overwrite each + // other's answer -- the object name already carries the arch for + // exactly that reason. + e.depfile = obj + ".d"; + e.command.insert(e.command.end(), { "-MMD", "-MF", e.depfile }); e.command.insert(e.command.end(), { "-c", root + "/" + src, "-o", obj }); e.inputs = { root + "/" + src }; e.outputs = { obj }; @@ -757,6 +777,9 @@ inline bool submit(std::span edges) { for (auto const& c : e.command) a.arg(c.c_str()); for (auto const& i : e.inputs) a.input(i.c_str()); for (auto const& o : e.outputs) a.output(o.c_str()); + // Empty for an edge that declares none, which serialises identically to + // an action from before the field existed -- see mcpp::action::depfile. + if (!e.depfile.empty()) a.depfile = e.depfile.c_str(); a.submit(); } return true; diff --git a/rules/hip.cppm b/rules/hip.cppm index f216889..f5eceef 100644 --- a/rules/hip.cppm +++ b/rules/hip.cppm @@ -213,6 +213,20 @@ inline std::string hip_version(const std::string& hip_root) { struct edge { std::string id, description; std::vector command, inputs, outputs; + // A depfile the COMMAND writes and ninja reads back. `inputs` is fixed + // when this program runs, before the compiler has seen the source, so a + // device unit that `#include`s a header had no edge to it: editing the + // header rebuilt nothing and the build stayed green over a stale object. + // The compiler already computes the answer while parsing. + // + // `-MMD` and not `-MD`, which is the choice mcpp makes for its own C and + // GAS units: user includes only. `-MD` was measured on BiSheng and pulled + // in fifty host headers under /usr/include -- correct, and useless, because + // it makes the object depend on absolute host paths that a shared build + // directory must not carry. A toolkit header is not missed by this: the + // toolkit's path carries its version, so a different toolkit is a different + // command line, which ninja already tracks. + std::string depfile; }; // ─── This rule's share of the device sources ─────────────────────────────── @@ -432,6 +446,12 @@ inline std::vector plan(std::span sources, options opt e.command.push_back("-I" + (std::filesystem::path(inc).is_absolute() ? inc : root + "/" + inc)); for (auto const& f : opt.flags) e.command.push_back(f); + // Where the compiler reports what it read. `.d`, so two + // architectures of one source do not share a file and overwrite each + // other's answer -- the object name already carries the arch for + // exactly that reason. + e.depfile = obj + ".d"; + e.command.insert(e.command.end(), { "-MMD", "-MF", e.depfile }); e.command.insert(e.command.end(), { "-c", root + "/" + src, "-o", obj }); e.inputs = { root + "/" + src }; e.outputs = { obj }; @@ -449,6 +469,9 @@ inline bool submit(std::span edges) { for (auto const& c : e.command) a.arg(c.c_str()); for (auto const& i : e.inputs) a.input(i.c_str()); for (auto const& o : e.outputs) a.output(o.c_str()); + // Empty for an edge that declares none, which serialises identically to + // an action from before the field existed -- see mcpp::action::depfile. + if (!e.depfile.empty()) a.depfile = e.depfile.c_str(); a.submit(); } return true; diff --git a/rules/slang.cppm b/rules/slang.cppm index 932be07..094bc3d 100644 --- a/rules/slang.cppm +++ b/rules/slang.cppm @@ -50,6 +50,7 @@ import mcpp; // consumer names, shared with `mcpp.rules.spirv` so a project that has both // reaches them through one shape. import mcpp.plugins; +import mcpp.plugins.declare; // `std::println` is avoided here for the reason every file in this package // records: it is not header-only, and the symbols its overloads reach for were @@ -340,7 +341,10 @@ inline bool compile(std::span shaders, options opt = {}) { const std::string baseDir = opt.base_dir.empty() ? common_base_dir(shaders) : opt.base_dir; const std::string moduleName = opt.module_name.empty() - ? mcpp::plugins::surface::module_root_from_package() + ".shaders" + ? mcpp::plugins::surface::module_root_for( + mcpp::package_name(), + std::filesystem::path(mcpp::manifest_dir()).filename().string()) + + ".shaders" : opt.module_name; // Two shaders whose stem and directory both match would produce one output, @@ -424,6 +428,17 @@ inline bool compile(std::span shaders, options opt = {}) { a.arg("-source-embed-style"); a.arg("u32"); a.arg("-source-embed-name"); a.arg(sym.c_str()); a.arg("-o"); a.arg(inc.c_str()); + // What the shader `#include`s, which only slangc can know. `a.input()` + // below names the `.slang` and is fixed here, before the compiler has + // read a line; a shader including a second `.slang` therefore had no + // edge to it, and editing that file left a stale module behind a green + // build. slangc computes the answer while parsing and writes it out. + // + // Measured: `slangc ... -depfile s.d` writes + // `out.spv: .slang .slang`. + const std::string dep = inc + ".d"; + a.arg("-depfile"); a.arg(dep.c_str()); + a.depfile = dep.c_str(); a.arg(input.c_str()); a.input(input.c_str()); a.output(inc.c_str()); @@ -438,12 +453,13 @@ inline bool compile(std::span shaders, options opt = {}) { so.module_name = moduleName; so.out_dir = gen; so.produced_by = "mcpp.rules.slang"; + // Answered here, not read there: `mcpp.plugins.surface` compiles into a + // plain binary as well as into this build program, so it takes its inputs. + so.target_os = mcpp::target_os(); + so.has_gas_assembler = std::string_view(mcpp::compiler()) != "msvc"; - const auto out = mcpp::plugins::surface::emit(items, so); - if (!out) return false; - mcpp::generated(out->interface_file.c_str()); - mcpp::generated(out->impl_file.c_str()); - if (!out->include_dir.empty()) mcpp::include_dir(out->include_dir.c_str()); + const auto out = mcpp::plugins::surface_for(items, so); + if (!out.ok) return false; mcpp::fact("mcpp.plugins", std::string(mcpp::plugins::version).c_str()); return true; diff --git a/rules/spirv.cppm b/rules/spirv.cppm index 4880587..1f8e145 100644 --- a/rules/spirv.cppm +++ b/rules/spirv.cppm @@ -62,6 +62,7 @@ import mcpp; // The lib root, which carries `mcpp::plugins::surface` -- the declarations a // consumer names, written once for every member that embeds a payload. import mcpp.plugins; +import mcpp.plugins.declare; // WHY NOTHING HERE USES `std::println`, AND WHY THAT IS NOT A STYLE CHOICE. @@ -126,8 +127,12 @@ struct options { // The module a consumer imports, and the namespace the declarations sit in: // `myapp.shaders` gives `myapp::shaders::blur_comp()`. Empty derives it from - // the package directory, so a project that states nothing still gets a name - // no other package in the build can claim. + // the PACKAGE NAME -- `[package] name`, not the directory the package + // happens to sit in -- so a project that states nothing still gets a name + // no other package in the build can claim. The two are different questions + // whenever a package is laid out under a generic directory, and + // `mcpp::plugins::surface::module_root_for` records what it cost to answer + // the wrong one. std::string module_name; // The directory shader paths are made relative to when deriving namespaces. @@ -643,7 +648,10 @@ inline bool compile(std::span shaders, options opt = {}) { // named one, so two packages in one build cannot claim the same module. const std::string moduleName = opt.module_name.empty() - ? mcpp::plugins::surface::module_root_from_package() + ".shaders" + ? mcpp::plugins::surface::module_root_for( + mcpp::package_name(), + std::filesystem::path(mcpp::manifest_dir()).filename().string()) + + ".shaders" : opt.module_name; // TWO SHADERS THAT DIFFER ONLY BY DIRECTORY PRODUCE ONE HEADER AND ONE @@ -826,6 +834,23 @@ inline bool compile(std::span shaders, options opt = {}) { // compilers write the same thing: a bare SPIR-V module. The one place // the two flavours differed disappears with the storage that needed it. a.arg("-o"); a.arg(output.c_str()); + // WHAT THE SHADER `#include`s, WHICH ONLY THE COMPILER CAN KNOW. + // + // `a.input()` below names the `.comp` and nothing else, and it is fixed + // when this program runs, before the compiler has read a line. A shader that includes a + // `.glsl` therefore had no edge to that file: editing it rebuilt + // nothing and the build stayed green over a stale SPIR-V module. Both + // compilers already compute the answer while parsing and will write it + // out; the field to receive it arrived in mcpp 2026.9.7.1. + // + // Two spellings of one idea, as everywhere else in this rule. + const std::string dep = output + ".d"; + if (cc.kind == flavour::glslc) { + a.arg("-MD"); a.arg("-MF"); a.arg(dep.c_str()); + } else { + a.arg("--depfile"); a.arg(dep.c_str()); + } + a.depfile = dep.c_str(); a.arg(input.c_str()); a.input(input.c_str()); a.output(output.c_str()); @@ -845,22 +870,31 @@ inline bool compile(std::span shaders, options opt = {}) { so.module_name = moduleName; so.out_dir = gen; so.produced_by = "mcpp.rules.spirv"; + // Answered here, not read there: `mcpp.plugins.surface` compiles into a + // plain binary as well as into this build program, so it takes its inputs. + so.target_os = mcpp::target_os(); + so.has_gas_assembler = std::string_view(mcpp::compiler()) != "msvc"; + + // Generated and declared by whichever route the storage requires: plan + // time for `header` and `sidecar`, an action for `object`. See + // `mcpp::plugins::surface_for` for why that is a property of the storage + // rather than a choice this rule makes. + const auto out = mcpp::plugins::surface_for(items, so); + if (!out.ok) return false; + + // THE DEGRADATION IS REPORTED HERE BECAUSE THE GENERATOR CANNOT REPORT IT. + // + // `mcpp::warning` is a build-program channel and `mcpp.plugins.surface` + // deliberately has none: it compiles into a plain binary as well. So the + // generator states the storage it actually used and the caller says so. + // Whoever sets `options::store` owns this: leaving it out would turn a + // stated fallback into a silent one, which is the shape this whole round + // is about. + if (out.files.store != so.store) + mcpp::warning("mcpp.rules.spirv: object storage needs a GAS assembler and this " + "toolchain has none; the payload is compiled in as generated source " + "instead. The declarations a consumer sees are unchanged."); - const auto out = mcpp::plugins::surface::emit(items, so); - if (!out) return false; - - // Both generated files are written above, so the ordinary source scan sees - // real content rather than a placeholder -- which is what lets a generated - // module interface be an ordinary node in the module graph with nothing - // declared about it. - mcpp::generated(out->interface_file.c_str()); - mcpp::generated(out->impl_file.c_str()); - // The `.S` under object storage. It is an ordinary source: mcpp assembles - // it, and `.incbin` reads the payload the action above produced, which by - // then exists because a `role = "source"` action is ordered before this - // package's compiles. - if (!out->assembly_file.empty()) mcpp::generated(out->assembly_file.c_str()); - if (!out->include_dir.empty()) mcpp::include_dir(out->include_dir.c_str()); // Which collection produced these actions. The version was `0.1.1` while // the package was `0.2.6` for as long as nothing read it; a fact is a diff --git a/rules/sycl.cppm b/rules/sycl.cppm index fe0ba89..15f1ccc 100644 --- a/rules/sycl.cppm +++ b/rules/sycl.cppm @@ -266,6 +266,20 @@ inline std::string compiler_version(const std::string& exe) { struct edge { std::string id, description, role; std::vector command, inputs, outputs; + // A depfile the COMMAND writes and ninja reads back. `inputs` is fixed + // when this program runs, before the compiler has seen the source, so a + // device unit that `#include`s a header had no edge to it: editing the + // header rebuilt nothing and the build stayed green over a stale object. + // The compiler already computes the answer while parsing. + // + // `-MMD` and not `-MD`, which is the choice mcpp makes for its own C and + // GAS units: user includes only. `-MD` was measured on BiSheng and pulled + // in fifty host headers under /usr/include -- correct, and useless, because + // it makes the object depend on absolute host paths that a shared build + // directory must not carry. A toolkit header is not missed by this: the + // toolkit's path carries its version, so a different toolkit is a different + // command line, which ninja already tracks. + std::string depfile; }; // ─── This rule's share of the device sources ─────────────────────────────── @@ -499,6 +513,12 @@ inline std::vector plan(std::span sources, options opt // no compiler knows it; without this the driver classifies the file as // a LINKER INPUT, warns `'linker' input unused`, exits 0 and produces // nothing. + // Where the compiler reports what it read. `.d`, so two + // architectures of one source do not share a file and overwrite each + // other's answer -- the object name already carries the arch for + // exactly that reason. + e.depfile = obj + ".d"; + e.command.insert(e.command.end(), { "-MMD", "-MF", e.depfile }); e.command.insert(e.command.end(), { "-x", "c++", "-c", root + "/" + src, "-o", obj }); e.inputs = { root + "/" + src }; e.outputs = { obj }; @@ -539,6 +559,9 @@ inline bool submit(std::span edges) { for (auto const& c : e.command) a.arg(c.c_str()); for (auto const& i : e.inputs) a.input(i.c_str()); for (auto const& o : e.outputs) a.output(o.c_str()); + // Empty for an edge that declares none, which serialises identically to + // an action from before the field existed -- see mcpp::action::depfile. + if (!e.depfile.empty()) a.depfile = e.depfile.c_str(); a.submit(); } return true; diff --git a/src/declare.cppm b/src/declare.cppm new file mode 100644 index 0000000..8feb442 --- /dev/null +++ b/src/declare.cppm @@ -0,0 +1,205 @@ +// mcpp.plugins.declare -- the build-program half of the surface. +// +// WHY THIS IS A SECOND UNIT AND NOT MORE OF THE LIB ROOT. +// +// `mcpp.plugins.surface` decides every name a consumer sees and writes every +// generated file. It imports only `std`, and that is load-bearing rather than +// tidy: the same code has to compile into `mcpp-embed`, an ordinary program, +// because a payload can only be a declared input of the edge that embeds it if +// that edge is an ACTION -- and an action's command is a binary. +// +// This unit is the other side. It knows `mcpp::action`, `mcpp::generated`, +// `mcpp::dep_bin` and the rest of the build-program contract, and it decides +// HOW the generation reaches the graph. Keeping the two apart is what lets one +// of them exist twice. +// +// A second unit beside the lib root was not possible before mcpp 2026.9.8.1: +// the units a host-module package contributes were ordered by PATH, so +// `rules/spirv.cppm` was compiled before `src/declare.cppm` and importing it +// failed with "failed to read compiled module". They are ordered by their +// import graph now, which is what makes this file expressible. + +export module mcpp.plugins.declare; + +import std; +import mcpp; +import mcpp.plugins; + +export namespace mcpp::plugins { + +// WHERE THE GENERATION HAPPENS, AND WHY IT DEPENDS ON THE STORAGE. +// +// Two of the three storages can be generated at PLAN time, and one cannot. +// +// header The bytes reach the artifact through generated data headers that +// the payload's own compiler writes as an action output. The edge +// from a changed payload to the artifact already exists, through +// that action. Generating the surface here costs nothing and needs +// no program. +// +// sidecar The bytes are never compiled. The accessor opens the file at RUN +// time, so a rebuilt payload is picked up by the next run with +// nothing in the graph to keep current. +// +// object The bytes become a section, through `.incbin` in a generated +// assembly source. The assembly's TEXT is plan-time knowledge; the +// OBJECT it produces is the payload's bytes. A file written at plan +// time cannot be an edge to a build-time product, so the object was +// assembled once and every later payload change was a green build +// over stale bytes -- reproduced against `mcpp:plugins` 0.3.0 in a +// sandbox, from published packages. +// +// So under this storage the generation IS an action, its command is +// `mcpp-embed`, and the payloads are its declared inputs. The edge +// is then an ordinary one, expressed with the graph primitive the +// engine already has. +// +// TWO ACTIONS, NOT ONE, AND THAT IS NOT A LIMITATION BUT A FACT ABOUT INPUTS. +// The interface is a function of the item list alone; the body additionally of +// the payloads. One action would rewrite the interface whenever a payload +// changed and rebuild every BMI importing it. (`mcpp::action::provides` is also +// per-action rather than per-output, so a single action declaring three outputs +// and one module makes the scanner report "already provided by" -- measured.) +struct declared { + surface::emitted files; + bool ok = false; +}; + +// Writes the manifest `mcpp-embed` reads. Plan-time knowledge only: every value +// here is something the rule computed from the glob and the options. +inline bool write_manifest(const std::string& path, + std::span items, + const surface::options& opt, + std::string_view which) +{ + auto surface_name = opt.surface == surface::kind::c_header ? "c_header" : "module"; + auto storage_name = opt.store == surface::storage::object ? "object" + : opt.store == surface::storage::sidecar ? "sidecar" : "header"; + auto element_name = opt.elem == surface::element::byte_ ? "byte" : "word32"; + std::string m; + m += std::format("# written by mcpp.plugins.declare; read by mcpp-embed\n" + "surface {}\nstorage {}\nelement {}\nmodule {}\noutdir {}\n" + "by {}\nos {}\ngas {}\nemit {}\n", + surface_name, storage_name, element_name, opt.module_name, + opt.out_dir, opt.produced_by, opt.target_os, + opt.has_gas_assembler ? 1 : 0, which); + for (auto const& it : items) { + m += std::format("item {}\n", it.identifier); + for (auto const& n : it.name_space) m += std::format("ns {}\n", n); + if (!it.data_header.empty()) m += std::format("header {}\n", it.data_header); + if (!it.data_symbol.empty()) m += std::format("symbol {}\n", it.data_symbol); + if (!it.payload_path.empty()) m += std::format("payload {}\n", it.payload_path); + if (!it.sidecar_name.empty()) m += std::format("sidecar {}\n", it.sidecar_name); + if (!it.data_size_expr.empty()) m += std::format("size {}\n", it.data_size_expr); + } + std::ofstream out(path, std::ios::binary | std::ios::trunc); + if (!out) { + std::cerr << std::format("mcpp.plugins.declare: cannot write {}\n", path); + return false; + } + out << m; + out.close(); + // Checked on close as well as on open: a full filesystem fails here and + // nowhere else, and a silently short manifest would produce a surface + // missing its last payloads. + if (!out) { + std::cerr << std::format("mcpp.plugins.declare: failed while writing {}\n", path); + return false; + } + return true; +} + +// Generate the surface and tell mcpp about it, by whichever route the storage +// requires. Returns the file names so the caller can put a directory on the +// include path or state the module it provides. +inline declared surface_for(std::span items, + const surface::options& opt) +{ + declared d; + d.files = surface::outputs(opt); + if (items.empty()) { d.ok = true; return d; } + + if (d.files.store != surface::storage::object) { + // Plan time is correct here: nothing this writes is an edge to a + // build-time product. + if (!surface::write(items, opt, surface::half::interface_)) return d; + if (!surface::write(items, opt, surface::half::body)) return d; + mcpp::generated(d.files.interface_file.c_str()); + mcpp::generated(d.files.impl_file.c_str()); + if (!d.files.include_dir.empty()) mcpp::include_dir(d.files.include_dir.c_str()); + d.ok = true; + return d; + } + + // ── object storage: the generation is an action ───────────────────────── + const std::string tool = mcpp::dep_bin("plugins", "mcpp-embed"); + if (tool.empty()) { + // NAMED, NOT GUESSED. The tool is default-off, so a project that asks + // for this storage without asking for the tool gets the one message + // that says what to add and where. + std::cerr << std::format( + "{}: storage::object needs the `mcpp-embed` tool, and this build did not " + "ask for it.\n" + " The generated assembly names each payload in `.incbin`, so it has to be\n" + " produced by an action whose inputs are those payloads -- and an action's\n" + " command is a program. Add it to the dependency that already brings the\n" + " rules in:\n\n" + " [build-dependencies.mcpp]\n" + " plugins = {{ version = \"...\", features = [...], host-module = true,\n" + " tools = [\"mcpp-embed\"] }}\n\n" + " Or use storage::header, which needs no program.\n", + opt.produced_by.empty() ? "mcpp.plugins.declare" : opt.produced_by); + return d; + } + + const std::string base = opt.out_dir + "/" + opt.module_name; + const std::string mi = base + ".surface-interface.txt"; + const std::string mb = base + ".surface-body.txt"; + if (!write_manifest(mi, items, opt, "interface")) return d; + if (!write_manifest(mb, items, opt, "body")) return d; + + // The interface: the item list decides it, so the manifest is its only + // input. `provides` is what lets a generated `.cppm` be a graph node + // without being scanned. + // THE ID CARRIES THE PRODUCER, NOT ONLY THE MODULE. + // + // `mcpp.rules.spirv` and `mcpp.rules.slang` both default their module to + // `.shaders`, so a project driving both would submit two actions + // under one id if the id were the module alone. That collision is not + // introduced here -- two rules defaulting to one module name is a question + // of its own -- but an id that cannot collide costs nothing. + const std::string who = opt.produced_by.empty() ? std::string("mcpp.plugins") + : opt.produced_by; + mcpp::action iface; + const std::string ifaceId = who + ":" + opt.module_name + ":surface-interface"; + iface.id = ifaceId.c_str(); + iface.role = "source"; + iface.description = "mcpp-embed interface"; + iface.arg(tool.c_str()).arg(mi.c_str()) + .input(mi.c_str()) + .output(d.files.interface_file.c_str()); + if (opt.surface == surface::kind::module_) iface.provides(opt.module_name.c_str()); + iface.submit(); + + // The body: the item list AND the payloads. This is the edge the whole + // round is about -- a payload named here is a payload the graph knows the + // object depends on. + mcpp::action body; + const std::string bodyId = who + ":" + opt.module_name + ":surface-body"; + body.id = bodyId.c_str(); + body.role = "source"; + body.description = "mcpp-embed body"; + body.arg(tool.c_str()).arg(mb.c_str()) + .input(mb.c_str()) + .output(d.files.impl_file.c_str()) + .output(d.files.assembly_file.c_str()); + for (auto const& it : items) + if (!it.payload_path.empty()) body.input(it.payload_path.c_str()); + body.submit(); + + if (!d.files.include_dir.empty()) mcpp::include_dir(d.files.include_dir.c_str()); + d.ok = true; + return d; +} + +} // namespace mcpp::plugins diff --git a/src/plugins.cppm b/src/plugins.cppm index 344df85..7da9170 100644 --- a/src/plugins.cppm +++ b/src/plugins.cppm @@ -23,7 +23,18 @@ module; export module mcpp.plugins; import std; -import mcpp; + +// NO `import mcpp;`. That module exists only inside a build program, and this +// unit has to compile into an ordinary binary too: `mcpp.tools.embed`'s +// executable form is what lets a generated `.S` be an ACTION's output, which +// is the only way the payload it embeds can be a declared input of the edge +// that assembles it. Measured before this: an ordinary build of this package +// failed with `mcpp: failed to read compiled module`. +// +// Everything this unit used to read from that module -- the target OS, whether +// the toolchain has a GAS assembler, the package's name -- is now a parameter. +// The members under rules/ still import it; they are feature-gated, so the +// tool's own build never compiles them. export namespace mcpp::plugins { @@ -38,7 +49,7 @@ export namespace mcpp::plugins { // // One package, one version: the number lives in mcpp.toml, and the CI step // `the collection states its own version` compares the two. -inline constexpr std::string_view version = "0.3.0"; +inline constexpr std::string_view version = "0.4.0"; } // namespace mcpp::plugins @@ -113,6 +124,19 @@ export namespace mcpp::plugins::surface { // namespaces. Only the file a consumer reaches them through differs. enum class kind { module_, c_header }; +// WHICH HALF OF THE SURFACE A CALL WRITES. +// +// The two are separate because their INPUTS are. The interface is a function of +// the item list: names, namespaces, the accessor declarations. The body is a +// function of the item list AND, under `storage::object`, of the payloads +// themselves -- the assembly names each one in `.incbin`, and the object it +// produces is those bytes. +// +// Declaring them as one action would make a payload change rewrite the +// interface too, and rebuild every BMI that imports it. Declaring them +// separately states what is true and costs a consumer nothing. +enum class half { interface_, body }; + // The element the accessor hands back. `word32` is what an API taking // `const uint32_t*` wants -- `vkCreateShaderModule` is the case this package // has -- and asking for it here is cheaper than a reinterpret_cast at every @@ -213,6 +237,34 @@ struct options { // A short phrase naming what produced the payloads, for the generated // files' first line: "mcpp.rules.spirv". std::string produced_by; + + // ── What this generator is NOT allowed to find out for itself ─────────── + // + // THE TWO FIELDS BELOW ARE WHY THIS MODULE IMPORTS ONLY `std`. + // + // They were `mcpp::target_os()` and `mcpp::compiler()`, read here. That + // made the generator unusable anywhere except inside a build program -- + // which is the one place it must NOT be the only usable form, because a + // payload dependency can only enter the build graph if the generation is + // an ACTION, and an action's command is a binary. A binary cannot import + // the build-program module: measured, an ordinary build of this package + // fails with "mcpp: failed to read compiled module". + // + // So they are parameters. The caller has both answers already, and a + // generator that takes its inputs rather than reading its environment is + // the same code in a build program and in a tool. + + // The target's operating system, as mcpp spells it -- what decides the + // assembly dialect: the section directive and whether a symbol carries a + // leading underscore. Required under `storage::object`; ignored otherwise. + std::string target_os; + + // Whether a GAS-compatible assembler exists for this toolchain. False + // under MSVC, which has none, and where mcpp refuses `.S` outright. It is + // the CALLER's answer because the caller knows the compiler; this + // generator degrades `object` to `header` when it is false and reports + // that through `emitted::store` rather than by printing. + bool has_gas_assembler = true; }; // What the caller hands back to mcpp. @@ -487,19 +539,60 @@ inline std::string assembly_for(std::span items, const options& opt, // ---- the tool --------------------------------------------------------------- -// Writes the interface and the implementation, and returns what the caller has -// to tell mcpp about them. Nothing here reads a payload's bytes, so it runs at -// plan time and works equally for a payload the graph has not produced yet. -inline std::optional emit(std::span items, const options& opt) { - if (items.empty()) return emitted{}; +// EVERY FILE THIS SURFACE WILL PRODUCE, WITHOUT PRODUCING ANY OF THEM. +// +// A rule declares these as an action's outputs, and mcpp requires an output to +// be NAMED before the graph is built even though its content arrives later. So +// the names cannot come from having written the files -- which is the coupling +// that forced generation to happen at plan time, and with it the whole +// staleness this round removes. +// +// Pure: no items, no filesystem, no environment. The one decision it makes is +// the MSVC degradation, which is a property of the toolchain rather than of any +// payload, and `emitted::store` is where the caller reads the verdict. +inline emitted outputs(const options& opt) { + emitted out; + out.store = opt.store; + // MSVC HAS NO GAS, AND mcpp REFUSES `.S` UNDER IT. `src/build/prepare.cppm` + // says so outright: "GAS assembly sources (.S/.s) are not supported by the + // MSVC toolchain". Object storage degrades to header storage there rather + // than producing a file the build will refuse, and the SURFACE does not + // change -- the declarations are identical under both. + if (out.store == storage::object && !opt.has_gas_assembler) + out.store = storage::header; + + const auto dir = std::filesystem::path(opt.out_dir); + out.interface_file = (dir / (opt.module_name + + (opt.surface == kind::module_ ? ".cppm" : ".h"))).string(); + out.impl_file = (dir / (opt.module_name + ".impl.cpp")).string(); + // Named only under the storage that produces one, so a caller can test the + // string rather than having to test the storage a second time. + if (out.store == storage::object) + out.assembly_file = (dir / (opt.module_name + ".payload.S")).string(); + if (opt.surface == kind::module_) out.module_name = opt.module_name; + else out.include_dir = dir.string(); + return out; +} + + +// Writes ONE half of the surface. The names come from `outputs`, which the +// caller has already asked; this produces the content. +// +// Nothing here reads a payload's BYTES. Under object storage the assembly names +// each payload in `.incbin` and the assembler reads it later, which is why this +// can be an action whose command runs before, after or independently of the +// payload's own producer -- what matters is that the payload is a declared +// INPUT of that action, and the graph then holds the edge. +inline bool write(std::span items, const options& opt, half which) { + if (items.empty()) return true; if (opt.module_name.empty()) { std::cerr << "mcpp.plugins.surface: options::module_name is required; it names both " "the module a consumer imports and the namespace the declarations sit in\n"; - return std::nullopt; + return false; } if (opt.out_dir.empty()) { std::cerr << "mcpp.plugins.surface: options::out_dir is required\n"; - return std::nullopt; + return false; } const auto segs = split_module_name(opt.module_name); @@ -515,54 +608,44 @@ inline std::optional emit(std::span items, const options& o "is not a C++ identifier.\n Each segment becomes a namespace, so it has " "to be one; `{}` would work.\n", opt.module_name, segs[i], identifier(segs[i], "part")); - return std::nullopt; + return false; } const auto by = opt.produced_by.empty() ? std::string("mcpp.plugins.surface") : opt.produced_by; - // MSVC HAS NO GAS, AND mcpp REFUSES `.S` UNDER IT. - // - // `src/build/prepare.cppm` states that outright: "GAS assembly sources - // (.S/.s) are not supported by the MSVC toolchain". So object storage - // degrades to header storage there rather than producing a file the build - // will refuse. The SURFACE does not change -- the declarations are the same - // under both -- so a consumer compiled either way is the same source. - emitted out; - out.store = opt.store; - if (out.store == storage::object - && std::string_view(mcpp::compiler()) == "msvc") { - out.store = storage::header; - mcpp::warning("mcpp.plugins.surface: object storage needs a GAS assembler and the " - "MSVC toolchain has none; the payload is compiled in as generated " - "source instead. The declarations a consumer sees are unchanged."); - } - const auto dir = std::filesystem::path(opt.out_dir); - const auto body = declarations(items, opt); + // ONE derivation of the names, shared with the caller. Recomputing them + // here would be the second copy of a decision that this package has paid + // for before. + const emitted out = outputs(opt); + const auto dir = std::filesystem::path(opt.out_dir); + const auto body = declarations(items, opt); // ---- interface ---- - std::string iface; - iface += std::format("// Generated by mcpp.plugins.surface for {}. Do not edit.\n", by); - if (opt.surface == kind::module_) iface += std::format("export module {};\n\n", opt.module_name); - else iface += "#pragma once\n\n"; - iface += extern_c_declarations(items, opt); - iface += "\n"; - // `export` on the opening namespace exports everything the block contains, - // including the nested namespaces a payload's directory produced. - if (opt.surface == kind::module_) iface += "export "; - iface += open_namespaces(segs); - iface += "\n" + body + "\n"; - iface += close_namespaces(segs); - - out.interface_file = (dir / (opt.module_name - + (opt.surface == kind::module_ ? ".cppm" : ".h"))).string(); - if (!write_if_different(out.interface_file, iface)) { - std::cerr << std::format("mcpp.plugins.surface: cannot write {}\n", out.interface_file); - return std::nullopt; + if (which == half::interface_) { + std::string iface; + iface += std::format("// Generated by mcpp.plugins.surface for {}. Do not edit.\n", by); + if (opt.surface == kind::module_) + iface += std::format("export module {};\n\n", opt.module_name); + else + iface += "#pragma once\n\n"; + iface += extern_c_declarations(items, opt); + iface += "\n"; + // `export` on the opening namespace exports everything the block + // contains, including the nested namespaces a payload's directory + // produced. + if (opt.surface == kind::module_) iface += "export "; + iface += open_namespaces(segs); + iface += "\n" + body + "\n"; + iface += close_namespaces(segs); + if (!write_if_different(out.interface_file, iface)) { + std::cerr << std::format("mcpp.plugins.surface: cannot write {}\n", + out.interface_file); + return false; + } + return true; } - if (opt.surface == kind::module_) out.module_name = opt.module_name; - else out.include_dir = dir.string(); - // ---- implementation ---- + // ---- implementation ---- (half::body from here down) // // The one translation unit that defines the accessors. A plain `.cpp` under // every surface and every storage, because the accessors have C language @@ -673,22 +756,50 @@ blob& load(const char* path, blob& b) {{ } } - out.impl_file = (dir / (opt.module_name + ".impl.cpp")).string(); if (!write_if_different(out.impl_file, impl)) { std::cerr << std::format("mcpp.plugins.surface: cannot write {}\n", out.impl_file); - return std::nullopt; + return false; } if (out.store == storage::object) { - out.assembly_file = (dir / (opt.module_name + ".payload.S")).string(); + // A CALLER THAT ASKS FOR THIS STORAGE MUST HAVE SAID WHERE THE BYTES + // ARE, AND THE REFUSAL IS WHAT KEEPS THAT TRUE. + // + // `item::payload_path` is documented as required under `object`, and + // today only `mcpp.rules.spirv` reaches this storage and only it sets + // the field. That is a COINCIDENCE, not a guarantee: the first caller + // to give `rules-slang` or `tools-embed` a storage option would get an + // `.incbin ""` in a generated `.S`, and the declaration below would + // then name an empty dependency -- so the build would be wrong in + // exactly the silent way this whole change exists to remove. + // + // A constraint written only in a comment has nothing enforcing it, + // which is a shape this project has recorded before. This is the + // enforcement. + for (auto const& it : items) { + if (!it.payload_path.empty()) continue; + std::cerr << std::format( + "mcpp.plugins.surface: `{}` was given storage::object with no " + "payload_path.\n" + " Under this storage the generated assembly names the payload " + "in `.incbin`, so\n" + " the caller has to say where it is. Set `item::payload_path`, " + "or use storage::header.\n", + it.identifier); + return false; + } if (!write_if_different(out.assembly_file, - assembly_for(items, opt, mcpp::target_os()))) { + assembly_for(items, opt, opt.target_os))) { std::cerr << std::format("mcpp.plugins.surface: cannot write {}\n", out.assembly_file); - return std::nullopt; + return false; } } - return out; + // `storage::sidecar` needs no counterpart, and that was checked rather than + // assumed. Its payload is never read by a compile: the accessor opens the + // file at RUN time, so a rebuilt payload is picked up by the next run with + // nothing in the build graph to keep current. + return true; } // The surface a project gets when it asks for nothing. @@ -707,25 +818,29 @@ inline kind default_surface() { // `example:my-app` -> `my_app`. The module a rule names by default is this // followed by the group's own segment, so two packages in one build cannot // claim the same module. -inline std::string module_root_from_package() { - // THE PACKAGE'S NAME. NOT ITS DIRECTORY'S. - // - // These are different questions and they give different answers whenever a - // project lays a package out under a generic directory. The first version - // of this function asked the only question mcpp could answer -- the leaf of - // MCPP_MANIFEST_DIR -- so a package named `vulkan-saxpy` laid out as - // `vulkan/app/` generated `app.shaders`, and every `/app/` in a - // workspace claimed that same module. `mcpp::package_name()` was added to - // the build-program contract in mcpp 2026.9.7.1, which this package - // requires, so the right question is now askable. - // - // The directory leaf remains only as a value for the impossible case: a - // manifest without a `[package] name` does not load, so an empty answer - // here would mean the contract changed underneath. A rule that wants - // neither passes `options::module_name`, and this is not consulted. - std::string leaf{mcpp::package_name()}; - if (leaf.empty()) - leaf = std::filesystem::path(mcpp::manifest_dir()).filename().string(); +// The default module root for a package that named none: its PACKAGE NAME, +// sanitised into an identifier. +// +// THE PACKAGE'S NAME. NOT ITS DIRECTORY'S. These are different questions and +// they give different answers whenever a project lays a package out under a +// generic directory. The first version asked the only question mcpp could +// answer -- the leaf of MCPP_MANIFEST_DIR -- so a package named `vulkan-saxpy` +// laid out as `vulkan/app/` generated `app.shaders`, and every +// `/app/` in a workspace claimed that same module. +// +// The name ARRIVES rather than being read, for the reason `options::target_os` +// records: this module must compile into a plain binary as well as into a +// build program, so it may not reach for `mcpp::package_name()` itself. The +// caller passes what mcpp told it. +// +// `fallback` covers the impossible case -- a manifest without a `[package] +// name` does not load, so an empty first argument would mean the contract +// changed underneath. A rule that wants neither passes `options::module_name`, +// and this is not consulted. +inline std::string module_root_for(std::string_view package_name, + std::string_view fallback = {}) { + std::string leaf{package_name}; + if (leaf.empty()) leaf = std::string(fallback); return identifier(leaf, "app"); } diff --git a/tests/spirv-object-storage/mcpp.toml b/tests/spirv-object-storage/mcpp.toml index 4894d83..ad53d8a 100644 --- a/tests/spirv-object-storage/mcpp.toml +++ b/tests/spirv-object-storage/mcpp.toml @@ -23,7 +23,13 @@ import_std = true # docs/05 section 2.6.1 exists for. `host-module = true` says which build-time # product is wanted; the section says whether the package reaches the target. [build-dependencies.mcpp] -plugins = { path = "../..", features = ["rules-spirv"], host-module = true } +# `tools = [...]` and `host-module = true` are two axes of ONE edge, and this +# fixture needs both. Object storage generates its assembly through an ACTION +# so that the payloads can be that action's declared inputs, and an action's +# command is a program -- `mcpp-embed`, built from this same package, so its +# version cannot drift from the declarations it writes. The default storage +# needs no program and no `tools`. +plugins = { path = "../..", features = ["rules-spirv"], host-module = true, tools = ["mcpp-embed"] } # NO [xlings.workspace]. The rule declares the payloads it drives, under the # feature that selects it and the accelerator it serves, so this project names diff --git a/tools/embed.cppm b/tools/embed.cppm index 4e7a85a..92d1f9e 100644 --- a/tools/embed.cppm +++ b/tools/embed.cppm @@ -36,6 +36,7 @@ import mcpp; // embedded by this tool and a set of shaders compiled by `mcpp.rules.spirv` // reach a consumer through the same shape. import mcpp.plugins; +import mcpp.plugins.declare; // WHY NOTHING HERE USES `std::println`, AND WHY THAT IS NOT A STYLE CHOICE. @@ -288,13 +289,13 @@ inline bool group(std::span inputs, so.module_name = module_name; so.out_dir = dir; so.produced_by = "mcpp.tools.embed"; + // Answered here, not read there: `mcpp.plugins.surface` compiles into a + // plain binary as well as into this build program, so it takes its inputs. + so.target_os = mcpp::target_os(); + so.has_gas_assembler = std::string_view(mcpp::compiler()) != "msvc"; - const auto out = mcpp::plugins::surface::emit(items, so); - if (!out) return false; - mcpp::generated(out->interface_file.c_str()); - mcpp::generated(out->impl_file.c_str()); - if (!out->include_dir.empty()) mcpp::include_dir(out->include_dir.c_str()); - return true; + const auto out = mcpp::plugins::surface_for(items, so); + return out.ok; } } // namespace mcpp::tools::embed diff --git a/tools/embed_main.cpp b/tools/embed_main.cpp new file mode 100644 index 0000000..65eb2a4 --- /dev/null +++ b/tools/embed_main.cpp @@ -0,0 +1,132 @@ +// mcpp-embed -- the surface generator, as a program the BUILD GRAPH invokes. +// +// WHY THIS EXISTS AT ALL, AND WHY IT IS SO SMALL. +// +// `mcpp.plugins.surface` writes three files: the module interface, the +// implementation, and -- under `storage::object` -- an assembly source that +// names each payload in `.incbin`. Called from `build.mcpp`, all three are +// written at PLAN time, and that is correct for two of them: their content is +// a function of the item list, which the rule already knows. +// +// It is not correct for the third. The OBJECT the assembly produces is a +// function of the payload's BYTES, and those do not exist when build.mcpp +// runs -- the shader compiler has not been invoked yet. A file written at plan +// time cannot be an edge to a build-time product, so the object was assembled +// once and every later change to a payload was a green build over stale bytes. +// Measured against `mcpp:plugins` 0.3.0: editing a shader left the program +// reporting the previous payload's byte count. +// +// The fix is not a new channel for stating that edge. It is to stop writing +// the file at the wrong time. An `mcpp::action` declares its inputs, and a +// payload named as an input of the action that writes the assembly IS the +// edge -- expressed with the one graph primitive the engine has, and needing +// nothing added to it. Prototyped before this was written: the artifact +// followed the payload with no engine change at all. +// +// So this program exists to be an action's COMMAND. It holds no logic of its +// own: `mcpp.plugins.surface` decides every name, and this reads what the rule +// wrote down and calls it. The two forms cannot drift, because they are the +// same code -- which is why that module imports only `std`. +#include +#include + +import std; +import mcpp.plugins; + +namespace { + +// The manifest the rule writes at plan time. Deliberately a flat line-based +// format rather than JSON: the whole vocabulary is below, both ends are in +// this package, and a parser is a place for a defect that a `find`/`substr` +// pair is not. +// +// surface module|c_header +// storage header|object|sidecar +// element byte|word32 +// module +// outdir +// by +// os +// gas 0|1 +// emit interface|body (which half this invocation writes) +// item +// ns (repeats, applies to the current item) +// header +// symbol +// payload +// sidecar +// size +struct parsed { + mcpp::plugins::surface::options opt; + std::vector items; + std::string emit; // "interface" | "body" +}; + +std::optional read_manifest(const std::string& path) { + std::ifstream in(path); + if (!in) { + std::cerr << std::format("mcpp-embed: cannot read {}\n", path); + return std::nullopt; + } + parsed p; + std::string line; + while (std::getline(in, line)) { + if (line.empty() || line[0] == '#') continue; + auto sp = line.find(' '); + auto key = line.substr(0, sp); + auto val = sp == std::string::npos ? std::string() : line.substr(sp + 1); + namespace sf = mcpp::plugins::surface; + if (key == "surface") p.opt.surface = val == "c_header" ? sf::kind::c_header + : sf::kind::module_; + else if (key == "storage") p.opt.store = val == "object" ? sf::storage::object + : val == "sidecar" ? sf::storage::sidecar + : sf::storage::header; + else if (key == "element") p.opt.elem = val == "byte" ? sf::element::byte_ + : sf::element::word32; + else if (key == "module") p.opt.module_name = val; + else if (key == "outdir") p.opt.out_dir = val; + else if (key == "by") p.opt.produced_by = val; + else if (key == "os") p.opt.target_os = val; + else if (key == "gas") p.opt.has_gas_assembler = val != "0"; + else if (key == "emit") p.emit = val; + else if (key == "item") p.items.push_back({ .identifier = val }); + else if (p.items.empty()) { + std::cerr << std::format("mcpp-embed: `{}` before any `item`\n", key); + return std::nullopt; + } + else if (key == "ns") p.items.back().name_space.push_back(val); + else if (key == "header") p.items.back().data_header = val; + else if (key == "symbol") p.items.back().data_symbol = val; + else if (key == "payload") p.items.back().payload_path = val; + else if (key == "sidecar") p.items.back().sidecar_name = val; + else if (key == "size") p.items.back().data_size_expr = val; + else { + // Refused, not ignored: an unknown key means the rule and the tool + // disagree about the vocabulary, and they ship in one package at + // one version, so that can only be a defect. + std::cerr << std::format("mcpp-embed: unknown key `{}`\n", key); + return std::nullopt; + } + } + return p; +} + +} // namespace + +int main(int argc, char** argv) { + if (argc != 2) { + std::cerr << "usage: mcpp-embed \n"; + return 2; + } + auto p = read_manifest(argv[1]); + if (!p) return 1; + namespace sf = mcpp::plugins::surface; + if (p->emit != "interface" && p->emit != "body") { + std::cerr << std::format("mcpp-embed: `emit` must be interface or body, got `{}`\n", + p->emit); + return 2; + } + const bool ok = sf::write(p->items, p->opt, + p->emit == "interface" ? sf::half::interface_ : sf::half::body); + return ok ? 0 : 1; +} From f90b77185a097e274dc460fedadc26ae1f6ee1e7 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Tue, 8 Sep 2026 09:45:32 +0800 Subject: [PATCH 2/3] CI: the all-rules denominator counts members, not internal features MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 这一步比较包里每一个 `[features.]` 与夹具激活的那一列,好让第七个成员不能被悄悄漏出 「every rule module compiles for this host」。`[features.surface]` 让它红了,而且红得对: 那是一个夹具没点名的新 feature。 但它不是成员。它承载 surface 的构建程序那一半,每个成员都 implies 它,消费者永远不写它。 把它加进夹具是错的修法 —— 夹具那一列的含义是「消费者能激活的成员」,为了让检查变绿而往里 填东西,只会让检查的含义变少。 所以分母减去「被别的 feature implies 的」。这条规则从 manifest 推出来而不是列在这里,正是 这个检查原本就有的性质:第七个**成员**仍会被抓到,因为没有东西 implies 它。 两条腿都验过:修正后两列一致;临时加一个新成员,它出现在分母里并会让这一步失败。 --- .github/workflows/ci.yml | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 3f15e6b..1d1a307 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -752,8 +752,25 @@ jobs: # ` = { ... }`. Reading keys out of the table BODY matched # `sources` once per feature and reported the fixture as # incomplete while the extractor was what broke. - feats=$(grep -oE '^\[features\.[a-z0-9-]+\]' mcpp.toml \ - | sed 's/^\[features\.//; s/\]$//' | sort) + # MEMBERS, NOT EVERY FEATURE. A feature that another feature + # IMPLIES is internal -- `surface` carries the build-program half of + # the generated surface and every member implies it, and a consumer + # never writes it. Subtracting the implied ones keeps this check + # meaning "the fixture names every member a consumer can activate"; + # padding the fixture instead would make it mean less. + # + # The rule is read out of the manifest, not listed here, so a seventh + # MEMBER is still caught: nothing implies it. + # A TEMP FILE, NOT `<(...)`. This step also runs on windows-2022 + # through Git Bash, where process substitution is emulated and not + # dependable -- and a check that behaves differently on one of the + # three hosts is the exact class of difference this job exists to + # catch, so it must not introduce one. + grep -oE '^implies[[:space:]]*=.*' mcpp.toml \ + | grep -oE '"[a-z0-9-]+"' | tr -d '"' | sort -u > /tmp/implied.txt + grep -oE '^\[features\.[a-z0-9-]+\]' mcpp.toml \ + | sed 's/^\[features\.//; s/\]$//' | sort > /tmp/allfeats.txt + feats=$(comm -23 /tmp/allfeats.txt /tmp/implied.txt) used=$(sed -n '/features = \[/,/\], host-module/p' tests/all-rules-compile/mcpp.toml \ | grep -oE '"[a-z-]+"' | tr -d '"' | sort) [ -n "$feats" ] || { From 853dd8739d56af508176d48b36669f949aa3aa55 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Tue, 8 Sep 2026 10:06:33 +0800 Subject: [PATCH 3/3] CI: the denominator fix belonged in both jobs, not the first one found MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 「every rule module compiles for this host」这一步在 `consumers` 与 `rules-cross-platform` 里各有一份。上一次只改了第一份 —— 而那个 job 本来就是绿的; 红的是另一个。 **做这次替换的脚本在锚点「存在」时就放行了。存在不等于唯一,而这个差别就是缺陷本身:** 它验证了自己将要改的东西在,没验证它是唯一的一处。现在按**出现次数**断言,并在写回前 再数一次两个 job 是否都拿到了修好的形状。 顺带把 `<(...)` 换成临时文件:这一步也在 windows-2022 的 Git Bash 上跑,进程替换在那里 是模拟的、不可依赖 —— 而「一个在三台宿主上行为不同的检查」正是这个 job 存在的理由所要 抓的东西,它自己不能引入一个。 --- .github/workflows/ci.yml | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 1d1a307..ba57cdf 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -1066,8 +1066,23 @@ jobs: # ` = { ... }`. Reading keys out of the table BODY matched # `sources` once per feature and reported the fixture as # incomplete while the extractor was what broke. - feats=$(grep -oE '^\[features\.[a-z0-9-]+\]' mcpp.toml \ - | sed 's/^\[features\.//; s/\]$//' | sort) + # MEMBERS, NOT EVERY FEATURE. A feature that another feature + # IMPLIES is internal -- `surface` carries the build-program half of + # the generated surface and every member implies it, and a consumer + # never writes it. Subtracting the implied ones keeps this check + # meaning "the fixture names every member a consumer can activate"; + # padding the fixture instead would make it mean less. The rule is + # read out of the manifest, so a seventh MEMBER is still caught. + # + # A temp file, not `<(...)`: this step also runs on windows-2022 + # through Git Bash, where process substitution is emulated and not + # dependable -- and a check that behaves differently on one of the + # three hosts is the class of difference this job exists to catch. + grep -oE '^implies[[:space:]]*=.*' mcpp.toml \ + | grep -oE '"[a-z0-9-]+"' | tr -d '"' | sort -u > /tmp/implied.txt + grep -oE '^\[features\.[a-z0-9-]+\]' mcpp.toml \ + | sed 's/^\[features\.//; s/\]$//' | sort > /tmp/allfeats.txt + feats=$(comm -23 /tmp/allfeats.txt /tmp/implied.txt) used=$(sed -n '/features = \[/,/\], host-module/p' tests/all-rules-compile/mcpp.toml \ | grep -oE '"[a-z-]+"' | tr -d '"' | sort) [ -n "$feats" ] || {