From c04dd06c67ed51deb61751b032ae3a73c81963be Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Fri, 11 Sep 2026 01:59:03 +0800 Subject: [PATCH 1/5] feat(pack): `--format ` dispatches to a package, and no distribution format lives in the engine (2026.9.11.1) `tar` and `dir` answer the same question `msi` and `appimage` answer -- what shape does the output take -- so they are values of one flag rather than the beginning of a second one. The split that keeps every other format out of the engine is: `mcpp pack` owns the mechanism and the one universal format, and every other format lives in a package that `mcpp pack` dispatches to. The universal format is what it already produces: an archive that extracts and runs, universal in the only sense that matters here -- it needs no knowledge of anyone else's release. Everything past it does. dpkg's control fields, AppImage's runtime, WiX's schema, Apple's notarisation, Android's signing scheme: each one bound into the engine couples an mcpp release to a release mcpp does not control. The project already made this argument for languages, where Slang is supported without being named in the engine, and a distribution format has less claim to a name in the engine than a language does. Three additions, each FORMAT-NEUTRAL, which is the test for whether something belongs in the engine at all: - A staged tree an artifact action can consume. `mcpp pack` already computes one -- the dependency closure after the strip policy, the debug-symbol split and `include`/`exclude` -- and then compressed it and the directory was gone, so a `.deb`, an AppImage, a `.app` and an `.msi` each had to rebuild it. `${mcpp.stage_dir}` exposes it. - The rest of `[package]` in the build program: `MCPP_PKG_VERSION` / `_DESCRIPTION` / `_LICENSE` / `_AUTHORS` / `_REPO` and the matching `mcpp::package_*()`. Every installer states a version; without these a project restates it in the member's own options, where the copy drifts from `[package]` with nothing able to detect it. - `--format` resolving its value through the graph. A package declares with `mcpp::provides_pack_format("")`; `--format ` finds the provider among the resolved dependencies. The refusal for an unknown value names what IS available rather than a constant, and arrives before anything is compiled. DECLARE UNCONDITIONALLY, SUBMIT CONDITIONALLY. This is the load-bearing rule of the dispatch and the one a member author is most likely to get wrong, because a member that gets it wrong still works for whoever wrote it -- they always pass their own format. The declaration must not be gated, or the engine can never answer "which formats does this graph provide"; the submission must be, or a plain `mcpp build` grows an edge it must not have. A format nothing submitted for is refused by name rather than reported as a pack that produced no package. `mcpp pack --format ` PREPARES TWICE, AND NOTHING IS RE-DERIVED BETWEEN THE PASSES. An artifact action is a ninja edge and the staged tree is produced after the link, so the tree cannot be an input of the pass that built it. The first pass collects declarations and refuses an unknown format; the build and the staging follow; the second pass sets `pack_format` and `pack_stage_dir` and builds the submitted edge. Its triple and staged path come from what the first pass and `make_plan` already answered -- `stagingRoot` is a function of the resolved triple, and a second derivation of it before prepare is the shape where two answers agree on every machine the author has. build.ninja's header line gains a fourth field, `dist=`, and the fast paths require it to read `none`. The format is deliberately NOT in the fingerprint -- putting it there would cost a full recompile to package an already-built tree -- so the two graphs share a directory, and `target///build.ninja` is shared mutable state two fast paths replay. That is the third instance of the failure `graph=` and `accel=` each already record. The criterion is a unit test rather than an end-to-end assertion: measured on 2026-09-11, a plain build after the pack pass regenerates the graph even with the field ignored, so an end-to-end check would pass whether or not the field works and would keep passing if it were deleted. `${mcpp.stage_dir}` REFUSES rather than expanding to nothing, in two places: a build that is not packaging, and a role other than `artifact`. An empty path is still a token the command accepts, and the tool then reads the build directory root, which exists -- so the mistake produces a plausible artifact instead of a diagnostic. The measured prototype is a valid, empty, 52 KB installer with nothing said about it. An action that names the placeholder automatically gains a dependency on `.stage-manifest` -- a sibling, never a member, so it never travels inside anyone's installer. The engine adds it because the use implies it: without it the edge is dirty only when a link output changes, and a closure that grew a dependency's shared library while the program's own bytes did not would leave the previous distributable in place, reported as up to date. Build-program protocol v9 (`mcpp:pack-format=`). The row carries a non-empty `tag`, so the declaration is replayed from the build program's cache record -- the pass that reads the set is `mcpp pack`, which is never a project's first build, and an unpersisted declaration would be absent exactly when a user names a format. Tests: 6 unit tests for the staged-tree contract, the graph-shape field in both of its two readers, and the directive row's persistence; one e2e holding the four properties of the dispatch, each with the wrong answer it excludes. Docs: `docs/10` for the `--format` axis, `docs/30` for the three-category taxonomy and the new placeholders and accessors, `docs/31` for the six constraints a distribution member owes its consumer. Both languages. --- ...tion-plugins-and-platform-decomposition.md | 801 ++++++++++++++++++ .agents/docs/README.md | 7 +- CHANGELOG.md | 65 ++ docs/10-pack-and-release.md | 55 +- docs/30-build-mcpp.md | 113 ++- docs/31-authoring-a-rule-package.md | 55 ++ docs/zh/10-pack-and-release.md | 46 +- docs/zh/30-build-mcpp.md | 90 +- docs/zh/31-authoring-a-rule-package.md | 44 + mcpp.toml | 2 +- modules/buildmcpp/src/directives.cppm | 38 +- modules/buildmcpp/src/program_protocol.cppm | 7 +- modules/manifest/src/types.cppm | 21 + modules/versioning/src/version.cppm | 2 +- src/build/build_program.cppm | 48 ++ src/build/graph_shape.cppm | 53 +- src/build/hostprogram.cppm | 74 ++ src/build/ninja_backend.cppm | 3 +- src/build/plan.cppm | 14 + src/build/prepare.cppm | 156 +++- src/cli.cppm | 8 +- src/cli/cmd_publish.cppm | 35 +- src/pack/pack.cppm | 24 +- src/pack/pipeline.cppm | 163 +++- src/pack/stage_tree.cppm | 125 +++ tests/e2e/638_pack_format_dispatch.sh | 245 ++++++ tests/unit/test_build_directives.cpp | 41 +- tests/unit/test_graph_shape.cpp | 53 +- tests/unit/test_loader_contract.cpp | 21 +- tests/unit/test_pack_stage_tree.cpp | 141 +++ 30 files changed, 2496 insertions(+), 54 deletions(-) create mode 100644 .agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md create mode 100644 src/pack/stage_tree.cppm create mode 100755 tests/e2e/638_pack_format_dispatch.sh create mode 100644 tests/unit/test_pack_stage_tree.cpp diff --git a/.agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md b/.agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md new file mode 100644 index 000000000..a50896e96 --- /dev/null +++ b/.agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md @@ -0,0 +1,801 @@ +--- +subject: plugins +status: active +--- + +# The category the plugin taxonomy does not name, and what a platform actually decomposes into + +Date: 2026-09-11. Base: mcpp `2d102817`, `mcpp-plugins` `b4f7590` (0.2.6). + +**How to read this document.** Sections 1 to 3 are analysis of the base commits +above and can be checked against them. Sections 4 to 7 are a proposal and are +not implemented. Section 8 is what the proposal does not solve. + +The thesis is one sentence: **the engine should own the mechanism by which a +distributable is produced, and no format should live in the engine at all.** + +The occasion is a framework outside this project — HuxerUI, a declarative UI +library with a CMake build and a six-platform reach — being ported to mcpp, and +its ecosystem library `Lib-Live2D` being examined as the case that a package +manager either serves or does not. Two things came out of it that are about +mcpp rather than about that framework, and this document is those two. + +## 1. What is already true + +`mcpp:plugins` 0.2.6 is one package whose members are selected by features: + +```toml +[features] +default = [] +rules-ascendc = { sources = ["rules/ascendc.cppm"] } +rules-cuda = { sources = ["rules/cuda.cppm"] } +rules-hip = { sources = ["rules/hip.cppm"] } +rules-spirv = { sources = ["rules/spirv.cppm"] } +rules-sycl = { sources = ["rules/sycl.cppm"] } +tools-embed = { sources = ["tools/embed.cppm"] } +``` + +The taxonomy behind the two prefixes is stated in `docs/30-build-mcpp.md:789`: + +> **A tool is not a rule.** A rule states how a translation unit is compiled by +> a compiler mcpp does not drive: it submits an action and the engine schedules +> it. A tool states something the build program needs that no compiler +> performs, and does it while the program runs. + +And the reserved prefix, from the same file: + +> `warning: build rule 'mcpplibs.plugins' declares the module 'mcpp.rules.spirv'; +> the 'mcpp.' prefix is reserved for rules maintained by the mcpp project.` + +Five extension points do the extending, and `docs/31-authoring-a-rule-package.md` +says what they buy: **the engine holds no name that comes through any of them**. +Slang is the evidence — "the first language mcpp supports without naming it in +the engine". + +## 2. The category the taxonomy does not name + +A third kind of work fits neither definition. It does not compile a translation +unit, so it is not a rule; and it does not run while the build program runs, so +it is not a tool. It consumes **link outputs** and produces something a user +installs: + +| Work | Compiles a TU? | Runs in the build program? | +|---|---|---| +| SPIR-V from a shader | yes | no | +| A data file into a header | no | yes | +| **An MSI from the linked program** | **no** | **no** | +| **codesign on a `.app`** | **no** | **no** | +| **A `.deb`, an AppImage, an `.apk`** | **no** | **no** | + +The engine already has the mechanism: `role = "artifact"`, whose inputs are link +outputs, so ninja sequences it after the link. `docs/30-build-mcpp.md:464` names +the intended uses — "codesign, packaging, size budgets". What is missing is a +name for the packages that ship such actions, and therefore a place for them. + +**This is not hypothetical.** The HuxerUI port implements a Windows installer +this way and it is green on Windows CI: one `role = "artifact"` action running +`wix build`, with the program passed as `-d Executable=${mcpp.target_file:}` +and the definition naming it with ``. The +resulting MSI is 1572 KB and its `File` table carries one row, the 6.9 MB +application. Nothing in that action is specific to a UI framework except a +default icon path. + +An earlier revision of the same action bound a directory (`-bindpath +Application=bin`) and harvested it. When the path resolved to nothing on +Windows, WiX produced a **valid, empty, 52 KB installer with no diagnostic**. +That failure is worth recording because it is the argument for a shared package +rather than a per-project action: the mistake is not obvious, it is silent, and +every project that writes its own installer step gets to make it once. + +### 2.1 `mcpp pack` keeps the universal format and dispatches the rest + +`mcpp pack` is the right home, and the question is which half of it is the +engine's. `docs/10-pack-and-release.md:389` documents `[pack]` as declarative +configuration over a fixed set of modes (`static`, `bundle-project`, +`bundle-all`, `system`), and lists `.deb`, `.rpm` and AppImage under **Planned +Support**. + +Those three should not land there, and the split that keeps them out is: + +> **`mcpp pack` owns the mechanism and the one universal format. Every other +> format lives in a package, and `mcpp pack` dispatches to it.** + +The universal format is what it already produces: an archive that extracts and +runs. It is universal in the only sense that matters here — it needs no +knowledge of anyone else's release. Everything past it does. dpkg's control +fields, AppImage's runtime, WiX's schema, Apple's notarisation, Android's +signing scheme v3: each one bound into the engine couples an mcpp release to a +release mcpp does not control. The project already made this argument for +languages — Slang is supported "without naming it in the engine" — and a +distribution format has less claim to a name in the engine than a language +does. + +Dispatch is what keeps this from fragmenting into "packaging is a thing you do +outside `mcpp pack`" — and **the flag for it already exists**: + +``` +mcpp pack --format tar (default; .zip for a Windows target) | dir +``` + +`tar` and `dir` answer the same question `msi` and `appimage` answer — what +shape does the output take — so they belong on one axis, and the proposal is +not a new flag but a wider set of values for this one: + +```bash +mcpp pack --format msi +mcpp pack --format appimage +``` + +exactly as `--target` reaches a triple the engine did not have to know about +individually. Two consequences follow from the values no longer being a fixed +list: `--help` says "plus any format the resolved graph provides", and an +unknown value names what *is* available rather than a constant. + +### 2.2 What the engine must expose for that to work + +Three additions, and each is **format-neutral** — which is the test for whether +something belongs in the engine at all. + +**(a) A staged tree an artifact action can consume.** `mcpp pack` already +computes one: the dependency closure, the strip policy, the debug-symbol split, +`include`/`exclude`. It then compresses it and the directory is gone. A `.deb`, +an AppImage, a `.app` and an `.msi` all want exactly that directory and today +must each rebuild it. Exposing it — a `${mcpp.stage_dir}` placeholder, or a +`--stage-only` mode whose output path an action can name — is one addition that +serves every format, present and future, and encodes no format's knowledge. + +**(b) Package metadata in the build program.** A build program is told +`package_name()` and `package_namespace()` and not the version, description, +license or authors. Every installer needs the version; the HuxerUI +implementation therefore asks the project to restate it in the rule's options, +where it can drift from `[package] version` with nothing to detect it. This is +a read of data mcpp already parsed. + +**(c) `--format` resolves its value through the graph.** A package declares that +it provides a format name; `--format ` finds the provider among the +resolved dependencies and hands it the staged tree. The engine holds the +*dispatch*, not the format — the same shape as `device_extensions`, where the +engine classifies a source it cannot compile and a package supplies the +compiler. This is the smallest of the three additions, because the flag, its +parsing and its position in the command are already there. + +With (a) to (c), `[pack]`'s built-in modes become one provider among several +rather than the model, and nothing new needs to join them. + +## 3. Where a platform actually decomposes + +The second question was whether Android, iOS and Web can be reached from +packages. The answer differs per platform, and the difference is not a matter +of degree. + +A target is **not** extensible from a package. `modules/toolchain-model/src/triple.cppm` +declares the identity as three strings — + +```cpp +struct Triple { + std::string arch; // "x86_64" | "aarch64" | "riscv64" | ... + std::string os; // "linux" | "macos" | "windows" + std::string env; // "gnu" | "musl" | "msvc" | "" +}; +``` + +— with `os == "none"` added for freestanding, and a **23-row +`kKnownTargets` table compiled into the binary**: + +```cpp +{ "x86_64-windows-msvc", "verified", "PE", "", "", false }, +{ "aarch64-macos", "verified", "", "", "", false }, +{ "aarch64-linux-gnu", "planned", "", "", "", false }, +{ "riscv64-none-elf", "verified", "bare", "llvm@22.1.8", "xim:picolibc-riscv@1.8.12", true }, +``` + +Bare-metal targets are **rows in that table**, not something a board-support +package introduced; a BSP supplies the runtime for an already-known target. So +the engine boundary is exact: a package can add a language, a tool, an action, +a payload and a generated module. It cannot add a triple. + +There is no separate object-format model to extend either, and that is the +sharper form of the same fact. The binary format is not a field; it is derived +from `os` at each site that needs it — `is_pe()` is `os == "windows"`, +`is_freestanding()` is `os == "none"`, and `family()`, the `cfg()` dimension, +answers `windows` or `unix` and **nothing at all** for anything else. A fourth +answer to "what does this produce" is therefore not one +addition but an addition at every such site, which is why wasm is a different +size of change from a table row. (`nasm_format()` looks like an object-format +switch and is not one: it is NASM's `-f` flag, x86-only by construction, and it +hard-errors off x86 rather than choosing.) + +With that boundary fixed, each platform decomposes into layers, and only the +first is engine work: + +| Layer | Android | iOS | Web | Owner | +|---|---|---|---|---| +| **Toolchain does modules** | **yes, with a supplied surface + one define** (3.1) | not measured | **yes, with a supplied surface** (3.1) | **prerequisite, and answered for two** | +| Target identity | `env = "android"`, one row | `os = "ios"`, one row | **new arch, new os, new object format** | **engine** | +| Toolchain payload | `xim:android-ndk` | `xim:iphoneos-sdk` | `xim:emsdk` | index | +| Sysroot | table column, or `[target.].sysroot` | same | same | engine / manifest | +| Compile and link flags | `-target aarch64-linux-android` | `-miphoneos-version-min` | `-sUSE_WEBGL2` etc. | **plugin** | +| Packaging | `.apk` | `.app` | `.html` + `.wasm` + `.js` | **plugin** | +| Signing | `apksigner` | `codesign` | — | **plugin** | +| Running | `adb install` | simulator | a browser | **plugin** (`mcpp::runner`) | +| Non-C++ glue | Gradle | Xcode project | JS bridge | **plugin** | + +The distances are unequal, and the ranking is the opposite of the usual demand +ranking: + +- **Android is the smallest.** `aarch64-linux-gnu` is already a `planned` row, + ELF is already an object format, and `aarch64` is already an arch. What is + missing is an `env` value and a sysroot that points at an NDK. +- **iOS is next.** `aarch64-macos` is `verified`, so Mach-O and the Apple half + of the toolchain model exist. What is missing is an `os` value and the + iPhoneOS SDK. +- **Web is the outlier.** A new arch (`wasm32`), a new os, a new object format, + and a driver that is a wrapper rather than a compiler mcpp drives directly. + This is [#597](https://github.com/mcpp-community/mcpp/issues/597), and it is + the only one of the three that changes the target model rather than extending + a table. + +**The consequence for planning:** every layer below the first is the same +mechanism on all three platforms. A `.apk` step, a `.app` step and an +`.html`+`.wasm` step are three members of the category section 2 names, and +none of them waits on the engine to be *written* — only to be *useful*. + +## 3.1 The prerequisite nobody lists: does the platform's toolchain do modules? + +Section 3 treats the target row as the engine's part of the work. That is true +and incomplete. mcpp is module-first — `import std` availability is one of the +eleven fingerprint inputs, and a package's interface is a BMI — so a target row +for a toolchain that cannot compile a module interface unit would resolve, build +nothing, and be worse than its absence. + +This was measured on this machine rather than assumed, and the result is better +than the shipped state of either toolchain suggests: **`import std` works on +both Android and Emscripten today, and neither needs a fork or a compiler +upgrade.** What both need is a directory their vendor chose not to install. + +### The gap, stated once + +An LLVM installation that supports `import std` carries a generated module +surface beside its headers. Neither vendor ships it: + +| | `xim:llvm` 20.1.7 | NDK r27 | Emscripten 4.0.19 | +|---|---|---|---| +| clang | 20.1.7 | 18.0.1 | 22.0.0git | +| `_LIBCPP_VERSION` | 200100 | 180000 | **200100** | +| `_LIBCPP_ABI_NAMESPACE` | `__1` | `__ndk1` | `__2` | +| `share/libc++/v1/std.cppm` | yes | — | — | +| `std/*.inc` + `std.compat/*.inc` | 110 + 21 | 0 | 0 | +| total | 133 files, 620 KB | **0** | **0** | + +The NDK additionally sets `_LIBCPP_HAS_NO_STD_MODULES` in its `__config_site`. +That macro appears **exactly once in the whole NDK — in `__config_site` +itself**; no header consults it. It disabled the *installation* of the module +files at libc++ build time and gates nothing in the library, which is what makes +supplying them legitimate rather than a workaround around a disabled feature. +Emscripten does not set it at all. + +### Why "just upgrade libc++ to 22" is not the answer + +Tried, and it fails for a reason worth recording. libc++'s headers are **not +portable across configurations**: `__config_site` is generated per build and +records the ABI, threading and locale decisions that build made. Pointing NDK +clang at llvm 22.1.8's headers gives + +``` +__config:13:10: fatal error: '__config_site' file not found +``` + +and the three `_LIBCPP_ABI_NAMESPACE` values in the table above are the deeper +form of the same fact: `__1`, `__ndk1` and `__2` are deliberately incompatible, +so replacing a vendor's libc++ renames every `std` symbol's ABI namespace. +"Upgrade to 22" therefore means *building* libc++ from source for that target +and accepting a platform ABI change — not a file swap. It is a real option for +a project that already static-links its C++ runtime, and it is not needed, +because matching the revision works. + +### Android: measured, works, one define + +Named modules work as shipped: + +``` +$ clang++ --target=aarch64-linux-android24 -std=c++20 --precompile m.cppm # OK +$ clang++ --target=aarch64-linux-android24 -std=c++20 -fmodule-file=m=m.pcm -c main.cpp # OK +``` + +`import std;` does not — `fatal error: module 'std' not found` — for the reason +the table gives. Supplying the surface is mechanical, because `std.cppm` is +generated: + +``` +// WARNING, this entire header is generated by utils/generate_libcxx_cppm_in.py +``` + +from `libcxx/modules/std.cppm.in` and one CMake substitution that fills +`@LIBCXX_MODULE_STD_INCLUDE_SOURCES@` with `#include` lines naming the +`std/*.inc` partitions. Taking `libcxx/modules/` from **llvmorg-18.1.8** — the +release matching `_LIBCPP_VERSION 180000` — and performing that substitution +reproduces the vendor's own 133 files. + +Compiling it against the NDK's own headers then failed, and the failure is the +interesting part: + +``` +std/cctype.inc:11: error: using declaration referring to 'isalnum' with + internal linkage cannot be exported +bionic ctype.h:127: __BIONIC_CTYPE_INLINE int isalnum(int __ch) { … } +``` + +28 errors, all of that one kind, confined to 2 of the 110 partitions +(`cctype.inc` and `locale.inc`). bionic defines the ctype functions +`static __inline`, and a using-declaration naming an internal-linkage entity +cannot be exported from a module. + +bionic anticipated this. The macro is overridable and says why: + +```c +/* All the functions in this file are trivial … we inline them by default. + * This macro is meant for internal use only, so that we can also provide + * actual symbols for any caller that needs them. */ +#if !defined(__BIONIC_CTYPE_INLINE) +#define __BIONIC_CTYPE_INLINE static __inline +#endif +``` + +With `-D__BIONIC_CTYPE_INLINE=` on the `std.cppm` compile, the surface builds +(27.5 MB BMI) and a program that uses it links: + +```cpp +import std; +int main() { + std::vector v{3,1,2}; + std::ranges::sort(v); + return std::format("{}-{}-{}", v[0], v[1], v[2]) == "1-2-3" ? 0 : 1; +} +``` + +``` +app: ELF 64-bit LSB pie executable, ARM aarch64, interpreter /system/bin/… +``` + +The binary was not executed — no device or emulator here — so this is "compiles +and links", not "runs". + +### Emscripten: measured, works, no define + +Named modules work as shipped. The module surface is absent as it is on +Android, and the version to match is **not** the one the compiler reports: +`em++` is clang 22.0.0git while its libc++ is `_LIBCPP_VERSION 200100`, LLVM +20.1. llvm 22.1.8's surface therefore fails on `'flat_set' file not found`; +llvm 20.1.7's builds with **no additional flags at all** (31 MB BMI). + +End to end, and this one did run: + +```cpp +import std; +int main() { + std::vector v{3,1,2}; + std::ranges::sort(v); + std::print("{}-{}-{}\n", v[0], v[1], v[2]); +} +``` + +``` +$ em++ -std=c++23 -fmodule-file=std=std.pcm -c app.cpp && em++ app.o std.pcm -o app.js +$ node app.js +1-2-3 +``` + +`app.wasm` is 470 KB. So the Web question splits cleanly in two, and only one +half is open: **the standard library story is answered**, and what remains is +[#597](https://github.com/mcpp-community/mcpp/issues/597)'s target model. + +### Apple: not measured, and the recipe may not transfer + +iOS cannot be measured on a Linux host. Unlike the other two, the remedy may not +exist: Apple's libc++ is not a build of a public revision, so there is no +matching `libcxx/modules/` to take a surface from, and the `_LIBCPP_VERSION` +check that makes the other two recipes safe has nothing to compare against. +Two honest possibilities, and which holds is a measurement not taken — Xcode's +toolchain already ships the surface, or iOS waits on Apple. + +### What this means for xim-pkgindex + +The index has **253 recipes and none of the three**: no `android-ndk`, no +`emsdk`, no `iphoneos-sdk`. `llvm` is there and is the shape to copy. + +| Recipe | Beyond the archive it must carry | Blocked on | +|---|---|---| +| `xim:android-ndk` | `share/libc++/v1/` from llvmorg-18.1.8, a `_LIBCPP_VERSION` check, and `-D__BIONIC_CTYPE_INLINE=` on the std BMI | **nothing — measured working** | +| `xim:emsdk` | `share/libc++/v1/` from the release matching *libc++*'s version, not the compiler's | **nothing — measured working** | +| `xim:iphoneos-sdk` | licensing decides whether it installs or merely locates | a licence reading, and one measurement | + +Both writable recipes owe the same two things: **pin both halves to one +revision and refuse on mismatch** — the check is exact, `_LIBCPP_VERSION` in +`__config_site` against the release the surface came from — and **re-derive on +every vendor bump**, since the surface is a function of the vendor's libc++, +not a constant. + +The licensing asymmetry is worth stating plainly. The NDK is Apache-2.0 and +Emscripten is MIT, both redistributable; the iPhoneOS SDK is neither, which is +why cross-platform toolchains reach it through a locally installed Xcode. +`xim:iphoneos-sdk` may therefore have to be a *locator* — a recipe that finds +and pins what the machine already has, the way `msvc@system` does — rather than +an installer. mcpp already has that shape. + +## 4. The members this proposes + +Nothing here is a new repository. `mcpp-plugins` exists and its shape — one +package, one module interface unit per feature — already fits. + +| Feature | Module | Category | What it does | +|---|---|---|---| +| `dist-wix` | `mcpp.dist.wix` | dist | An MSI from a linked program and a definition it renders. Windows only. | +| `dist-appimage` | `mcpp.dist.appimage` | dist | An AppImage from a staged tree. Linux only. | +| `dist-apple` | `mcpp.dist.apple` | dist | A `.app` bundle, `Info.plist`, and `codesign`. macOS now; iOS when the row exists. | +| `dist-android` | `mcpp.dist.android` | dist | An `.apk`: Gradle invocation or direct `aapt2`/`d8`/`apksigner`. When the row exists. | +| `dist-web` | `mcpp.dist.web` | dist | The `.html`/`.js`/`.wasm` set and its loader. When the target model admits wasm. | + +Two of these can be written **today**, against the engine as it is: `dist-wix` +(a working implementation exists and would be a port, not a design) and +`dist-appimage`. `dist-apple` can be written for macOS today and gains iOS when +the row lands. The other two wait on section 3's first layer. + +### Why `dist-` and not `rules-` + +Because the taxonomy in section 1 is load-bearing and these are not rules. A +consumer reading `rules-wix` would expect a compiler it does not drive and a +translation unit; there is neither. The prefix should say which of the three +questions the member answers: + +``` +rules-* how is this translation unit compiled +tools-* what does the build program need to do itself +dist-* what comes out of the link, and in what form a user installs it +``` + +`mcpp.` stays reserved for members of this repository, unchanged, and +`mcpp.build.*` remains the engine's own module family — `mcpp.dist.*` collides +with neither. + +## 5. What `tools-embed` is missing + +`mcpp.tools.embed` covers one file at a time. `files()` writes **one header per +input**, deriving an identifier from each, and refuses `options::identifier` +because it "names one symbol and `files()` writes several". + +The case it does not cover, taken from Lib-Live2D's `cmake/EmbeddedShaders.cmake` +(34 lines of `file(READ)` and string concatenation, with a second 59-line +variant for Metal): **N inputs, one header, one table**, where each row carries +the file's name alongside its contents and the consumer iterates. + +```cpp +inline constexpr EmbeddedShader embedded_shaders[]{ + {"Standard.vert", R"(…)"}, + {"Standard.frag", R"(…)"}, +}; +``` + +This is not a new plugin. It is a `table()` entry point beside `file()` and +`files()`, with an option for the row type's name and for how the key is +derived from the path. The `write_if_different` behaviour already there is the +part that makes it cheap to call unconditionally, and it carries over. + +I have not measured how many projects want the table shape rather than the +per-file shape. One does, and its 93 lines of CMake are the evidence that the +shape is worth having; that is an argument for adding it, not a measurement of +demand. + +## 6. What a dist member owes its consumer + +The rules in `docs/30-build-mcpp.md` apply unchanged, and two of them bind +harder here than for a rule: + +**Expose a plan/submit pair.** A dist member's output is the last thing before +a user's hands, so it is the most likely to need a project-specific edit — a +different compression level, an extra file, a second signature. `generate_all(opt)` +being `submit(plan_all(opt))` is what keeps that edit from becoming a +reimplementation. + +**Failure and advice use different channels.** A packaging step that succeeds +while carrying nothing is the failure mode section 2 measured. Where a dist +member can detect an empty or implausible result, it must say so on a +*successful* build through `mcpp::warning`, because stderr on success is +discarded. + +**One `(name, version)` names one payload.** A dist member that wraps a signing +tool inherits that tool's compatibility surface. Versioning in lock-step with +the wrapped tool is legitimate and says something true. + +Two more that are specific to this category: + +**Name the input, do not harvest a directory.** Section 2's 52 KB installer is +the general case: a path that resolves to nothing is silent, and a named input +that is missing is an error. `${mcpp.target_file:}` is the mechanism — +a build program is told neither the triple nor the fingerprint, and an unknown +target name is refused rather than expanded to an empty path. + +**Declare the tool where it will be looked up.** `xpkg_dir` answers from +`MCPP_XPKG_*_DIR`, which mcpp sets for the *building* package. A dependency's +declaration provisions the payload — the log says +`Provisioning [xlings.workspace] entries (...)` — without making it visible to +a consumer's build program. A dist member that runs a payload tool therefore +declares it itself, and says so when the lookup returns empty rather than +pointing at a manifest the reader does not own. + +## 7. Staging + +The order is not the demand order, and the reason is that the lower layers are +shared: + +1. **`dist-wix`, then `dist-appimage`.** No engine change. `dist-wix` is a port + of a working implementation; `dist-appimage` is the same shape on the + platform where it is easiest to test. Both restage by hand, which is the + evidence for step 2 rather than a reason to delay them. +2. **The two engine additions of 2.1: a consumable staged tree, and package + metadata in the build program.** Both are format-neutral, and step 1 will + have shown what each costs to do without. After this, no format needs the + engine again. + +3. **`tools-embed`'s `table()`.** Independent of everything else, and the + smallest of these. +4. **`dist-apple` for macOS.** Establishes the bundle-and-sign shape on a target + that already exists, so that iOS later adds a row and not a design. +5. **`xim:android-ndk` and `xim:emsdk`, each carrying the module surface (3.1).** + Both are measured working and neither waits on the engine. Writable before + the rows, and doing them first turns each row into a small verifiable change + rather than a change plus an unknown. +6. **The Android row.** One `env` value, one table row, one payload. The + smallest of the three platform steps, and it makes `dist-android` writable. +7. **The iOS row**, then `dist-apple` extends to it — preceded by the same + measurement for Apple clang, and by whether `xim:iphoneos-sdk` can be a + payload or must be a locator. +8. **Web**, as [#597](https://github.com/mcpp-community/mcpp/issues/597) + describes. It is a target-model change and not a table row — but it is now + *only* that: 3.1 measured the standard-library half and it works, so #597 is + one problem rather than two. + +Steps 1 to 4 do not wait on a platform. Doing them first means that when a +platform row lands, the layer above it already exists — and that the row is the +only thing that had to land. + +## 8. What this does not solve + +**A published package still carries no consumer dependencies from the target +axis.** `mcpp emit xpkg` derives `xpm..deps` from the top-level +`[xlings.workspace]` only. A dist member declaring its tool on the target axis — +which is where a payload the produced code links against belongs — publishes +cleanly and fails in the consumer's link. The workaround is to declare on both +axes, and the second copy states a target fact on the host axis, which mcpp's +own guidance calls the wrong form. + +**`mcpp test` is not configurable.** A library with an existing test tree that +is not `tests/**/*.cpp` cannot use `mcpp test` at all; HuxerUI reached this and +answered it by adding a separate package whose `tests/` selects instead of +discovering. This is unrelated to plugins and is noted because it is the second +thing an ecosystem library hits. + +**Apple was not measured, and its recipe may not exist.** Android and +Emscripten were carried to a running or linking artifact; iOS cannot be +measured on a Linux host, and 3.1 states the two possibilities rather than +choosing one. The other two results are the argument for not guessing: every +intermediate guess along the way was wrong — that the gap was one file (it is +133), that a mismatched surface fails obscurely (it names the missing header), +that the blocker was libc++ (it was bionic's `static inline` ctype), and that +Emscripten's libc++ version follows its clang version (it does not). + +**Neither working recipe was carried to a released payload.** Both were built +and exercised in a scratch directory. Turning each into an xim recipe — the +fetch, the substitution, the version check, the layout — is the next step and +is not done here. + +**Nothing here shortens the platform work itself.** Sections 3 and 7 say which +layer is engine and which is package; they do not make the engine layer +smaller. Android is small because the model nearly admits it already, not +because a plugin can stand in for the row. + +## 9. Review of sections 4 to 7, and what each risk is measured against + +The proposal survives review with one correction, one omission that would have +been a silent defect, and one boundary the sections above draw in the wrong +place. They are stated here before the task list because each one moves a task. + +### 9.1 The correction: `--stage-only` already shipped, under another name + +Section 2.2(a) asks for "a `${mcpp.stage_dir}` placeholder, or a `--stage-only` +mode whose output path an action can name". The second half exists. +`--format dir` sets `writeArchive = false`, and `mcpp pack` then reports +`plan.stagingRoot` — `target/dist//` — as the output. That tree +is the closure after the strip policy, the debug split and `include`/`exclude` +have run: the thing section 2.2(a) describes. + +So only the placeholder is new. `--stage-only` would be a second spelling of a +mode that ships, which is the shape this project refuses elsewhere +(`mcpp sbom` versus `mcpp emit sbom`). + +### 9.2 The omission: the requested format is a graph input, so it is graph state + +`mcpp pack --format appimage` changes what the build program submits. Anything +that changes what a build program submits changes the graph, and +`target///build.ninja` is shared mutable state that two fast paths +replay. `mcpp.build.graph_shape` exists because of exactly this class of +defect, and it already carries two such axes: `graph=` (a test graph replayed +for a plain build) and `accel=` (a device variant a flag chose, replayed for a +build that chose nothing). + +A `dist` axis with no entry on that line reproduces the same failure a third +time: `mcpp pack --format appimage`, then `mcpp build`, replays a graph +carrying a dist edge that a plain build must not have. The line therefore gains +a third field, and `is_plain_build_graph` requires it to read `none`. + +The format deliberately does **not** enter the build fingerprint. It would put +the packaging pass in its own directory and cost a full recompile to produce a +distributable from an already-built tree. The header line is the cheaper half +of that pair and is the half that answers the question the fast paths ask. + +### 9.3 The boundary in the wrong place: a staged tree is not a root filesystem + +Section 2.1 lists `.deb`, `.rpm` and AppImage as one group and 2.2(a) offers +one staged tree to all of them. Two shapes are being conflated: + +| Format | Wants | +|---|---| +| AppImage, `.app`, `.msi` | a **bundle** tree: `bin/`, `lib/`, relocatable, rooted anywhere | +| `.deb`, `.rpm` | an **FHS** tree: `usr/bin/`, `usr/lib//`, rooted at `/` | + +`mcpp pack` stages the first — it is what `--mode vendored` means, and the +`$ORIGIN` rewriting and the `run.sh` wrapper are what make it relocatable. A +`.deb` member consuming that tree must re-lay it out, and the re-layout is +`.deb`'s knowledge rather than the engine's, so this is not an argument for a +second staged tree in the engine. It is an argument about which member goes +first: **a bundle-shaped format exercises `${mcpp.stage_dir}` as it is, and an +FHS-shaped one exercises a re-layout step that would then be the thing under +test.** Section 7's choice of AppImage for step 1 is right, and the reason is +this rather than "the platform where it is easiest to test". + +### 9.4 The ordering problem sections 2.2 and 7 do not state + +A `role = "artifact"` action is a ninja edge. The staged tree is produced by +`mcpp::pack::run` in C++ **after** ninja has finished. So an artifact action +cannot depend on the staged tree in the pass that builds it, and +`${mcpp.stage_dir}` is not expressible in a single-pass pack. + +Two passes are, and every value the second one needs is already answered by the +first: + +1. `prepare_build`, with no format set. Build programs run and declare the + formats they provide; none submits a dist action, because none was asked + for. An unknown `--format` is refused **here**, before anything is compiled, + naming the set that was declared. +2. The ordinary build. The link outputs exist. +3. `make_plan` and `pack::run`. The staged tree exists at `plan.stagingRoot`. +4. `prepare_build` again, with `pack_format` and `pack_stage_dir` set to what + steps 1 and 3 answered — **not re-derived**. The claiming member submits its + action. Its command is what ninja runs. + +The re-derivation is what would have been the defect. `plan.stagingRoot` is a +function of the package name, the version, the resolved triple and the mode, +and the resolved triple is not known until prepare has run. Computing it a +second time before prepare — from `host_triple()`, say — is the shape where two +derivations of one value agree on every machine the author has and disagree on +one they do not. + +The second prepare is not free. It is bounded by the build program re-running: +the contract values are part of its re-run key unconditionally, so changing +`MCPP_PACK_FORMAT` invalidates exactly that one entry and nothing else. + +### 9.5 The declaration must not be gated on the request + +A member that emitted `mcpp:pack-format=appimage` only when +`pack_format() == "appimage"` would make the set unknowable: the engine could +never answer "which formats does this graph provide" and `--format bogus` could +name nothing. So the contract has two halves that must not be merged: + +> **Declare unconditionally. Submit conditionally.** + +This is the load-bearing rule of the whole dispatch, and it is the rule most +likely to be got wrong by a member author, because a member that gets it wrong +still works for the person who wrote it — they always pass their own format. +It therefore needs a test whose failure mode is the wrong half: a build that +requests **no** format and asserts the set is still non-empty. + +### 9.6 Risks, each with the measurement that would catch it + +| Risk | Why it would pass unnoticed | Criterion | +|---|---|---| +| The dist graph is replayed for a plain build | Both graphs live in one directory; the fast path predates any plan | `pack --format X`, then `build`, then assert no dist edge ran — the `A then B then A` shape | +| The second prepare replays a cached build program | A cached run re-emits the first pass's output, which submits nothing, so the pass succeeds and produces nothing | Assert the dist output **exists**, never that the command exited 0 | +| `${mcpp.stage_dir}` in a non-packing build | Expands to an empty string; the command then reads the build directory root, which exists | Refuse at expansion, and assert the refusal text | +| A member declares a built-in name (`tar`, `dir`) | The built-in wins and the member is silently unreachable | Refuse the collision, naming both | +| The dist action runs before staging | Only in a single-pass design; recorded so the two-pass ordering is not "simplified" away later | Assert the staged tree is non-empty **from inside the action** | +| A dist member's tool is absent | The tool is a host lookup, and an empty path becomes an argv token | Refuse in the build program, naming the tool and where it was looked for | +| The produced distributable is valid and empty | Section 2's measured 52 KB installer | Each member asserts a floor on its own output, on the **success** path, through `mcpp::warning` | + +### 9.7 What sections 4 to 7 promise that this pass does not deliver + +Stated here rather than discovered later. Each is blocked on something no +amount of engine work supplies: + +- **`xim:android-ndk`, `xim:emsdk`** (§7 step 5) and the **Android row** + (step 6). Both recipes are measured working in a scratch directory (§3.1); + turning either into a payload means fetching and republishing a + multi-gigabyte vendor toolchain with a derived 133-file module surface. That + is its own release, not a side effect of this one. +- **The iOS row** (step 7) and `dist-apple`'s iOS half. Not measurable on a + Linux host, and §3.1 states the two possibilities rather than choosing. +- **Web** (step 8). [#597](https://github.com/mcpp-community/mcpp/issues/597) + is a target-model change. +- **`dist-android`, `dist-web`**. Each waits on its row. + +The engine additions in §2.2 are what make each of those a single verifiable +change when it comes. None of them is a prerequisite for the others. + +### 9.8 The iOS SDK: a three-tier policy rather than an open question + +Section 3.1 leaves `xim:iphoneos-sdk` as "licensing decides whether it installs +or merely locates", and §8 repeats it. The decision procedure is not a +measurement — it is a preference order, and stating it removes the open +question without taking the measurement: + +1. **Redistribute, if the licence permits it.** A published payload with a + GitCode mirror, like every other `xim` toolchain package. Publicly mirrored + SDK trees exist — `https://github.com/xybp888/iOS-SDKs` is one — and whether + this tier is reachable is a licence reading of the SDK itself, not of the + mirror. +2. **Fetch from upstream, without a CN mirror.** A recipe that downloads at + install time from the upstream URL and mirrors nothing. This is what a + licence that permits use but not redistribution allows, and declining the + mirror is the point: a mirrored copy *is* redistribution, so the tier is + defined by what it refuses to do. +3. **Locate what the machine already has.** The `msvc@system` shape: find, pin + and report an installed Xcode, install nothing. Correct under any licence, + and the only tier that cannot serve a machine without Xcode. + +Tier 3 always works and is therefore the floor, not the goal. The recipe should +reach for the lowest tier the licence allows and say in its description which +tier it took, because a consumer reading "locator" needs to know that is a +licence conclusion rather than an unfinished recipe. + +## 10. The task list + +Four repositories. One pull request each, in this order, because each depends +on the one above it being released rather than merely merged. + +### 10.1 `mcpp` — the three format-neutral additions + +| # | Task | Depends on | +|---|---|---| +| E1 | `MCPP_PKG_VERSION` / `_DESCRIPTION` / `_LICENSE` / `_AUTHORS` / `_REPO` in the build-program environment, with accessors | — | +| E2 | `MCPP_PACK_FORMAT` in that environment, and `mcpp::pack_format()` | — | +| E3 | The `mcpp:pack-format=` outlet, collected onto the plan | — | +| E4 | `${mcpp.stage_dir}`: expansion for Artifact actions, refusal elsewhere, the stage manifest as an implicit input | E2 | +| E5 | `--format` accepts a provided name; the refusal names the available set | E3 | +| E6 | The two-pass pack pipeline (§9.4) | E4, E5 | +| E7 | `dist=` on the graph header line; `is_plain_build_graph` requires `none` | E6 | +| E8 | `docs/10`, `docs/30`, `docs/31`, and a new `docs/35`; the `zh` mirror of each | E1–E7 | +| E9 | Unit tests for E3–E5, e2e for E6–E7, each with the §9.6 criterion | E1–E7 | + +### 10.2 `mcpp-plugins` — one member per verified platform + +| # | Task | Depends on | +|---|---|---| +| P1 | `tools-embed`: a `table()` entry point (§5) | — | +| P2 | `dist-appimage` → `mcpp.dist.appimage` | mcpp released, X1 | +| P3 | `dist-wix` → `mcpp.dist.wix` | mcpp released | +| P4 | `dist-apple` → `mcpp.dist.apple`, macOS half only | mcpp released | +| P5 | `MCPP_VERSION` in CI raised to the engine that carries E1–E7 | E1–E7 released | +| P6 | Plan-level tests on all three runners; end-to-end each on its own | P1–P4 | + +### 10.3 `xim-pkgindex` — the one payload this needs + +| # | Task | Depends on | +|---|---|---| +| X1 | `xim:appimagetool` | — | + +### 10.4 `mcpp-index` — publication + +| # | Task | Depends on | +|---|---|---| +| I1 | `mcpp:plugins@0.3.0` | P1–P6 released | + +The engine tasks are the only ones on the critical path. P1 and X1 do not wait +on anything. diff --git a/.agents/docs/README.md b/.agents/docs/README.md index 7fce20b7d..40d7d388e 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded --- ``` -274 records. +275 records. ## By subject @@ -40,10 +40,15 @@ Records that declare one. Everything else is listed by date below. - [Two answers and two silences: the scanner's second grammar, and the manifest keys nothing reads](2026-09-09-two-answers-and-two-silences.md) — active +### plugins + +- [The category the plugin taxonomy does not name, and what a platform actually decomposes into](2026-09-11-distribution-plugins-and-platform-decomposition.md) — active + ## By date ### 2026-09 +- [The category the plugin taxonomy does not name, and what a platform actually decomposes into](2026-09-11-distribution-plugins-and-platform-decomposition.md) — active - [Two answers and two silences: the scanner's second grammar, and the manifest keys nothing reads](2026-09-09-two-answers-and-two-silences.md) — active - [A dlopen surface no closure walks, and a process with two unwinders](2026-09-09-dlopen-surface-and-two-unwinders.md) — landed - [The documentation as a book: a chapter-by-chapter design](2026-09-08-the-documentation-as-a-book.md) — active diff --git a/CHANGELOG.md b/CHANGELOG.md index cf282c38f..035f6f3e8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,71 @@ ## [Unreleased] +## [2026.9.11.1] - 2026-09-11 + +### `mcpp pack --format ` 分派到包,而引擎里不再需要住进任何一种分发格式 + +`tar` 与 `dir` 回答的问题,和 `msi` 与 `appimage` 回答的问题是同一个 —— 输出取什么 +形状 —— 所以它们是一个 flag 的取值,而不是第二个 flag 的开端。分界是:`mcpp pack` +拥有机制,以及那一种通用格式(一个解开就能跑的归档,它不需要知道任何别人的发布); +其余每一种格式都住在包里。dpkg 的 control 字段、AppImage 的 runtime、WiX 的 schema、 +Apple 的公证,其中任何一个被绑进引擎,都会把一次 mcpp 的发布耦合到一次 mcpp 并不控制 +的发布上。这与本项目早已为语言做过的论证是同一个 —— Slang 被支持,而引擎里没有它的 +名字。 + +三样与格式无关的东西被加进引擎,「与格式无关」正是判断某样东西该不该进引擎的判据: + +- **一棵 `artifact` action 可以消费的暂存树。** `mcpp pack` 一直在算它 —— 依赖闭包, + 过了 strip 策略、调试信息拆分与 `include`/`exclude` —— 然后把它压掉,目录就没了。 + `${mcpp.stage_dir}` 把它暴露出来。它是一个 **bundle** 树(`bin/`、`lib/`、可重定位), + 这正是 AppImage、`.app`、`.msi` 要的形状;要一棵 FHS 树的格式(`.deb`、`.rpm`)自己 + 负责重排,因为一个文件该落在哪个目录是那个格式的知识。 +- **`[package]` 的其余字段进入构建程序。** `MCPP_PKG_VERSION` / `_DESCRIPTION` / + `_LICENSE` / `_AUTHORS` / `_REPO`,以及对应的 `mcpp::package_*()`。每一种安装包格式 + 都要写版本号;在这之前,项目只能把版本号在成员自己的 options 里再写一遍,而那份副本 + 会与 `[package]` 漂移,且没有任何东西能发现。 +- **`--format` 经图解析它的取值。** 包用 `mcpp::provides_pack_format("")` 声明, + `--format ` 在解析后的依赖里找到提供方并把暂存树交给它。未知取值点名**当下确实 + 可用**的那些,而不是一份固定清单,并且这次拒绝发生在任何东西被编译之前。 + +**无条件声明,有条件提交。** 这是整套分派最承重的一条规则,也是最容易被成员作者写错 +的一条 —— 因为写错了对作者自己仍然照常工作:他永远传的是自己那个格式。声明必须不加闸, +否则引擎永远回答不出「这张图提供哪些格式」;提交必须加闸,否则普通 `mcpp build` 会多出 +一条它不该有的边。对一个谁都没为之提交的格式,mcpp 会拒绝并点名,而不是报告一次「什么 +包都没产出」的成功打包。 + +**`mcpp pack --format ` 会 prepare 两次,而两次之间没有任何值被重新推导。** 一条 +`artifact` action 是一条 ninja 边,而暂存树是 mcpp 在链接**之后**产出的,所以这棵树不 +可能成为构建出它自己那一次 pass 的输入。第一趟收集声明并拒绝未知格式;随后是构建与暂存; +第二趟设上 `pack_format` 与 `pack_stage_dir` 并构建提供方提交的那条边。第二趟用的三元组 +与暂存路径,都是第一趟和 `make_plan` 已经回答过的 —— 重新推导一遍会得到那种「在作者所有 +机器上都一致、只在他没有的那台上不一致」的缺陷。 + +**build.ninja 的头行多了第四个字段 `dist=`,而快路径要求它读作 `none`。** 格式故意 +**不进指纹**:进了就要为「把一棵已经构建好的树打成包」付一次全量重编。于是两张图落在同 +一个目录里,而 `target///build.ninja` 是被两条快路径回放的共享可变状态 —— +这正是这一行上另外两个字段(`graph=`、`accel=`)已经各自记过一次的那种失败。判据放在 +单元测试里而不是端到端:实测(2026-09-11)即使忽略这个字段,pack 之后的普通构建也会因为 +更早的一个新鲜度条件而重新生成图,所以端到端断言无论字段是否生效都会通过,连字段被删掉 +都照样通过。 + +**`${mcpp.stage_dir}` 在两种位置上是拒绝而不是空展开:** 本次构建不在打包时,以及 +role 不是 `artifact` 时。一个空路径仍然是命令接受的 token,而工具随后读到的是构建目录 +根 —— 那个目录存在,所以这个错误会产出一个看起来合理的产物而不是一条诊断。实测过的原型 +是那个「有效的、空的、52 KB 的安装包,并且没有任何诊断」。 + +写了 `${mcpp.stage_dir}` 的 action 会自动获得一条对 `<暂存树>.stage-manifest` 的依赖 +(一个兄弟文件,永不是成员,所以它不会跑进任何人的安装包里)。依赖由引擎添加,因为「用 +了」本身就意味着「依赖」:没有它,这条边只在链接产物变化时才变脏,而一个闭包多出了某个 +依赖的共享库、同时程序自己的字节没变的情况,会把上一次的可分发物原地留下并报告为最新。 + +构建程序协议升到 v9(`mcpp:pack-format=`)。这条指令带非空 `tag`,因此会随构建程序的 +缓存记录一起被回放 —— 读它的那一趟是 `mcpp pack`,而那从来不是一个项目的第一次构建。 + +文档:`docs/10`(`--format` 那一个轴)、`docs/30`(三类成员的分类表、新占位符与新访问器)、 +`docs/31`(分发成员的六条约束),中英双份。 + + ## [2026.9.10.2] - 2026-09-10 ### dlopen 面检查:不适用的那一趟也会发布记录,并且不会盖掉已经量出来的答案 diff --git a/docs/10-pack-and-release.md b/docs/10-pack-and-release.md index d99bb095e..b008014db 100644 --- a/docs/10-pack-and-release.md +++ b/docs/10-pack-and-release.md @@ -140,6 +140,7 @@ mcpp pack --mode self-contained # alias: --mode bundle-all mcpp pack --target x86_64-linux-musl # equivalent to --mode static mcpp pack --target aarch64-linux-musl # ARM64 equivalent mcpp pack --format dir # output as a directory, no tarball +mcpp pack --format appimage # a format a package in the graph provides mcpp pack -o myapp.tar.gz # filename only: lands at target/dist/myapp.tar.gz mcpp pack -o /abs/path/myapp.tar.gz # includes a directory: output to the literal path mcpp pack --profile dev # build with a different profile (default: release) @@ -147,6 +148,47 @@ mcpp pack --no-strip # ship the artifacts as built mcpp pack --debug-symbols dbg/ # write the separated *.debug files under dbg/ ``` +### `--format` owns one axis, and the engine owns two of its values + +`tar` and `dir` answer the same question `msi` and `appimage` answer — what +shape does the output take — so they are values of one flag rather than the +beginning of a second one. The split between what the engine holds and what a +package holds is: + +> **`mcpp pack` owns the mechanism and the one universal format. Every other +> format lives in a package, and `mcpp pack` dispatches to it.** + +The universal format is what it already produces: an archive that extracts and +runs. It is universal in the only sense that matters here — it needs no +knowledge of anyone else's release. Everything past it does. dpkg's control +fields, AppImage's runtime, WiX's schema, Apple's notarisation, Android's +signing scheme: each one bound into the engine would couple an mcpp release to +a release mcpp does not control. The same argument the project already made for +languages, where Slang is supported without being named in the engine. + +So the value set is open (mcpp 2026.9.11.1+). `--format ` finds the +package in the resolved graph that declares `` and hands it the staged +tree; an unknown value names what *is* available rather than a fixed list: + +``` +error: unknown --format 'bogus'. + available in this build: tar, dir, appimage + A format past `tar` and `dir` comes from a package in the resolved graph, which declares + it with `mcpp::provides_pack_format("")` in its build program. Add the package + that provides 'bogus' to [build-dependencies] and activate its feature. +``` + +The refusal arrives before anything is compiled. Writing such a package is +[Producing a distributable](30-build-mcpp.md#producing-a-distributable-pack_format--stage_dir-20269111); +the engine's three additions are a staged tree an `artifact` action can consume, +the rest of `[package]` in the build program, and this dispatch. Each is +format-neutral, which is the test for whether something belongs in the engine +at all. + +A dispatched format applies to a **program** target. A library package ships an +interface plus prebuilt binaries per triple and has no single staged tree, so +`mcpp pack --format ` is refused rather than ignored. + When `-o` is given a bare filename, the output is placed under `target/dist/`; when it includes a directory (relative or absolute), the literal path is used. @@ -421,8 +463,13 @@ macOS **program** bundling (the Mach-O dependency closure, via `otool -L` / `LC_LOAD_DYLIB`, and `install_name_tool` for relocation) is still on the roadmap; until it lands `mcpp pack ` refuses on that format rather than producing something that only looks like a bundle. Windows DLL bundling beyond -the current `.zip`, and distribution formats such as `.deb` / `.rpm` / AppImage, -are also on the roadmap. This document evolves alongside the -`mcpp pack` implementation; for the latest options, refer to -`mcpp pack --help`. +the current `.zip` is also on the roadmap. + +Distribution formats such as `.deb`, `.rpm`, AppImage and `.msi` are **not** on +this list, and that is a decision rather than an omission: they live in +packages and reach the user through `--format `, for the reason the +section above gives. Nothing further needs to join `[pack]`'s built-in modes. + +This document evolves alongside the `mcpp pack` implementation; for the latest +options, refer to `mcpp pack --help`. diff --git a/docs/30-build-mcpp.md b/docs/30-build-mcpp.md index 56e7b9195..a7dd89f5c 100644 --- a/docs/30-build-mcpp.md +++ b/docs/30-build-mcpp.md @@ -463,6 +463,12 @@ attach: | `object` | join the **link** set | the link edge consumes them | a resource compiler, `objcopy` embedding a blob, a generated `.def`, a pre-built `.o` | | `artifact` | a new file | its *inputs* are link outputs, so it runs after the link | codesign, packaging, size budgets | +`artifact` is also the only role that may name `${mcpp.stage_dir}` — see +[Producing a distributable](#producing-a-distributable-pack_format--stage_dir-20269111) +below. The other three run before or alongside the link, so there is nothing +staged for them to read, and mcpp refuses the placeholder rather than expanding +it to a path that happens to exist. + No phase machinery is involved. `object` and `artifact` are sequenced by ninja's own file dependencies — which is also why an `artifact` action cannot double-apply itself the way a naive "post-build hook" would. `source` and a @@ -553,6 +559,81 @@ scan agrees with what the generator will emit — the same assertion-plus- verification trade `[modules].scan_overrides` makes, and the compiler's own P1689 output checks it at build time. +### Producing a distributable: `pack_format` / `stage_dir` (2026.9.11.1+) + +An `.msi`, a `.deb`, an AppImage and a signed `.app` are none of the four roles' +usual work and all of them are `artifact`: each consumes **link outputs** and +produces something a user installs. What the engine adds for them is a +mechanism and no format at all. + +`mcpp pack --format ` resolves `` through the resolved graph, the +same way `--target` reaches a triple the engine did not have to know +individually. `tar` and `dir` remain the archive shapes `mcpp pack` owns; +everything past them comes from a package. + +A provider has two halves, and they must not be merged: + +```cpp +import mcpp; +#include +#include + +int main() { + // Half one, unconditional. + mcpp::provides_pack_format("appimage"); + + // Half two, conditional. + if (std::string_view(mcpp::pack_format()) != "appimage") return 0; + + const std::string out = std::string(mcpp::out_dir()) + "/app.AppImage"; + mcpp::action a; + a.id = "appimage"; + a.role = "artifact"; + a.arg(tool).arg("${mcpp.stage_dir}").arg(out.c_str()) + .input("${mcpp.target_file:app}") + .output(out.c_str()) + .submit(); + return 0; +} +``` + +**Declare unconditionally, submit conditionally.** The declaration is what lets +the engine answer a question the requesting build cannot: `--format bogus` +names what *is* available, and `--help` says "any format the resolved graph +provides". Both read the set collected from a pass that asked for nothing. A +member that declared only when asked still works for its author — they always +pass their own format — and makes the set unknowable for everyone else. mcpp +refuses a format nothing submitted for, rather than reporting a pack that +produced no package. + +**`mcpp pack --format ` prepares twice.** An `artifact` action is a ninja +edge and the staged tree is produced by mcpp *after* the link, so the tree +cannot be an input of the pass that built it. The first pass collects the +declarations and refuses an unknown format before anything is compiled; the +build and the staging then happen; the second pass sets `pack_format` and +`pack_stage_dir` and builds the edge the provider submits. Nothing in the +second pass is re-derived — the triple and the staged path are what the first +pass and the staging already answered. + +**The staged tree is a bundle, not a root filesystem.** It is what +`--mode vendored` means: `bin/`, `lib/`, relocatable, rooted anywhere, after the +strip policy, the debug-symbol split and `include`/`exclude`. An AppImage, a +`.app` and an `.msi` want it as it stands. A format that wants an FHS tree +(`.deb`, `.rpm`) owns the re-layout, because which directory a file belongs in +is that format's knowledge and not the engine's. + +`mcpp pack --format dir` writes the same tree to a path and stops, which is how +a person inspects what a member will be handed. + +**An action that names `${mcpp.stage_dir}` gains a dependency on the tree's +manifest.** mcpp writes `.stage-manifest` — a sibling, never a +member, so it does not travel inside anyone's installer — listing each staged +file's size and relative path. The dependency is added by the engine because +the use implies it: without it the edge is dirty only when a link output +changes, and a closure that grew a dependency's shared library while the +program's own bytes did not would leave the previous distributable in place, +reported as up to date. + Commands are an **argv, not a shell string** (no shell is assumed — Windows has none to rely on), and the only interpolations are a closed set: @@ -562,6 +643,7 @@ none to rely on), and the only interpolations are a closed set: | `${mcpp.bin_dir}` | where produced binaries land | | `${mcpp.compile_db}` | path to `compile_commands.json` (what clang-tidy's `-p` wants) | | `${mcpp.target_file:}` | the built file of target `` | +| `${mcpp.stage_dir}` *(2026.9.11.1+)* | the tree `mcpp pack` staged, absolute. `artifact` role only, and only under `mcpp pack --format ` | The raw stdout protocol above remains the low-level substrate; `import mcpp;` is the typed layer over it. @@ -671,6 +753,13 @@ The running program receives the build context as `MCPP_*` variables | `MCPP_LANGUAGE_MODULES` *(2026.9.7.1+)* | -- | `1` when the declaring package sets `[language] modules`, `0` otherwise. A rule that GENERATES a consumer-facing declaration reads it to choose between a module interface and a header, so a project states that once and never again. An older engine leaves it absent, which a rule reads as `0` -- the behaviour every consumer had before the variable existed | | `MCPP_PKG_NAME` *(2026.9.7.1+)* | -- | The `[package] name` of the package this program builds. Every name a rule generates is derived from it: the module a consumer imports, the namespace the accessors sit in, the symbols in a generated header. Before it existed the closest available answer was the leaf of `MCPP_MANIFEST_DIR`, which is a directory name -- so a package named `vulkan-saxpy` in a directory named `app` generated `app.shaders`, and every `/app/` in a workspace claimed the same module. Absent under an older engine, which a rule reads as a signal to keep its previous derivation | | `MCPP_PKG_NAMESPACE` *(2026.9.7.1+)* | -- | The `[package] namespace`. Empty when the package declares none. A rule that must produce a name unique across an index uses the pair rather than the name alone, because package identity is `(namespace, name)` | +| `MCPP_PKG_VERSION` *(2026.9.11.1+)* | `mcpp::package_version()` | The `[package] version`. Every installer format states a version; without this the project had to restate it in the member's own options, where the copy drifts from `[package]` with nothing able to detect it | +| `MCPP_PKG_DESCRIPTION` *(2026.9.11.1+)* | `mcpp::package_description()` | The `[package] description`. Empty when the package declares none | +| `MCPP_PKG_LICENSE` *(2026.9.11.1+)* | `mcpp::package_license()` | The `[package] license` | +| `MCPP_PKG_AUTHORS` *(2026.9.11.1+)* | `mcpp::package_authors()` | The `[package] authors`, joined with `;`. Not `,`: an author entry is conventionally `Name ` and a name may carry a comma, so a comma-joined list cannot be split back into the entries it was made from | +| `MCPP_PKG_REPO` *(2026.9.11.1+)* | `mcpp::package_repo()` | The `[package] repo` | +| `MCPP_PACK_FORMAT` *(2026.9.11.1+)* | `mcpp::pack_format()` | The `--format` value of the `mcpp pack` pass this program is part of; empty for every ordinary build. The empty value is the one that carries the meaning — a member gates its submission on this, so `mcpp build` has the graph it always had | +| `MCPP_PACK_STAGE_DIR` *(2026.9.11.1+)* | `mcpp::pack_stage_dir()` | Where `mcpp pack` has already staged the closure, absolute; empty when this build is not packaging. Read it to decide the shape of the work; write `${mcpp.stage_dir}` into the action, so the path in the graph and the path the program read cannot disagree | | `MCPP_DEVICE_SOURCES` *(2026.9.5.2+)* | `mcpp::device_sources()` | the device-kind sources (`.cu`, `.hip`, …) the package's effective `sources` match, package-root-relative, one per line; empty when there are none. The engine compiles none of them — the rule package this program imports turns each into an `mcpp::action`. Already narrowed: a `{ glob, accel }` entry the build does not cover contributes nothing, so `--no-accel` yields an empty list | | `MCPP_OUT_DIR` | `mcpp::out_dir()` | a writable scratch/output dir owned by mcpp | | `MCPP_MANIFEST_DIR` | `mcpp::manifest_dir()` | the package root (= CWD) | @@ -786,14 +875,30 @@ and warns when the two disagree — Nothing breaks; the name claims an origin the package does not have. A rule outside the project picks its own prefix. -**A tool is not a rule.** A rule states how a translation unit is compiled by a -compiler mcpp does not drive: it submits an action and the engine schedules it. -A tool states something the build program needs that no compiler performs, and -does it while the program runs. `mcpp.tools.embed` (feature `tools-embed`, +**A tool is not a rule, and a distributable is neither.** Three kinds of work, +and the member's prefix says which of the three questions it answers: + +| Prefix | Question | Compiles a TU | Runs in the build program | +|---|---|---|---| +| `rules-*` | how is this translation unit compiled | yes | no | +| `tools-*` | what does the build program need to do itself | no | yes | +| `dist-*` *(2026.9.11.1+)* | what comes out of the link, and in what form a user installs it | no | no | + +A rule states how a translation unit is compiled by a compiler mcpp does not +drive: it submits an action and the engine schedules it. A tool states +something the build program needs that no compiler performs, and does it while +the program runs. `mcpp.tools.embed` (feature `tools-embed`, mcpp 2026.9.5.4+) is the first: it writes a data file into a header the program compiles in, as a byte array or a 32-bit word array, and rewrites nothing when the content is unchanged, so calling it unconditionally costs no rebuild. +A `dist-*` member fits neither definition. It does not compile a translation +unit and it does not run while the build program runs: it consumes link outputs +and produces something a user installs, through an `artifact` action and +`mcpp pack --format `. The prefix matters because the taxonomy is +load-bearing — a consumer reading `rules-wix` would expect a compiler and a +translation unit, and there is neither. + `examples/09-heterogeneous/cuda` and `examples/09-heterogeneous/vulkan` consume `mcpp.rules.cuda` and `mcpp.rules.spirv` from `mcpp:plugins`, the way any project does. diff --git a/docs/31-authoring-a-rule-package.md b/docs/31-authoring-a-rule-package.md index dd770f425..92ffd6c9b 100644 --- a/docs/31-authoring-a-rule-package.md +++ b/docs/31-authoring-a-rule-package.md @@ -301,6 +301,61 @@ that declares the edge under another key can supply it, and refuses with a message naming what it looked for rather than running a command with an empty path. +## Authoring a distribution member (mcpp 2026.9.11.1+) + +A `dist-*` member is neither a rule nor a tool. It does not compile a +translation unit and it does not do its work while the build program runs: it +consumes **link outputs** and produces something a user installs — an `.msi`, a +`.deb`, an AppImage, a signed `.app`. The mechanism is an `artifact` action and +`mcpp pack --format `, documented in +[30 — Producing a distributable](30-build-mcpp.md#producing-a-distributable-pack_format--stage_dir-20269111). + +Everything in this chapter applies unchanged. Six things bind harder here, and +each is a mistake this category makes and a rule does not. + +**Declare unconditionally, submit conditionally.** `provides_pack_format` is +what the engine reads to answer "which formats does this graph provide", on a +build that asked for none. A member that declares only when asked works for its +author, who always passes their own format, and makes the set unknowable for +everyone else. + +**Expose a plan and a submit.** A distributable is the last thing before a +user's hands, so it is the most likely part of a build to need a +project-specific edit: a different compression level, one extra file, a second +signature. `generate_all(opt)` being exactly `submit(plan_all(opt))` is what +keeps such an edit from becoming a reimplementation of the member. + +**Name the input; do not harvest a directory.** A path that resolves to nothing +is silent, and a named input that is missing is an error. Measured: a WiX action +that bound a directory and harvested it produced a **valid, empty, 52 KB +installer with no diagnostic** when the path resolved to nothing on Windows. +`${mcpp.target_file:}` is the mechanism — a build program is told neither +the triple nor the fingerprint, and an unknown target name is refused rather +than expanded to an empty path. + +**Assert a floor on the member's own output, on the success path.** Where a +member can tell that its result is empty or implausible, it must say so through +`mcpp::warning`, because stderr on a successful build is discarded. This is the +same failure the paragraph above measured, caught one layer later. + +**Declare the tool where it will be looked up.** `xpkg_dir` answers from +`MCPP_XPKG_*_DIR`, which mcpp sets for the package *being built*. A +dependency's declaration provisions the payload without making it visible to a +consumer's build program, so a member that runs a payload tool declares it +itself — and says so when the lookup returns empty, rather than pointing at a +manifest the reader does not own. + +**A build must not reach the network, and a wrapped tool may.** Measured on +`appimagetool` 1.9.1: it downloads its type-2 runtime stub from a GitHub +release on every invocation unless `--runtime-file` names a local copy. A member +that wraps such a tool has to supply the file from its declared payload. +Install time is when a download is legitimate; build time is not, and a build +that fetches is neither reproducible nor usable offline. + +**One `(name, version)` names one payload.** A member that wraps a signing or +packaging tool inherits that tool's compatibility surface, so versioning in +lock-step with the wrapped tool is legitimate and says something true. + ## Current limitations - A rule feature that is in the package's own `[features] default` does not diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index 62e5d6c18..a787f1576 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -106,6 +106,7 @@ mcpp pack --mode self-contained # 别名:--mode bundle-all mcpp pack --target x86_64-linux-musl # 等价 --mode static mcpp pack --target aarch64-linux-musl # ARM64 等价写法 mcpp pack --format dir # 输出为目录,不打包 tarball +mcpp pack --format appimage # 由图里某个包提供的格式 mcpp pack -o myapp.tar.gz # 仅文件名:落到 target/dist/myapp.tar.gz mcpp pack -o /abs/path/myapp.tar.gz # 含目录:按字面路径输出 mcpp pack --profile dev # 换一个 profile 构建(默认 release) @@ -113,6 +114,42 @@ mcpp pack --no-strip # 按构建原样发货,不剥符号 mcpp pack --debug-symbols dbg/ # 把分离出的 *.debug 写到 dbg/ ``` +### `--format` 是一个轴,引擎只拥有其中两个取值 + +`tar` 与 `dir` 回答的问题,和 `msi` 与 `appimage` 回答的问题是同一个 —— 输出取什么 +形状 —— 所以它们是一个 flag 的取值,而不是第二个 flag 的开端。引擎持有什么、包持有 +什么,分界是: + +> **`mcpp pack` 拥有机制,以及那一种通用格式。其余每一种格式都住在包里,由 +> `mcpp pack` 分派过去。** + +那种通用格式就是它已经在产出的东西:一个解开就能跑的归档。它「通用」只在这里唯一 +要紧的那个意义上 —— 它不需要知道任何别人的发布。此外的一切都需要。dpkg 的 control +字段、AppImage 的 runtime、WiX 的 schema、Apple 的公证、Android 的签名方案:其中任何 +一个被绑进引擎,都会把一次 mcpp 的发布耦合到一次 mcpp 并不控制的发布上。这与本项目 +早已为语言做过的论证是同一个 —— Slang 被支持,而引擎里没有它的名字。 + +所以取值集合是开放的(mcpp 2026.9.11.1+)。`--format ` 在解析后的图里找到声明 +了 `` 的那个包,并把暂存树交给它;一个未知的取值会点名**当下确实可用**的那些, +而不是一份固定清单: + +``` +error: unknown --format 'bogus'. + available in this build: tar, dir, appimage + A format past `tar` and `dir` comes from a package in the resolved graph, which declares + it with `mcpp::provides_pack_format("")` in its build program. Add the package + that provides 'bogus' to [build-dependencies] and activate its feature. +``` + +这次拒绝发生在任何东西被编译之前。怎么写这样一个包,见 +[产出可分发物](30-build-mcpp.md#产出可分发物pack_format-与-stage_dir20269111); +引擎加的三样东西是:一棵 `artifact` action 可以消费的暂存树、`[package]` 的其余字段 +进入构建程序、以及这次分派本身。每一样都与格式无关 —— 而「与格式无关」正是判断某样 +东西该不该进引擎的判据。 + +被分派的格式作用于一个**程序** target。库包发的是一份接口加上每个三元组的预构建产物, +没有单独一棵暂存树,所以 `mcpp pack <库> --format ` 会被拒绝,而不是被忽略。 + `-o` 接受裸文件名时自动归到 `target/dist/`;含目录(相对或绝对) 时按字面路径输出。 @@ -355,6 +392,11 @@ force_bundle = ["libfoo.so"] # 即使命中 PEP 600 名单也强制打包 macOS **程序** bundling(Mach-O 依赖闭包,走 `otool -L` / `LC_LOAD_DYLIB`, 重定位走 `install_name_tool`)仍在规划中;在它落地之前,`mcpp pack <程序>` 会在该格式上拒绝,而不是产出一个只是看起来像 bundle 的东西。当前 `.zip` -之外的 Windows DLL 分发,以及 `.deb` / `.rpm` / AppImage 等格式,同样在规划中。本文档随 `mcpp pack` 实现演进,最新选项以 -`mcpp pack --help` 为准。 +之外的 Windows DLL 分发,同样在规划中。 + +`.deb`、`.rpm`、AppImage、`.msi` 这些分发格式**不在**这份清单上,而这是一个决定而不是 +一处遗漏:它们住在包里,经 `--format ` 到达用户,理由见上一节。`[pack]` 的内建 +模式不需要再添任何新成员。 + +本文档随 `mcpp pack` 实现演进,最新选项以 `mcpp pack --help` 为准。 diff --git a/docs/zh/30-build-mcpp.md b/docs/zh/30-build-mcpp.md index e4587085b..e541d651f 100644 --- a/docs/zh/30-build-mcpp.md +++ b/docs/zh/30-build-mcpp.md @@ -399,6 +399,11 @@ int main() { | `object` | 进**链接**集 | 链接边消费它们 | 资源编译器、`objcopy` 嵌 blob、生成的 `.def`、预编译 `.o` | | `artifact` | 一个新文件 | 它的**输入**是链接产物,所以在链接之后跑 | 签名、打包、size budget | +`artifact` 也是唯一允许写 `${mcpp.stage_dir}` 的 role —— 见下文 +[产出可分发物](#产出可分发物pack_format-与-stage_dir20269111)。另外三个跑在链接之前 +或与链接并行,没有任何已暂存的东西可读,所以 mcpp 会拒绝这个占位符,而不是把它展开成 +一个恰好存在的路径。 + 全程不涉及任何 phase 机制。`object` 与 `artifact` 由 ninja 自己的文件依赖定序 —— 这也是为什么 `artifact` 不会像朴素的「post 构建钩子」那样把自己重复施加一遍。 `source` 与 blocking 的 `check` 则由一条 order-only 边定序:从声明它的那个包的 @@ -470,6 +475,72 @@ mcpp 会播下一个带着该声明的占位文件,使 prepare 期的扫描与 内容一致 —— 与 `[modules].scan_overrides` 同一条「声明 + 验证」的取舍,build 期由 编译器自己的 P1689 输出复核。 +### 产出可分发物:`pack_format` 与 `stage_dir`(2026.9.11.1+) + +一个 `.msi`、一个 `.deb`、一个 AppImage、一个签过名的 `.app`,都不是那四个 role 的 +惯常活计,而它们全都是 `artifact`:每一个都消费**链接产物**,产出用户去安装的东西。 +引擎为它们加的是一套机制,而不是任何一种格式。 + +`mcpp pack --format ` 把 `` 交给解析后的图去解决,方式与 `--target` +够到一个引擎不必逐个认识的三元组相同。`tar` 与 `dir` 仍然是 `mcpp pack` 自己拥有的 +归档形状;此外的一切都来自某个包。 + +一个提供方有两半,而这两半不许被并成一半: + +```cpp +import mcpp; +#include +#include + +int main() { + // 第一半,无条件。 + mcpp::provides_pack_format("appimage"); + + // 第二半,有条件。 + if (std::string_view(mcpp::pack_format()) != "appimage") return 0; + + const std::string out = std::string(mcpp::out_dir()) + "/app.AppImage"; + mcpp::action a; + a.id = "appimage"; + a.role = "artifact"; + a.arg(tool).arg("${mcpp.stage_dir}").arg(out.c_str()) + .input("${mcpp.target_file:app}") + .output(out.c_str()) + .submit(); + return 0; +} +``` + +**无条件声明,有条件提交。** 声明是让引擎能回答一个发起请求的那次构建自己回答不了的 +问题:`--format bogus` 要点名**当下确实可用**的那些格式,`--help` 要说「解析后的图 +提供的任何格式」。两者读的都是一次「什么格式都没要」的 pass 收集到的集合。一个只在被 +问到时才声明的成员,对它的作者仍然照常工作 —— 作者永远传的是自己那个格式 —— 而对其他 +所有人,这个集合变成不可知的。对一个谁都没为之提交的格式,mcpp 会拒绝,而不是报告一次 +「什么包都没产出」的成功打包。 + +**`mcpp pack --format ` 会 prepare 两次。** 一条 `artifact` action 是一条 +ninja 边,而那棵暂存树是 mcpp 在链接**之后**产出的,所以这棵树不可能成为构建出它自己 +那一次 pass 的输入。第一次 pass 收集声明,并在任何东西被编译之前拒绝未知的格式;随后 +才是构建与暂存;第二次 pass 设上 `pack_format` 与 `pack_stage_dir`,并构建提供方提交 +的那条边。第二次 pass 里没有任何值是重新推导出来的 —— 三元组与暂存路径都是第一次 +pass 和那次暂存已经回答过的。 + +**暂存树是一个 bundle,不是一个根文件系统。** 它就是 `--mode vendored` 的含义: +`bin/`、`lib/`,可重定位、根在哪儿都行,并且已经过了 strip 策略、调试信息拆分与 +`include`/`exclude`。一个 AppImage、一个 `.app`、一个 `.msi` 要的就是它现在这个样子。 +而要一棵 FHS 树的格式(`.deb`、`.rpm`)自己负责重排布局,因为一个文件该落在哪个目录是 +那个格式的知识,不是引擎的。 + +`mcpp pack --format dir` 把同一棵树写到一个路径上就停下,这是人去查看一个成员将会拿到 +什么的方式。 + +**写了 `${mcpp.stage_dir}` 的 action 会自动获得一条对这棵树的 manifest 的依赖。** +mcpp 会写出 `<暂存树>.stage-manifest` —— 一个兄弟文件,永不是成员,所以它不会跑进任何 +人的安装包里 —— 逐条列出每个已暂存文件的大小与相对路径。这条依赖由引擎添加,因为「用 +了」本身就意味着「依赖」:没有它,这条边只在链接产物变化时才变脏,而一个闭包多出了某个 +依赖的共享库、同时程序自己的字节并没有变的情况,会把上一次的可分发物原地留下,并报告为 +已是最新。 + 命令是 **argv 而不是 shell 字符串**(不假设存在 shell —— Windows 没有能依赖的那个), 插值只有封闭的一组: @@ -479,6 +550,7 @@ mcpp 会播下一个带着该声明的占位文件,使 prepare 期的扫描与 | `${mcpp.bin_dir}` | 产出的二进制所在目录 | | `${mcpp.compile_db}` | `compile_commands.json` 的路径(clang-tidy 的 `-p` 要的就是它) | | `${mcpp.target_file:}` | target `` 构建出的文件 | +| `${mcpp.stage_dir}` *(2026.9.11.1+)* | `mcpp pack` 暂存出的那棵树,绝对路径。仅 `artifact` role 可用,且仅在 `mcpp pack --format ` 下可用 | 上面的裸 stdout 协议仍是底层基底;`import mcpp;` 是其上的类型化层。 @@ -575,6 +647,13 @@ mcpp 会把它自己构建时用的**同一份** std 模块暂存过来,缓存 | `MCPP_LANGUAGE_MODULES` *(2026.9.7.1+)* | -- | 声明它的那个包设了 `[language] modules` 时为 `1`,否则 `0`。**生成**面向消费者声明的规则读它来在模块接口与头文件之间选择,项目因此只需说一次。旧引擎不设这个变量,规则把缺席读作 `0` —— 也就是这个变量存在之前每个消费者的行为 | | `MCPP_PKG_NAME` *(2026.9.7.1+)* | -- | 这个程序所构建的包的 `[package] name`。规则生成的每个名字都由它推导:消费者导入的模块、访问器所在的命名空间、生成头里的符号。在它存在之前,可用的最接近的答案是 `MCPP_MANIFEST_DIR` 的末段,那是目录名 —— 于是一个叫 `vulkan-saxpy` 的包放在名为 `app` 的目录下会生成 `app.shaders`,而工作区里每一个 `/app/` 都声称拥有同一个模块。旧引擎下缺席,规则把缺席读作「沿用先前的推导」 | | `MCPP_PKG_NAMESPACE` *(2026.9.7.1+)* | -- | `[package] namespace`。包未声明命名空间时为空。需要产出在索引范围内唯一的名字的规则用这一对而不是单用名字,因为包身份是 `(namespace, name)` | +| `MCPP_PKG_VERSION` *(2026.9.11.1+)* | `mcpp::package_version()` | `[package] version`。每一种安装包格式都要写版本号;在这个变量之前,项目只能把版本号在成员自己的 options 里再写一遍,而那份副本会与 `[package]` 漂移,且没有任何东西能发现 | +| `MCPP_PKG_DESCRIPTION` *(2026.9.11.1+)* | `mcpp::package_description()` | `[package] description`。包未声明时为空 | +| `MCPP_PKG_LICENSE` *(2026.9.11.1+)* | `mcpp::package_license()` | `[package] license` | +| `MCPP_PKG_AUTHORS` *(2026.9.11.1+)* | `mcpp::package_authors()` | `[package] authors`,以 `;` 连接。不用 `,`:一条 author 的惯例写法是 `Name `,名字里可能带逗号,以逗号连接的列表无法再切回原来的条目 | +| `MCPP_PKG_REPO` *(2026.9.11.1+)* | `mcpp::package_repo()` | `[package] repo` | +| `MCPP_PACK_FORMAT` *(2026.9.11.1+)* | `mcpp::pack_format()` | 本程序所处的这次 `mcpp pack` 的 `--format` 取值;任何普通构建下都为空。承载含义的正是这个空值 —— 成员据此为自己的提交加闸,于是 `mcpp build` 拿到的还是它一直以来的那张图 | +| `MCPP_PACK_STAGE_DIR` *(2026.9.11.1+)* | `mcpp::pack_stage_dir()` | `mcpp pack` 已经把闭包暂存到的位置,绝对路径;本次构建不在打包时为空。读它来判断这次要干的活是什么形状,而把 `${mcpp.stage_dir}` 写进 action —— 这样图里的路径与程序读到的路径不可能不一致 | | `MCPP_DEVICE_SOURCES` *(2026.9.5.2+)* | `mcpp::device_sources()` | 本包有效 `sources` 匹配到的设备类源文件(`.cu`、`.hip`…),相对包根,一行一个;没有时为空串。引擎一个都不编译 —— 由本程序引入的规则包把每一个变成一条 `mcpp::action`。已经过收窄:构建未覆盖的 `{ glob, accel }` 条目贡献为空,因此 `--no-accel` 得到空列表 | | `MCPP_OUT_DIR` | `mcpp::out_dir()` | mcpp 提供的可写输出/暂存目录 | | `MCPP_MANIFEST_DIR` | `mcpp::manifest_dir()` | 包根(= CWD) | @@ -668,7 +747,16 @@ shim,而可用的那份就在项目自己的环境里,根本不在 `PATH` 上。 什么都不会坏;只是这个名字声称了一个该包并不具有的来源。项目之外的规则自选前缀。 -**工具不是规则。** 规则说明一个编译单元如何被 mcpp 并不驱动的编译器编译:它提交一条 +**工具不是规则,而可分发物两者都不是。** 三类活计,成员的前缀说明它回答三个问题中的 +哪一个: + +| 前缀 | 回答的问题 | 编译编译单元 | 在构建程序里执行 | +|---|---|---|---| +| `rules-*` | 这个编译单元如何被编译 | 是 | 否 | +| `tools-*` | 构建程序自己需要做什么 | 否 | 是 | +| `dist-*` *(2026.9.11.1+)* | 链接之后出来的是什么,以及用户以什么形态安装它 | 否 | 否 | + +规则说明一个编译单元如何被 mcpp 并不驱动的编译器编译:它提交一条 action,由引擎调度。工具说明的是构建程序需要、而没有任何编译器执行的事,并在构建程序 运行时当场做掉。`mcpp.tools.embed`(feature `tools-embed`,mcpp 2026.9.5.4+)是第一个: 它把数据文件写成程序编译进去的头文件(字节数组或 32 位字数组),内容未变时不重写文件, diff --git a/docs/zh/31-authoring-a-rule-package.md b/docs/zh/31-authoring-a-rule-package.md index fabeaa331..5b0060b73 100644 --- a/docs/zh/31-authoring-a-rule-package.md +++ b/docs/zh/31-authoring-a-rule-package.md @@ -269,6 +269,50 @@ mcpp::floor("cuda.driver", runtime_needs); 一个 option 暴露出来,好让用别的键声明这条边的消费者能提供它;并且在找不到时以 一条点名「找的是什么」的消息拒绝,而不是拿一个空路径去执行命令。 +## 编写一个分发成员(mcpp 2026.9.11.1+) + +一个 `dist-*` 成员既不是规则也不是工具。它不编译编译单元,也不在构建程序运行期间把活 +干完:它消费**链接产物**,产出用户去安装的东西 —— 一个 `.msi`、一个 `.deb`、一个 +AppImage、一个签过名的 `.app`。机制是一条 `artifact` action 加上 +`mcpp pack --format `,见 +[30 —— 产出可分发物](30-build-mcpp.md#产出可分发物pack_format-与-stage_dir20269111)。 + +本章其余内容原样适用。有六条在这里绑得更紧,每一条都是这个类别会犯、而规则不会犯的 +错。 + +**无条件声明,有条件提交。** `provides_pack_format` 是引擎用来回答「这张图提供哪些 +格式」的东西,而回答发生在一次「什么格式都没要」的构建上。一个只在被问到时才声明的 +成员,对它的作者照常工作 —— 作者永远传自己那个格式 —— 而对其他所有人,这个集合变成 +不可知的。 + +**同时提供 plan 与 submit。** 可分发物是交到用户手上之前的最后一环,因此它是整个构建 +里最可能需要项目自己改一笔的部分:换一个压缩等级、多带一个文件、加第二个签名。 +`generate_all(opt)` 恰好等于 `submit(plan_all(opt))`,正是让这样一笔改动不至于变成把 +成员重新实现一遍的东西。 + +**点名输入,不要去 harvest 一个目录。** 一个解析成空的路径是无声的,而一个缺失的具名 +输入是错误。实测:一条 WiX action 绑定一个目录并 harvest 它,当那个路径在 Windows 上 +解析成空时,产出了一个**有效的、空的、52 KB 的安装包,并且没有任何诊断**。 +`${mcpp.target_file:}` 就是这个机制 —— 构建程序既不知道三元组也不知道指纹,而一个 +不存在的 target 名会被拒绝,而不是展开成一个空路径。 + +**在成功路径上,为自己的输出断言一条下界。** 只要成员能判断自己的结果是空的或不合理 +的,它就必须经 `mcpp::warning` 说出来,因为成功构建的 stderr 会被丢弃。这与上一段实测到 +的是同一个失败,只是被拦在了后一层。 + +**在工具会被查找的那个位置声明它。** `xpkg_dir` 从 `MCPP_XPKG_*_DIR` 回答,而这些是 +mcpp 为**正在被构建的那个包**设置的。一个依赖的声明会把载荷装上,却不会让它对消费者的 +构建程序可见,所以运行载荷工具的成员要自己声明它 —— 并且在查找返回空时把这件事说出来, +而不是指向一份读者并不拥有的 manifest。 + +**构建不许碰网络,而被包起来的工具可能会碰。** 在 `appimagetool` 1.9.1 上实测:除非用 +`--runtime-file` 指定一份本地副本,它每次被调用都会从一个 GitHub release 下载它的 +type-2 runtime 存根。包装这类工具的成员必须从自己声明的载荷里把这个文件供上。安装期 +是下载合法的时候;构建期不是,而一个会去取东西的构建既不可复现也不能离线用。 + +**一个 `(name, version)` 只指一份载荷。** 包装签名或打包工具的成员继承了那个工具的兼容 +面,所以与被包装的工具同步版本是正当的,而且陈述了一件真事。 + ## 当前边界 - 规则包自己 `[features] default` 里的规则 feature 不隐含 `host-module`。 diff --git a/mcpp.toml b/mcpp.toml index 2dfc8efd0..4b3194b2e 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.9.10.2" +version = "2026.9.11.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/buildmcpp/src/directives.cppm b/modules/buildmcpp/src/directives.cppm index d0557d39b..5dc9dc742 100644 --- a/modules/buildmcpp/src/directives.cppm +++ b/modules/buildmcpp/src/directives.cppm @@ -135,6 +135,21 @@ enum class Slot : std::size_t { // vendor knowledge in the package that has it and out of the engine. Facts, Floors, + // A DISTRIBUTION FORMAT THIS PACKAGE PROVIDES (`mcpp:pack-format=`). + // + // `mcpp pack --format ` resolves `` through the graph the same + // way `--target` reaches a triple: the engine holds the DISPATCH and no + // format. The value is a bare name and means nothing to this file, which is + // what keeps dpkg's control fields, WiX's schema and Apple's notarisation + // out of an engine whose release would otherwise be coupled to theirs. + // + // COLLECTED FROM A BUILD THAT ASKED FOR NOTHING, which is why it is a slot + // and not a side effect of the request. `mcpp pack --format bogus` names + // what is available and `--help` says "plus any format the resolved graph + // provides"; both read this set on a pass where `MCPP_PACK_FORMAT` is + // empty. See `mcpp::provides_pack_format` for the author-facing half of the + // same rule -- declare unconditionally, submit conditionally. + PackFormats, Count }; inline constexpr std::size_t kSlotCount = static_cast(Slot::Count); @@ -217,7 +232,7 @@ struct Def { int sinceProtocol; }; -inline constexpr std::array kTable{{ +inline constexpr std::array kTable{{ // wire tag slot scope transform must missingPrefix missingSuffix since {"cxxflag", "cxxflag", Slot::CxxFlags, Scope::PackagePrivate, Transform::Verbatim, false, "", "", 1}, {"cflag", "cflag", Slot::CFlags, Scope::PackagePrivate, Transform::Verbatim, false, "", "", 1}, @@ -336,6 +351,20 @@ inline constexpr std::array kTable{{ // Slot::Facts for the shape of each value. {"fact", "fact", Slot::Facts, Scope::Claim, Transform::Verbatim, false, "", "", 7}, {"floor", "floor", Slot::Floors, Scope::Claim, Transform::Verbatim, false, "", "", 7}, + // `tag` IS NON-EMPTY FOR THE REASON `warning`'S IS, AND IT MATTERS MORE + // HERE. A build program's result is cached and a hit does not re-run it, so + // a declaration that lived only on the run path would be present on the + // first build of a project and absent on every later one -- and the pass + // that reads it is `mcpp pack`, which is never the first build. The set + // would then be empty exactly when a user asks for a format, and the + // refusal would name nothing. + // + // kCacheEpoch is NOT bumped. An entry written before this row carries no + // `d pack-format` line, and the program that wrote it could not emit one, + // so replaying it yields what that program said. An older engine reading a + // newer entry already discards the whole record through the unknown-tag + // path. + {"pack-format", "pack-format", Slot::PackFormats, Scope::Claim, Transform::Verbatim, false, "", "", 9}, }}; // ── Collected output of one run ──────────────────────────────────────────── @@ -813,6 +842,13 @@ void apply(mcpp::manifest::Manifest& m, const Directives& d) { m.runtimeConfig.requirements.push_back(std::move(req)); } + // A NAME, CARRIED AND NOT INTERPRETED. The engine compares it against + // `--format` and hands the request to whoever claimed it; nothing here + // parses it, because a distribution format has less claim to a name in the + // engine than a language does, and Slang is already supported without one. + for (auto const& f : d.at(Slot::PackFormats)) + bc.packFormats.push_back(f); + // Build-graph nodes. Decoded here rather than at parse time so the cache // stores the payload verbatim and a replay is byte-identical to a run. for (auto const& payload : d.at(Slot::Actions)) { diff --git a/modules/buildmcpp/src/program_protocol.cppm b/modules/buildmcpp/src/program_protocol.cppm index 7301dcfa7..501374f12 100644 --- a/modules/buildmcpp/src/program_protocol.cppm +++ b/modules/buildmcpp/src/program_protocol.cppm @@ -66,7 +66,12 @@ export namespace mcpp::build::program_protocol { // `--exclude-libs`) had no way out. Same cost as v5's: a package calling // `mcpp::link_flag()` fails on an older engine at the build.mcpp COMPILE, // because that engine's bundled module has no such function. -inline constexpr int kProtocolVersion = 8; +// v9: adds `pack-format` -- the outlet by which a package says which +// distribution format it provides, so `mcpp pack --format ` can dispatch +// to it. Same cost as v5's: a package calling `mcpp::provides_pack_format()` +// fails on an older engine at the build.mcpp COMPILE, because that engine's +// bundled module has no such function. +inline constexpr int kProtocolVersion = 9; // ── Cache-format epoch ───────────────────────────────────────────────────── // diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index b8b3718fa..cb2a9ef3f 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -385,6 +385,16 @@ struct BuildAction { // two are matched against each other. std::string packageName; Role role = Role::Source; + // Set by the engine, never by the build program: this action's command or + // inputs named `${mcpp.stage_dir}`. + // + // It is what makes the distributable ATTRIBUTABLE. `mcpp pack --format + // ` reports the file the pass produced, and the alternative -- taking + // every artifact action's output -- would name a codesign stamp or a size + // budget alongside it. It is also the flag the dispatch checks to refuse a + // member that declared a format and then submitted nothing for it, which + // would otherwise be a pack that succeeds and produces no package. + bool consumesStageDir = false; std::vector inputs; // absolute or package-relative std::vector outputs; // ditto; declared, see INV-D // Object only: which link units receive the outputs. Empty = every LINKED @@ -693,6 +703,17 @@ struct BuildConfig : BuildInputs { // (`mcpp:action=`). Empty for every package that does not use one, so an // ordinary build is untouched. std::vector actions; + // Distribution formats this package's build program declared it provides + // (`mcpp:pack-format=`). Empty for every package that ships no such member, + // so an ordinary build is untouched. + // + // THE ENGINE HOLDS THE DISPATCH AND NOT THE FORMAT. `mcpp pack --format + // ` looks the name up in the union of these lists, exactly as + // `--target` reaches a triple the engine did not have to know + // individually. `.deb`'s control fields, WiX's schema and Apple's + // notarisation each couple a release to a release mcpp does not control, + // and a name here is the whole of what the engine learns. + std::vector packFormats; bool staticStdlib = true; // #336 — the C++ runtime DISTRIBUTION contract: what the artifact promises // about the machine that runs it ("self-contained" | "toolchain-coupled" | diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index 6a73bf867..677aa8234 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.9.10.2"; +inline constexpr std::string_view MCPP_VERSION = "2026.9.11.1"; } // namespace mcpp diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index bd30891a7..b2f89602f 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -130,6 +130,47 @@ struct BuildProgramEnv { // way to ask. See hostprogram::package_name for what it replaced. std::string packageName; std::string packageNamespace; + // THE REST OF `[package]`, FOR THE MEMBER OF THE COLLECTION THAT NEEDS IT. + // + // A rule generates a declaration and needs the package's NAME. A member + // that produces a DISTRIBUTABLE needs more: every installer format carries + // a version, and most carry a description, a licence and a maintainer. + // Without these a project has to restate them in the member's options, + // where they can drift from `[package]` with nothing able to detect it -- + // the second copy of a value whose first copy mcpp has already parsed. + // + // `packageAuthors` is joined with ';' rather than ',' because an author + // entry is conventionally `Name ` and a name may carry a comma. + // Empty under an engine that predates these, which a member reads as "fall + // back to whatever you did before". + std::string packageVersion; + std::string packageDescription; + std::string packageLicense; + std::string packageAuthors; + std::string packageRepo; + // ── The packaging pass this build is part of (mcpp 2026.9.11.1+) ──────── + // + // Empty for every ordinary build, and that is the value that carries the + // meaning: a member which produces a distributable SUBMITS NOTHING unless + // the format it provides was asked for. `mcpp build` therefore has the + // graph it always had, and the dist edge exists only in the pass that + // wants it. + // + // The value is the `--format` argument verbatim -- `tar`, `dir`, or a name + // a package provides. It rides the same env vector as everything else here, + // so `contract_hash` folds it into the build program's re-run key: the + // second pass re-runs exactly the programs whose answer this changes. + std::string packFormat; + // Where `mcpp pack` has ALREADY STAGED the closure, absolute. Non-empty + // only in the second pass, and only then because a staged tree is produced + // by mcpp after the link -- so a graph generated before the link cannot + // name a directory that does not exist yet. + // + // This is the value `${mcpp.stage_dir}` expands to. A member reads it to + // decide the shape of the work (which of `bin/`, `lib/`, `share/` the tree + // actually has) and writes the placeholder into the action, so the two + // never disagree. + std::filesystem::path packStageDir; // Whether this package builds C++ modules (`[language] modules`). // // Reported because a rule package that GENERATES a consumer-facing @@ -530,6 +571,13 @@ contract_env(const fs::path& root, const fs::path& outDir, const BuildProgramEnv e.emplace_back("MCPP_MANIFEST_DIR", root.string()); e.emplace_back("MCPP_PKG_NAME", env.packageName); e.emplace_back("MCPP_PKG_NAMESPACE", env.packageNamespace); + e.emplace_back("MCPP_PKG_VERSION", env.packageVersion); + e.emplace_back("MCPP_PKG_DESCRIPTION", env.packageDescription); + e.emplace_back("MCPP_PKG_LICENSE", env.packageLicense); + e.emplace_back("MCPP_PKG_AUTHORS", env.packageAuthors); + e.emplace_back("MCPP_PKG_REPO", env.packageRepo); + e.emplace_back("MCPP_PACK_FORMAT", env.packFormat); + e.emplace_back("MCPP_PACK_STAGE_DIR", env.packStageDir.string()); std::string csv; for (auto const& f : env.features) { if (!csv.empty()) csv += ','; diff --git a/src/build/graph_shape.cppm b/src/build/graph_shape.cppm index f8581c8e7..679d2d043 100644 --- a/src/build/graph_shape.cppm +++ b/src/build/graph_shape.cppm @@ -62,11 +62,22 @@ std::string_view to_string(GraphShape shape) { // `mcpp build --no-accel`, `mcpp build` -- the third reported "Finished in // 0.00s" and `mcpp run` executed the CPU variant. A graph an override chose // says so, and the fast paths, which run only without overrides, decline it. +// `packFormat` records the DISTRIBUTION FORMAT this graph was generated for, +// and empty means "none" -- an ordinary build. It rides this line for exactly +// the reason the other two fields do, and it is the third instance of one +// failure: `mcpp pack --format appimage` makes a build program submit an +// artifact action a plain build must not have, and the two graphs land in the +// same directory because the format is deliberately NOT in the fingerprint +// (putting it there would cost a full recompile to package an already-built +// tree). So `pack --format X` then `build` would replay a graph carrying a dist +// edge, which is the `A then B then A` shape both other fields exist to stop. std::string header_line(GraphShape shape, std::string_view scheduleTag, - bool accelOverridden = false) { - return std::format("# mcpp:graph={};schedule={};accel={}", + bool accelOverridden = false, + std::string_view packFormat = {}) { + return std::format("# mcpp:graph={};schedule={};accel={};dist={}", to_string(shape), scheduleTag, - accelOverridden ? "override" : "default"); + accelOverridden ? "override" : "default", + packFormat.empty() ? std::string_view("none") : packFormat); } // Read the shape back. `nullopt` means "this file does not say" — a build.ninja @@ -162,13 +173,39 @@ std::string read_accel_selection(const std::filesystem::path& ninjaPath) { return {}; } -// A graph the fast paths may replay: the package's own targets, and the -// device variant the manifest names rather than one a flag chose. Both fast -// paths run only when no override is present, so a graph an override wrote is -// never the graph a plain build would produce. +// The distribution format this graph was generated for, or "none". Empty when +// the file predates the field, which callers treat as a miss for the reason +// read_shape gives. +std::string read_pack_format(const std::filesystem::path& ninjaPath) { + std::ifstream input(ninjaPath); + if (!input) return {}; + std::string line; + for (int i = 0; i < 8 && std::getline(input, line); ++i) { + constexpr std::string_view prefix = "# mcpp:graph="; + if (!line.starts_with(prefix)) continue; + auto value = std::string_view(line).substr(prefix.size()); + while (!value.empty() && (value.back() == '\r' || value.back() == ' ')) + value.remove_suffix(1); + constexpr std::string_view key = ";dist="; + const auto at = value.find(key); + if (at == std::string_view::npos) return {}; + auto rest = value.substr(at + key.size()); + if (const auto semi = rest.find(';'); semi != std::string_view::npos) + rest = rest.substr(0, semi); + return std::string(rest); + } + return {}; +} + +// A graph the fast paths may replay: the package's own targets, the device +// variant the manifest names rather than one a flag chose, and no distribution +// edge. All three fast-path callers run only for a plain build, so a graph any +// of the three axes was pointed at is never the graph a plain build would +// produce. bool is_plain_build_graph(const std::filesystem::path& ninjaPath) { return read_shape(ninjaPath) == GraphShape::Normal - && read_accel_selection(ninjaPath) == "default"; + && read_accel_selection(ninjaPath) == "default" + && read_pack_format(ninjaPath) == "none"; } } // namespace mcpp::build diff --git a/src/build/hostprogram.cppm b/src/build/hostprogram.cppm index 88e21b8d6..fed7c862b 100644 --- a/src/build/hostprogram.cppm +++ b/src/build/hostprogram.cppm @@ -124,6 +124,33 @@ inline void fact(const char* name, const char* version) { } inline void floor(const char* spec) { std::printf("mcpp:floor=%s\n", spec); } +// ── The distributable channel (mcpp 2026.9.11.1+) ─────────────────────────── +// +// `mcpp pack --format ` dispatches to whichever package provides +// ``, exactly as `--target` reaches a triple the engine did not have to +// know individually. This is how a package says which name it answers for: +// +// mcpp::provides_pack_format("appimage"); +// +// DECLARE UNCONDITIONALLY, SUBMIT CONDITIONALLY. The declaration must not be +// gated on `pack_format()`, and the reason is that the engine has to be able to +// answer a question the requesting build cannot: `mcpp pack --format bogus` +// names what IS available, and `--help` says "plus any format the resolved +// graph provides". Both read the set collected from this outlet, on a build +// that asked for nothing. A member that declared only when asked would still +// work for its author -- they always pass their own format -- and would make +// the set unknowable for everyone else. +// +// The work itself is the other half: +// +// if (std::string_view(mcpp::pack_format()) == "appimage") { ... submit ... } +// +// Two names are reserved for the engine's own archive shapes and are refused +// here: `tar` and `dir`. +inline void provides_pack_format(const char* name) { + std::printf("mcpp:pack-format=%s\n", name); +} + // The memory layout for a freestanding link. Reaches the CONSUMER's link line // (like link_lib/link_search, unlike include_dir), because the package that // knows a board's layout is not the package being built. @@ -390,6 +417,53 @@ inline const char* manifest_dir() { return env_or("MCPP_MANIFEST // rule package working unchanged. inline const char* package_name() { return env_or("MCPP_PKG_NAME"); } inline const char* package_namespace() { return env_or("MCPP_PKG_NAMESPACE"); } + +// THE REST OF `[package]`, BECAUSE A DISTRIBUTABLE CARRIES IT. +// +// `package_name()` above exists so a generated declaration can be named. These +// exist for the other member of the collection: every installer format states a +// version, and most state a description, a licence and a maintainer. A member +// without them has to ask the PROJECT to restate values mcpp has already +// parsed, in the member's own options, where the copy drifts from `[package]` +// and nothing can detect that it has. +// +// `package_authors()` is a ';'-separated list -- not ',', because an author is +// conventionally `Name ` and a name may carry a comma. +// +// Empty under an engine older than 2026.9.11.1. A member that needs one must +// say so itself when it is empty, naming the value it wanted: only the member +// knows whether the absence is fatal. +inline const char* package_version() { return env_or("MCPP_PKG_VERSION"); } +inline const char* package_description() { return env_or("MCPP_PKG_DESCRIPTION"); } +inline const char* package_license() { return env_or("MCPP_PKG_LICENSE"); } +inline const char* package_authors() { return env_or("MCPP_PKG_AUTHORS"); } +inline const char* package_repo() { return env_or("MCPP_PKG_REPO"); } + +// WHICH DISTRIBUTABLE THIS PASS WAS ASKED FOR, or "" for every ordinary build. +// +// The empty value is the one that carries the meaning: a member gates its +// submission on this, so `mcpp build` has the graph it always had and a dist +// edge exists only in the pass that wants one. See `provides_pack_format` for +// the half that must NOT be gated. +inline const char* pack_format() { return env_or("MCPP_PACK_FORMAT"); } + +// WHERE `mcpp pack` HAS ALREADY STAGED THE CLOSURE, absolute; "" when this +// build is not packing. +// +// The tree is what `mcpp pack` computes and then, until this existed, threw +// away: the dependency closure after the strip policy, the debug-symbol split +// and `include`/`exclude`. It is a BUNDLE tree -- `bin/`, `lib/`, relocatable, +// rooted anywhere -- which is what an AppImage, a `.app` and an `.msi` want as +// it stands. A format that wants a root filesystem instead (`.deb`, `.rpm`) +// owns the re-layout, because which directory a file belongs in is that +// format's knowledge and not the engine's. +// +// READ IT HERE TO DECIDE, WRITE `${mcpp.stage_dir}` INTO THE ACTION. The +// directory exists while this program runs, so a member enumerates it to learn +// which of `bin/`, `lib/`, `share/` the tree actually has; the action's command +// then names it through the placeholder, so the path in the graph and the path +// this program read cannot disagree. +inline const char* pack_stage_dir() { return env_or("MCPP_PACK_STAGE_DIR"); } inline bool has_feature(const char* name) { char buf[256] = "MCPP_FEATURE_"; unsigned long o = 13; diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 4ff138ce9..4f88f54af 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -626,7 +626,8 @@ std::string emit_ninja_string(const BuildPlan& plan) { // write this one file and the fast path has to know what it is about to // replay. Must stay within the first few lines — see read_shape. append(mcpp::build::header_line(plan.graphShape, plan.scheduleTag, - plan.accelOverridden) + "\n"); + plan.accelOverridden, + plan.packFormat) + "\n"); append("ninja_required_version = 1.11\n\n"); // All compile/link flags are computed once via flags.cppm. diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 34433a1f8..73dadccee 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -270,6 +270,20 @@ struct BuildPlan { // absolute and engine variables already substituted by the time they get // here, so the backend only has to spell edges. std::vector actions; + // The distribution formats the RESOLVED GRAPH provides, sorted and unique + // (`mcpp:pack-format=`). This is what `mcpp pack --format ` resolves + // against and what an unknown value's refusal names. + // + // Collected on EVERY prepare, including the pass that asked for no format + // at all -- which is the pass that has to answer "what is available". See + // `mcpp::provides_pack_format` for the author-facing rule that makes this + // possible: declare unconditionally, submit conditionally. + std::vector providedPackFormats; + // Non-empty when this prepare is the second pass of `mcpp pack --format + // `: the staged closure `${mcpp.stage_dir}` expanded to. Recorded on + // the plan so the graph header line can say which format wrote this graph, + // and the fast paths can decline to replay it for a plain build. + std::string packFormat; std::vector runtimeLibraryDirs; // ONLY the dependency packages' [runtime] library_dirs (not toolchain/ // payload dirs). These are the dirs that must be baked into the produced diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 69b9b47fb..ec0cc7c93 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -61,6 +61,7 @@ import mcpp.build.runtime_validation; // declared artifact -> identity verdict import mcpp.build.cache_key; import mcpp.pack.abi_tag; // the tag a prebuilt dependency is checked against import mcpp.pack.prebuilt; // …and the check itself +import mcpp.pack.stage_tree; // where `${mcpp.stage_dir}` points, and its manifest import mcpp.build.build_program; import mcpp.build.directives; // directive table: mark / fold_private_tail import mcpp.build.tool_store; // #355 host tools: store layout + key + overrides @@ -998,6 +999,24 @@ export struct BuildOverrides { // One says which PACKAGES enter the graph, the other which TOOLS are // installed, and `mcpp run` needs the second without the first. bool will_run = false; + // ── The packaging pass, when this prepare is one (mcpp 2026.9.11.1+) ──── + // + // `mcpp pack --format ` prepares TWICE, and these two fields are the + // whole difference between the passes. The first sets neither: build + // programs run, declare the formats they provide, and submit no dist + // action because none was asked for. The second sets both, after the link + // and after staging, so the claiming member submits an action whose input + // is a directory that by then exists. + // + // NEITHER VALUE IS DERIVED HERE, AND THAT IS THE POINT. `pack_stage_dir` is + // a function of the package name, the version, the resolved triple and the + // mode, and the resolved triple is not known until a prepare has run. + // Computing it a second time before prepare -- from the host triple, say -- + // is the shape where two derivations of one value agree on every machine + // the author has. Both are read out of what the first pass and `make_plan` + // already answered. + std::string pack_format; + std::filesystem::path pack_stage_dir; }; // ── git dependency helpers ────────────────────────────────────────────────── @@ -1134,6 +1153,34 @@ mcpp::platform::process::RunResult run_with_network_retry( return r; } +// `[package]`, for the build program of the package that declares it. +// +// ONE CALL RATHER THAN A FIELD PER SITE. Two places build a +// `BuildProgramEnv` -- the dependency loop and the root -- and the values a +// build program is told about its own package are the same question in both. +// Setting them field by field at each site is how the two answers drift: the +// root gained `packageName` and the dependency loop gained it separately, and +// a value added to only one of them is a rule package that works for a root +// project and not for a dependency, with nothing failing to say so. +void fill_package_build_env(mcpp::build::BuildProgramEnv& e, + const mcpp::manifest::Manifest& m) +{ + e.packageName = m.package.name; + e.packageNamespace = m.package.namespace_; + e.packageVersion = m.package.version; + e.packageDescription = m.package.description; + e.packageLicense = m.package.license; + e.packageRepo = m.package.repo; + // ';' rather than ',': an author entry is conventionally `Name ` + // and a name may carry a comma, so a comma-joined list cannot be split back + // into the entries it was made from. + e.packageAuthors.clear(); + for (auto const& a : m.package.authors) { + if (!e.packageAuthors.empty()) e.packageAuthors += ';'; + e.packageAuthors += a; + } +} + void fill_target_build_env(mcpp::build::BuildProgramEnv& e, const mcpp::toolchain::Toolchain* tc) { @@ -8198,8 +8245,9 @@ prepare_build(bool print_fingerprint, // The DECLARING package's setting, not the root project's: a rule // generating a declaration for this package must match how this // package is compiled. - bpEnv.packageName = pkg.manifest.package.name; - bpEnv.packageNamespace = pkg.manifest.package.namespace_; + fill_package_build_env(bpEnv, pkg.manifest); + bpEnv.packFormat = overrides.pack_format; + bpEnv.packStageDir = overrides.pack_stage_dir; bpEnv.languageModules = pkg.manifest.language.modules; bpEnv.ruleModules = pkg.manifest.buildConfig.ruleModules; if (auto dit = deviceSourcesByPackage.find(pkg.root.string()); dit != deviceSourcesByPackage.end()) @@ -9126,8 +9174,9 @@ prepare_build(bool print_fingerprint, bpEnv.toolsBin = projectSubosBin; bpEnv.profile = effectiveProfile; bpEnv.accel = resolvedAccel(); - bpEnv.packageName = m->package.name; - bpEnv.packageNamespace = m->package.namespace_; + fill_package_build_env(bpEnv, *m); + bpEnv.packFormat = overrides.pack_format; + bpEnv.packStageDir = overrides.pack_stage_dir; bpEnv.languageModules = m->language.modules; bpEnv.ruleModules = m->buildConfig.ruleModules; if (auto dit = deviceSourcesByPackage.find(root->string()); dit != deviceSourcesByPackage.end()) @@ -10242,7 +10291,29 @@ prepare_build(bool print_fingerprint, // become an edge with a blank path, and ninja reports that far away // from the typo that caused it. std::set unresolvedTargets; - auto substitute = [&](std::string s) { + // `${mcpp.stage_dir}` used where there is no staged tree, and used by an + // action whose role runs before the link. Both are refusals rather than + // empty expansions: an empty path is a token the command still accepts, + // and the tool then reads the build directory root -- which exists, so + // the mistake produces a plausible artifact instead of a diagnostic. + // Section 2 of the design record measured that shape: a valid, empty, + // 52 KB installer with nothing said about it. + std::set stageDirNoPass, stageDirWrongRole; + // WHETHER *THIS* ACTION REFERENCED THE STAGED TREE, and deliberately a + // flag rather than a set keyed on the action's id: an id is unique + // within the package that declared it and nothing more, so two packages + // may each submit a `dist` action called `package`. A set would then + // hand one package's implicit dependency to the other's edge -- the + // shape where a predicate is right and the object is wrong, which does + // not fail, it answers about something else. + // + // The diagnostic sets below stay keyed by id because a diagnostic + // NAMES ids and a collision there costs a duplicate line, not a wrong + // edge. + bool thisActionUsesStageDir = false; + const bool stagePass = !overrides.pack_stage_dir.empty(); + auto substitute = [&](std::string s, const char* actionId, + mcpp::manifest::BuildAction::Role role) { auto rep = [&](std::string_view what, const std::string& with) { for (std::size_t p; (p = s.find(what)) != std::string::npos; ) s.replace(p, what.size(), with); @@ -10250,6 +10321,21 @@ prepare_build(bool print_fingerprint, rep("${mcpp.out_dir}", ctx.plan.outputDir.string()); rep("${mcpp.bin_dir}", (ctx.plan.outputDir / "bin").string()); rep("${mcpp.compile_db}", ctx.plan.compileDbPath.string()); + // ABSOLUTE, unlike `${mcpp.target_file:}` and for the same reason + // stated the other way round: the staged tree lives outside the + // build directory and no ninja edge produces it, so there is no + // edge-declared spelling to agree with. `${mcpp.out_dir}` above is + // absolute on the same grounds. + if (s.find("${mcpp.stage_dir}") != std::string::npos) { + if (!stagePass) { + stageDirNoPass.insert(actionId); + } else if (role != mcpp::manifest::BuildAction::Role::Artifact) { + stageDirWrongRole.insert(actionId); + } else { + thisActionUsesStageDir = true; + } + rep("${mcpp.stage_dir}", overrides.pack_stage_dir.string()); + } constexpr std::string_view kTf = "${mcpp.target_file:"; for (std::size_t p; (p = s.find(kTf)) != std::string::npos; ) { auto close = s.find('}', p); @@ -10278,22 +10364,74 @@ prepare_build(bool print_fingerprint, // mcpp#534's ordering edge is scoped to this name. auto owner = mcpp::build::qualified_package_name(mm); for (auto a : mm.buildConfig.actions) { - for (auto& x : a.inputs) x = substitute(x); - for (auto& x : a.outputs) x = substitute(x); - for (auto& x : a.command) x = substitute(x); + thisActionUsesStageDir = false; + const auto sub = [&](std::string v) { + return substitute(std::move(v), a.id.c_str(), a.role); + }; + for (auto& x : a.inputs) x = sub(x); + for (auto& x : a.outputs) x = sub(x); + for (auto& x : a.command) x = sub(x); // Same closed vocabulary as outputs — a depfile commonly // wants to live at `${mcpp.out_dir}/.d`, beside the // output it describes, and `prepare_actions` above // deliberately left a `${mcpp.` depfile untouched for // exactly this phase to resolve. - if (!a.depfile.empty()) a.depfile = substitute(a.depfile); + if (!a.depfile.empty()) a.depfile = sub(a.depfile); + // THE DEPENDENCY IS IMPLIED BY THE USE, so a member author + // cannot forget it. Without this the edge is dirty only when a + // link output changes, and a staged set that grew a dependency's + // shared library while the program's own bytes did not would + // leave the previous distributable in place, reported as + // up to date. + if (thisActionUsesStageDir) { + a.consumesStageDir = true; + a.inputs.push_back( + mcpp::pack::stage_manifest_path(overrides.pack_stage_dir).string()); + } a.packageName = owner; ctx.plan.actions.push_back(std::move(a)); } + // Every package's declaration, on every pass. Sorted and de-duplicated + // below so the refusal's list reads the same whatever order resolution + // walked the graph in. + for (auto const& f : mm.buildConfig.packFormats) + ctx.plan.providedPackFormats.push_back(f); }; collect(*m); for (std::size_t i = 1; i < packages.size(); ++i) collect(packages[i].manifest); + std::ranges::sort(ctx.plan.providedPackFormats); + ctx.plan.providedPackFormats.erase( + std::ranges::unique(ctx.plan.providedPackFormats).begin(), + ctx.plan.providedPackFormats.end()); + ctx.plan.packFormat = overrides.pack_format; + if (!stageDirNoPass.empty()) { + std::string ids; + for (auto const& n : stageDirNoPass) ids += (ids.empty() ? "" : ", ") + n; + return std::unexpected(std::format( + "build.mcpp action(s) [{}] reference ${{mcpp.stage_dir}}, and this " + "build is not packaging.\n" + " The staged tree is produced by `mcpp pack` after the link, so " + "it does not exist during\n" + " a plain build and there is nothing for the placeholder to name.\n" + " Gate the submission on the format you provide:\n" + " mcpp::provides_pack_format(\"\"); // always\n" + " if (std::string_view(mcpp::pack_format()) == \"\") " + "// then submit\n" + " and reach the tree with `mcpp pack --format `.", ids)); + } + if (!stageDirWrongRole.empty()) { + std::string ids; + for (auto const& n : stageDirWrongRole) ids += (ids.empty() ? "" : ", ") + n; + return std::unexpected(std::format( + "build.mcpp action(s) [{}] reference ${{mcpp.stage_dir}} with a role " + "other than \"artifact\".\n" + " Only an artifact action runs after the link, and the staged tree " + "is a link output's\n" + " successor: a source, object or check action is scheduled before " + "there is anything to stage.\n" + " use: role = \"artifact\"", ids)); + } if (!unresolvedTargets.empty()) { std::string bad, known; for (auto const& n : unresolvedTargets) bad += (bad.empty() ? "" : ", ") + n; diff --git a/src/cli.cppm b/src/cli.cppm index a3e4db36a..c45023f2f 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -581,8 +581,14 @@ int run(int argc, char** argv) { .help("system | vendored (default) | self-contained | static")) .option(cl::Option("target").takes_value().multiple() .help("Triple, e.g. x86_64-linux-musl (repeatable: one leg per triple)")) + // "plus any format the resolved graph provides", because the + // values are no longer a fixed list. A package declares one with + // `mcpp::provides_pack_format` and `--format ` dispatches to + // it; an unknown value names what IS available rather than a + // constant, so the help text does not have to enumerate them. .option(cl::Option("format").takes_value() - .help("tar (default; .zip for a Windows target) | dir")) + .help("tar (default; .zip for a Windows target) | dir | any " + "format the resolved graph provides (e.g. appimage, msi)")) .option(cl::Option("output").short_name('o').takes_value() .help("Override output path")) // Packaging builds RELEASE by default — the artifact leaves this diff --git a/src/cli/cmd_publish.cppm b/src/cli/cmd_publish.cppm index d6da52da7..216232338 100644 --- a/src/cli/cmd_publish.cppm +++ b/src/cli/cmd_publish.cppm @@ -47,13 +47,25 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { opts.mode = *m; modeFromUser = true; } + // THE VALUE IS NOT VALIDATED HERE, AND THAT IS THE CHANGE. + // + // `tar` and `dir` are the archive shapes the engine owns. Everything else + // is a name a package provides, and which names those are is a property of + // the RESOLVED GRAPH -- so a refusal written here could only compare + // against a constant, which is exactly the coupling this whole mechanism + // exists to remove. The refusal moves to `build_and_pack`, after build + // programs have declared what they provide and before anything is + // compiled, where it can name what IS available instead of a fixed list. if (auto v = parsed.value("format")) { if (*v == "tar") opts.format = mcpp::pack::Format::Tar; else if (*v == "dir") opts.format = mcpp::pack::Format::Dir; - else { - mcpp::ui::error(std::format( - "invalid --format '{}'; expected tar | dir", *v)); + else if (v->empty()) { + mcpp::ui::error("--format needs a value: tar | dir | a format the " + "resolved graph provides"); return 2; + } else { + opts.format = mcpp::pack::Format::Dispatched; + opts.formatName = *v; } } if (auto v = parsed.value("output")) opts.output = *v; @@ -85,6 +97,23 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { auto route = mcpp::pack::route_pack_target(parsed.positional(0)); if (!route) { mcpp::ui::error(route.error()); return 2; } if (route->library) { + // A DISPATCHED FORMAT IS AN APPLICATION-BUNDLE OUTPUT, and a library + // package has no bundle: `mcpp pack ` produces an interface plus + // prebuilt binaries for one or more triples, and there is no single + // staged tree for a member to turn into an installer. Refused rather + // than ignored, because ignoring it would report `Packed` and hand back + // a library package while the user asked for an installer. + if (opts.format == mcpp::pack::Format::Dispatched) { + mcpp::ui::error(std::format( + "--format {} is a distributable produced from a program's staged " + "bundle, and '{}' is a library target.\n" + " A library package ships an interface plus prebuilt binaries " + "per triple; there is no\n" + " single staged tree to hand a distribution format.\n" + " use: --format tar | dir, or name a program target", + opts.formatName, route->targetName)); + return 2; + } if (modeFromUser) { mcpp::ui::warning(std::format( "--mode is an application-bundle depth and does not apply to the " diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index 413800c53..6b62d2919 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -50,11 +50,33 @@ export namespace mcpp::pack { enum class Mode { None, Static, BundleProject, BundleAll }; -enum class Format { Tar, Dir }; +// WHAT SHAPE THE OUTPUT TAKES, and the third value is the one that is not a +// shape the engine knows. +// +// `tar` and `dir` answer the same question `msi` and `appimage` answer, so they +// belong on one axis -- which is why this is a wider set of values for one flag +// rather than a second flag. `Dispatched` carries a name the engine has never +// heard: `mcpp pack --format appimage` finds the provider among the resolved +// dependencies and hands it the staged tree, exactly as `--target` reaches a +// triple the engine did not have to know individually. +// +// The engine keeps `Tar` and `Dir` because an archive that extracts and runs is +// universal in the only sense that matters here: it needs no knowledge of +// anyone else's release. dpkg's control fields, AppImage's runtime, WiX's +// schema and Apple's notarisation each couple an mcpp release to a release mcpp +// does not control. +enum class Format { Tar, Dir, Dispatched }; struct Options { Mode mode = Mode::BundleProject; Format format = Format::Tar; + // The `--format` value when `format == Dispatched`. Empty otherwise. + // + // Not validated by the CLI, and deliberately: the set of valid values is a + // property of the RESOLVED GRAPH, so the refusal has to wait until build + // programs have declared what they provide. It arrives before anything is + // compiled, which is the earliest point at which it can be exact. + std::string formatName; std::filesystem::path output; // empty = derive from manifest std::string targetTriple; // empty = host // Where a dependency NAME may be resolved to a file. diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index bebbe7629..65608f2d4 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -18,6 +18,7 @@ import mcpp.build.plan; import mcpp.config; import mcpp.fetcher.progress; import mcpp.pack; +import mcpp.pack.stage_tree; import mcpp.pack.strip; import mcpp.toolchain.model; import mcpp.toolchain.registry; @@ -83,15 +84,66 @@ export int build_and_pack(Options opts, bool modeFromUser, && opts.targetTriple.empty() && ctx->tc.targetTriple.find("-musl") == std::string::npos) { // Need to re-prepare the build with the musl target. - mcpp::build::BuildOverrides ov2; - ov2.target_triple = "x86_64-linux-musl"; - ov2.profile = opts.profile; - ov2.profile_fallback = "release"; - auto ctx2 = mcpp::build::prepare_build(false, false, {}, ov2); + // + // `ov` IS MUTATED RATHER THAN SHADOWED. It has to stay the record of + // what produced `ctx`, because the dispatch pass below re-enters + // prepare with the same overrides plus two fields -- and a second + // overrides object left behind here would make that pass differ from + // this build in a way nothing states. + ov.target_triple = "x86_64-linux-musl"; + auto ctx2 = mcpp::build::prepare_build(false, false, {}, ov); if (!ctx2) { mcpp::ui::error(ctx2.error()); return 2; } ctx = std::move(ctx2); } + // ─── Is the requested format one anything provides? ────────────── + // + // BEFORE THE BUILD, because a refusal that arrives after a full compile is + // a worse refusal, and because this is the earliest point at which it can + // be exact: build programs have now run and declared what they provide. + // + // The set is read from a pass that asked for NOTHING. That is what the + // "declare unconditionally, submit conditionally" rule buys -- a member + // that declared only when asked would leave this list empty exactly when a + // user names a format, and the refusal would name nothing. + if (opts.format == mcpp::pack::Format::Dispatched) { + auto const& provided = ctx->plan.providedPackFormats; + if (std::ranges::find(provided, opts.formatName) == provided.end()) { + std::string avail; + for (auto b : mcpp::pack::kBuiltinPackFormats) + avail += (avail.empty() ? "" : ", ") + std::string(b); + for (auto const& f : provided) { + if (mcpp::pack::is_builtin_pack_format(f)) continue; + avail += ", " + f; + } + mcpp::ui::error(std::format( + "unknown --format '{}'.\n" + " available in this build: {}\n" + " A format past `tar` and `dir` comes from a package in the " + "resolved graph, which declares\n" + " it with `mcpp::provides_pack_format(\"\")` in its build " + "program. Add the package\n" + " that provides '{}' to [build-dependencies] and activate its " + "feature.", + opts.formatName, avail, opts.formatName)); + return 2; + } + } + + // A package claiming a built-in name is silently unreachable, since the + // parser resolves `tar` and `dir` before consulting the graph at all. + // + // OUTSIDE THE DISPATCH BRANCH ABOVE, because the mistake is in the PACKAGE + // and does not depend on what this invocation asked for. Reported on every + // pack, so the author hears it on the plain `mcpp pack` they are most + // likely to run. + for (auto const& f : ctx->plan.providedPackFormats) + if (mcpp::pack::is_builtin_pack_format(f)) + mcpp::ui::warning(std::format( + "a package in this graph declares `mcpp:pack-format={}`, which " + "is one of the archive shapes `mcpp pack` owns; `--format {}` " + "will always select the built-in and never that package", f, f)); + auto be = mcpp::build::make_ninja_backend(); mcpp::build::BuildOptions bo; auto br = be->build(ctx->plan, bo); @@ -219,7 +271,108 @@ export int build_and_pack(Options opts, bool modeFromUser, return 1; } + // The staged tree is now on disk and final -- past the closure, the + // `$ORIGIN` rewriting, the strip and the debug split. Describe it, so an + // action that consumes it has something whose CONTENT changes when the + // staged set does. Best-effort: see write_stage_manifest. + mcpp::pack::write_stage_manifest(plan->stagingRoot); + auto pathCtx = mcpp::fetcher::make_path_ctx(&*cfg, ctx->projectRoot); + + // ─── The dispatch pass ─────────────────────────────────────────── + // + // A `role = "artifact"` action is a ninja edge, and the staged tree is + // produced here, in C++, AFTER ninja has finished. So an artifact action + // cannot depend on the staged tree in the pass that built it, and a + // single-pass `--format ` is not expressible. Two passes are, and + // every value this one needs was answered by the first: + // + // `ov` the overrides that produced the build above + // `plan->stagingRoot` from make_plan, which resolved the triple + // `opts.formatName` the request, already checked against the graph + // + // NOTHING IS RE-DERIVED, and that is the whole discipline of this block. + // `stagingRoot` is a function of the package name, the version, the + // resolved triple and the mode; the resolved triple is not known until a + // prepare has run, so computing it a second time before prepare -- from the + // host triple, say -- is the shape where two derivations of one value agree + // on every machine the author has and disagree on one they do not. + if (opts.format == mcpp::pack::Format::Dispatched) { + ov.pack_format = opts.formatName; + ov.pack_stage_dir = plan->stagingRoot; + auto distCtx = mcpp::build::prepare_build(false, false, {}, ov); + if (!distCtx) { mcpp::ui::error(distCtx.error()); return 2; } + + // WHICH ACTIONS ARE THE DISTRIBUTABLE. Only those that named + // `${mcpp.stage_dir}`: a codesign stamp or a size budget is also an + // artifact action, and reporting one as the package would be a wrong + // answer that looks like a right one. + std::vector distOutputs; + for (auto const& a : distCtx->plan.actions) { + if (!a.consumesStageDir) continue; + for (auto const& o : a.outputs) distOutputs.push_back(o); + } + // DECLARED AND THEN SUBMITTED NOTHING. The half of the contract a + // member is most likely to get wrong is the gate, and a member whose + // gate never opens leaves a pass that succeeds and produces no + // package. Refused by name rather than reported as success. + if (distOutputs.empty()) { + mcpp::ui::error(std::format( + "no action claimed --format '{}'.\n" + " A package declared it provides this format, and no build " + "program submitted an\n" + " artifact action referencing ${{mcpp.stage_dir}} when it was " + "asked for.\n" + " The provider must gate on the request and not on anything " + "else:\n" + " mcpp::provides_pack_format(\"{}\"); " + "// always\n" + " if (std::string_view(mcpp::pack_format()) == \"{}\") ..." + " // then submit", + opts.formatName, opts.formatName, opts.formatName)); + return 1; + } + + mcpp::ui::info("Distributing", std::format("{} v{} (--format {})", + plan->packageName, plan->packageVersion, opts.formatName)); + + // NO EXPLICIT GOALS. Everything but the dist edges is already up to + // date from the build above, so a full drive costs a graph scan and + // nothing else -- and an explicit goal set is how the 0.0.104 soname + // aliases went missing, because an edge reachable only through + // `default` is skipped under one. + mcpp::build::BuildOptions dbo; + auto dr = be->build(distCtx->plan, dbo); + if (!dr) { + if (!dr.error().diagnosticOutput.empty()) { + std::fputs(dr.error().diagnosticOutput.c_str(), stderr); + if (dr.error().diagnosticOutput.back() != '\n') std::fputs("\n", stderr); + } + mcpp::ui::error(dr.error().message); + return 1; + } + + // THE CRITERION IS THE FILE, NOT THE EXIT CODE. A cached build program + // replaying the first pass's answer, or a tool that writes nothing and + // exits 0, both leave ninja reporting success -- and section 2's + // measured failure was a packaging step that succeeded while carrying + // nothing. + std::error_code ec; + for (auto const& o : distOutputs) { + auto abs = std::filesystem::path(o).is_absolute() + ? std::filesystem::path(o) : distCtx->plan.outputDir / o; + if (!std::filesystem::is_regular_file(abs, ec) + && !std::filesystem::is_directory(abs, ec)) { + mcpp::ui::error(std::format( + "--format {} reported success and produced nothing at {}", + opts.formatName, abs.string())); + return 1; + } + mcpp::ui::status("Packed", mcpp::ui::shorten_path(abs, pathCtx)); + } + return 0; + } + auto outPath = (opts.format == mcpp::pack::Format::Tar) ? plan->archivePath : plan->stagingRoot; mcpp::ui::status("Packed", mcpp::ui::shorten_path(outPath, pathCtx)); diff --git a/src/pack/stage_tree.cppm b/src/pack/stage_tree.cppm new file mode 100644 index 000000000..b24f25241 --- /dev/null +++ b/src/pack/stage_tree.cppm @@ -0,0 +1,125 @@ +// mcpp.pack.stage_tree — what a staged tree promises an artifact action, and +// the one file that says so. +// +// `mcpp pack` has always computed a staged tree: the dependency closure after +// the strip policy, the debug-symbol split and `include`/`exclude`. Until +// `${mcpp.stage_dir}` it then compressed the tree and the directory was gone, +// so a `.deb`, an AppImage, a `.app` and an `.msi` each had to rebuild the same +// closure. Exposing it is one addition that serves every format and encodes no +// format's knowledge, which is the test for whether something belongs in the +// engine at all. +// +// WHY THERE IS A MANIFEST FILE AND NOT JUST A DIRECTORY. ninja identifies an +// input by a path and compares an mtime. A directory's mtime moves when its +// immediate entries change and not when a file two levels down is replaced, so +// naming the directory as an input would make the dist edge dirty for the wrong +// reasons and clean for the wrong reasons. The manifest is CONTENT-BEARING — +// one ` ` line per staged file, sorted — so it changes +// exactly when the staged set or any staged file's length changes, and it is a +// single ordinary file that ninja can compare. +// +// IT IS A SIBLING OF THE TREE, NOT A MEMBER OF IT. A file inside the staged +// directory would be collected by every format that packages the directory +// wholesale, and would then ship inside the user's installer. The sibling +// spelling is what keeps a mechanism the engine added from appearing in a +// product it does not own. +// +// The manifest deliberately records SIZES rather than content hashes. The tree +// can be hundreds of megabytes and is rebuilt on every pack; hashing it would +// make the common case pay for a distinction the uncommon case does not need, +// because a staged file whose length is unchanged and whose bytes differ can +// only have come from a rebuild, and a rebuild moved the link output that the +// dist edge also depends on. + +module; +#include + +export module mcpp.pack.stage_tree; + +import std; + +export namespace mcpp::pack { + +// The manifest that describes the tree staged at `stagingRoot`. +// +// A SIBLING, DERIVED FROM THE NAME RATHER THAN PLACED INSIDE. Spelled in one +// function because two readers need the same answer for different reasons: +// `prepare` names it as an implicit input while the file does not yet exist, +// and `pack::run` writes it. A second derivation of a path is a path that +// disagrees on the platform whose separator the author did not test. +std::filesystem::path stage_manifest_path(const std::filesystem::path& stagingRoot) { + auto p = stagingRoot; + p += ".stage-manifest"; + return p; +} + +// Write the manifest for the tree now on disk at `stagingRoot`. +// +// Best-effort by construction and deliberately so: the manifest is a +// dependency-tracking convenience, and a pack that produced a correct tree must +// not fail because a sibling bookkeeping file could not be written. A missing +// manifest makes the dist edge fail with ninja's own "missing and no known rule +// to make it", which names the file — a legible failure rather than a silent +// staleness. +bool write_stage_manifest(const std::filesystem::path& stagingRoot) { + std::error_code ec; + if (!std::filesystem::is_directory(stagingRoot, ec)) return false; + + std::vector lines; + for (auto const& entry : + std::filesystem::recursive_directory_iterator( + stagingRoot, std::filesystem::directory_options::skip_permission_denied, ec)) + { + if (ec) break; + // Symlinks are recorded by NAME AND NOT FOLLOWED. `bundle-all` + // dereferences a soname link while staging, so what remains is a real + // file; a link that survives points outside the tree, and following it + // would make the manifest describe a file the package does not carry. + if (!entry.is_regular_file(ec) || entry.is_symlink()) { + if (entry.is_symlink()) + lines.push_back(std::format("link {}", + std::filesystem::relative(entry.path(), stagingRoot, ec).generic_string())); + continue; + } + auto rel = std::filesystem::relative(entry.path(), stagingRoot, ec).generic_string(); + if (ec || rel.empty()) continue; + lines.push_back(std::format("{} {}", + std::filesystem::file_size(entry.path(), ec), rel)); + } + // Sorted, because a directory iteration order is not a promise. Two packs + // of one tree must produce identical bytes or the dist edge is dirty on + // every run for no reason. + std::ranges::sort(lines); + + std::string text; + for (auto const& l : lines) { text += l; text.push_back('\n'); } + + auto out = stage_manifest_path(stagingRoot); + // Compared before writing, for the reason `mcpp.build.stage` gives at + // length: rewriting identical bytes moves the mtime, and a moved mtime on + // an input is indistinguishable from a changed input. A pack that staged + // the same tree twice would rebuild the distributable both times. + if (std::ifstream in(out, std::ios::binary); in) { + std::string old((std::istreambuf_iterator(in)), + std::istreambuf_iterator()); + if (old == text) return true; + } + std::ofstream os(out, std::ios::binary | std::ios::trunc); + if (!os) return false; + os.write(text.data(), static_cast(text.size())); + return static_cast(os); +} + +// The two names `--format` answers for without consulting the graph. +// +// Held here rather than in the CLI because two layers need the same list: the +// parser decides whether a value is a built-in or a dispatch, and the +// declaration check refuses a package that claims one of them. A package +// claiming `tar` would be silently unreachable, since the built-in wins. +constexpr std::array kBuiltinPackFormats{"tar", "dir"}; + +bool is_builtin_pack_format(std::string_view name) { + return std::ranges::find(kBuiltinPackFormats, name) != kBuiltinPackFormats.end(); +} + +} // namespace mcpp::pack diff --git a/tests/e2e/638_pack_format_dispatch.sh b/tests/e2e/638_pack_format_dispatch.sh new file mode 100755 index 000000000..3ef21d1b7 --- /dev/null +++ b/tests/e2e/638_pack_format_dispatch.sh @@ -0,0 +1,245 @@ +#!/usr/bin/env bash +# requires: gcc +# 638_pack_format_dispatch.sh — `mcpp pack --format ` dispatches to a +# package, and no distribution format lives in the engine. +# +# `tar` and `dir` answer the same question `msi` and `appimage` answer -- what +# shape does the output take -- so they are values of ONE flag, and everything +# past the two the engine owns comes from the resolved graph. What the engine +# adds is the mechanism: a staged tree an artifact action can consume, the rest +# of `[package]` in the build program, and the dispatch itself. +# +# The four properties this holds, each with the wrong answer it excludes: +# +# 1. DECLARE UNCONDITIONALLY, SUBMIT CONDITIONALLY. A build that asks for no +# format still declares one, so `--format bogus` can name what IS +# available. A member that declared only when asked works for its author +# and makes the set unknowable for everyone else. +# 2. THE STAGED TREE IS REAL WHEN THE ACTION RUNS. Asserted from INSIDE the +# action, by listing the tree into the output -- not by the action's exit +# code, because a command that writes nothing and exits 0 is the measured +# failure this whole mechanism exists to prevent. +# 3. A PLAIN BUILD HAS NO DISTRIBUTION EDGE, before or after a pack. +# 4. THE PLACEHOLDER REFUSES OUTSIDE A PACKAGING PASS, rather than expanding +# to an empty string that the tool would accept. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +# ── The provider: one build program, both halves of the contract ─────────── +mkdir -p app/src +cd app + +cat > mcpp.toml <<'EOF' +[package] +name = "app" +version = "2.5.0" +description = "a program that ships" +license = "Apache-2.0" +authors = ["Ada "] +repo = "https://example.org/app" + +[targets.app] +kind = "bin" +main = "src/main.cpp" +EOF + +cat > src/main.cpp <<'EOF' +#include +int main() { std::puts("app"); return 0; } +EOF + +# The dist step. A separate script because an action's command is an argv with +# no shell assumed, which is the same reason 188_build_actions.sh writes one. +# +# It writes the package VERSION and the staged tree's top level into its +# output, so the assertions below read what the action actually saw rather than +# whether it exited 0. +cat > dist.sh <<'EOF' +#!/usr/bin/env bash +set -e +version="$1"; stage="$2"; out="$3" +{ + echo "version=$version" + echo "staged:" + ls -1 "$stage" | sort +} > "$out" +EOF +chmod +x dist.sh + +cat > build.mcpp <<'EOF' +import mcpp; +#include +#include +#include +int main() { + // Half one: unconditional. This is what makes the set knowable on a build + // that asked for nothing. + mcpp::provides_pack_format("zap"); + + // Half two: conditional. A plain build must have no such edge. + if (std::string_view(mcpp::pack_format()) != "zap") return 0; + + const std::string root = mcpp::manifest_dir(); + const std::string out = std::string(mcpp::out_dir()) + "/app.zap"; + mcpp::action a; + a.id = "zap"; + a.role = "artifact"; + a.description = "zap"; + a.arg((root + "/dist.sh").c_str()) + // The VERSION comes from `[package]`, not from an option the project + // restates: a second copy drifts with nothing able to detect it. + .arg(mcpp::package_version()) + .arg("${mcpp.stage_dir}") + .arg(out.c_str()) + .input("${mcpp.target_file:app}") + .output(out.c_str()) + .submit(); + return 0; +} +EOF + +MCPP="${MCPP:-mcpp}" + +graph_dist() { sed -n '2p' "$1" | sed 's/.*;dist=//;s/;.*//'; } +find_graph() { find target -name build.ninja | head -1; } + +# ── 1. a plain build declares, and submits nothing ───────────────────────── +"$MCPP" build --release > b1.log 2>&1 || { cat b1.log; echo "FAIL: plain build failed"; exit 1; } +G=$(find_graph) +[ -n "$G" ] || { echo "FAIL: no build.ninja"; exit 1; } +[ "$(graph_dist "$G")" = "none" ] \ + || { sed -n '2p' "$G"; echo "FAIL: a plain build's graph is not dist=none"; exit 1; } +[ -z "$(find target -name 'app.zap' 2>/dev/null)" ] \ + || { echo "FAIL: a plain build produced a distributable"; exit 1; } + +# ── 2. an unknown format is refused, BEFORE the build, naming what exists ── +set +e +"$MCPP" pack --format bogus > b2.log 2>&1 +rc=$? +set -e +[ "$rc" -ne 0 ] || { cat b2.log; echo "FAIL: --format bogus was accepted"; exit 1; } +grep -q "unknown --format 'bogus'" b2.log \ + || { cat b2.log; echo "FAIL: the refusal does not name the value"; exit 1; } +# THE LIST IS THE POINT. A refusal that named a fixed list would be the +# coupling this mechanism removes; this one names the graph's own answer. +grep -q "available in this build: tar, dir, zap" b2.log \ + || { cat b2.log; echo "FAIL: the refusal does not name what is available"; exit 1; } +# Nothing was compiled to find that out. +grep -q "Compiling app" b2.log \ + && { cat b2.log; echo "FAIL: the refusal arrived after a compile"; exit 1; } + +# ── 3. the dispatched format produces a file, and it saw the staged tree ─── +"$MCPP" pack --format zap > b3.log 2>&1 || { cat b3.log; echo "FAIL: pack --format zap failed"; exit 1; } +Z=$(find target -name 'app.zap' | head -1) +[ -n "$Z" ] || { cat b3.log; echo "FAIL: --format zap produced nothing"; exit 1; } +# The engine handed the build program the rest of `[package]`. +grep -qx "version=2.5.0" "$Z" \ + || { cat "$Z"; echo "FAIL: the action was not told the package version"; exit 1; } +# The action ran with a staged tree that already held the program. This is the +# assertion the ordering exists for: an artifact edge is scheduled by ninja and +# the tree is staged by mcpp after the link, so a single-pass design would run +# this against a directory that does not exist. +grep -qx "bin" "$Z" \ + || { cat "$Z"; echo "FAIL: the staged tree had no bin/ when the action ran"; exit 1; } +grep -q "Packed" b3.log \ + || { cat b3.log; echo "FAIL: the produced file was not reported"; exit 1; } +# The staged tree is described by a SIBLING of itself, never a member: a file +# inside it would ship inside every format that packages the directory. +[ -n "$(find target/dist -maxdepth 1 -name '*.stage-manifest' 2>/dev/null)" ] \ + || { echo "FAIL: no stage manifest beside the staged tree"; exit 1; } +[ -z "$(find target/dist -mindepth 2 -name '*.stage-manifest' 2>/dev/null)" ] \ + || { echo "FAIL: the stage manifest landed inside the staged tree"; exit 1; } + +# ── 4. a plain build AFTER the pack still has no distribution edge ───────── +rm -f "$Z" +"$MCPP" build --release > b4.log 2>&1 || { cat b4.log; echo "FAIL: build after pack failed"; exit 1; } +[ "$(graph_dist "$(find_graph)")" = "none" ] \ + || { echo "FAIL: the graph still says dist= after a plain build"; exit 1; } +[ ! -f "$Z" ] \ + || { echo "FAIL: a plain build regenerated the distributable"; exit 1; } + +# ── 5. the placeholder outside a packaging pass is refused ───────────────── +# The same action, ungated. An empty expansion would give the tool the build +# directory root, which exists -- so the mistake would produce a plausible +# artifact instead of a diagnostic. +cd "$TMP" +cp -r app ungated +cd ungated +cat > build.mcpp <<'EOF' +import mcpp; +#include +int main() { + const std::string root = mcpp::manifest_dir(); + const std::string out = std::string(mcpp::out_dir()) + "/app.zap"; + mcpp::action a; + a.id = "zap"; a.role = "artifact"; + a.arg((root + "/dist.sh").c_str()).arg("x").arg("${mcpp.stage_dir}").arg(out.c_str()) + .input("${mcpp.target_file:app}").output(out.c_str()).submit(); + return 0; +} +EOF +set +e +"$MCPP" build --release > b5.log 2>&1 +rc=$? +set -e +[ "$rc" -ne 0 ] || { cat b5.log; echo "FAIL: an ungated stage_dir built"; exit 1; } +grep -q "this build is not packaging" b5.log \ + || { cat b5.log; echo "FAIL: the refusal does not say why"; exit 1; } +grep -q "provides_pack_format" b5.log \ + || { cat b5.log; echo "FAIL: the refusal does not say what to do instead"; exit 1; } + +# ── 6. a role other than artifact cannot reach the staged tree ───────────── +# GATED, so this is a role refusal and not the one above. An ungated action is +# already refused in the FIRST pass of `mcpp pack`, before there is a staged +# tree, which is why the two cases need different fixtures rather than a +# one-word edit: only a gated action gets far enough for its role to matter. +cd "$TMP" +cp -r app wrongrole +cd wrongrole +cat > build.mcpp <<'EOF' +import mcpp; +#include +#include +int main() { + mcpp::provides_pack_format("zap"); + if (std::string_view(mcpp::pack_format()) != "zap") return 0; + const std::string root = mcpp::manifest_dir(); + const std::string out = std::string(mcpp::out_dir()) + "/app.zap"; + mcpp::action a; + a.id = "zap"; a.role = "source"; + a.arg((root + "/dist.sh").c_str()).arg("x").arg("${mcpp.stage_dir}").arg(out.c_str()) + .output(out.c_str()).submit(); + return 0; +} +EOF +set +e +"$MCPP" pack --format zap > b6.log 2>&1 +rc=$? +set -e +[ "$rc" -ne 0 ] || { cat b6.log; echo "FAIL: a source action reached the staged tree"; exit 1; } +grep -q 'other than "artifact"' b6.log \ + || { cat b6.log; echo "FAIL: the role refusal does not name the role"; exit 1; } + +# ── 7. declared and never submitted is a refusal, not a success ──────────── +# THE HALF A MEMBER AUTHOR IS MOST LIKELY TO GET WRONG. A gate that never +# opens leaves a pass that succeeds and produces no package, which reads as +# "packaging is not implemented yet" rather than as a defect in the member. +cd "$TMP" +cp -r app silent +cd silent +cat > build.mcpp <<'EOF' +import mcpp; +int main() { mcpp::provides_pack_format("zap"); return 0; } +EOF +set +e +"$MCPP" pack --format zap > b7.log 2>&1 +rc=$? +set -e +[ "$rc" -ne 0 ] || { cat b7.log; echo "FAIL: a format nothing claimed reported success"; exit 1; } +grep -q "no action claimed --format 'zap'" b7.log \ + || { cat b7.log; echo "FAIL: the refusal does not name the unclaimed format"; exit 1; } + +echo "PASS: 638_pack_format_dispatch" diff --git a/tests/unit/test_build_directives.cpp b/tests/unit/test_build_directives.cpp index 3943d9488..90f22ea86 100644 --- a/tests/unit/test_build_directives.cpp +++ b/tests/unit/test_build_directives.cpp @@ -208,7 +208,8 @@ TEST(BuildDirectives, SerializeDeserializeRoundTrip) { "mcpp:include-dir=inc\n" "mcpp:include-dir-after=after\n" "mcpp:fact=widget.driver=1.2\n" - "mcpp:floor=widget.driver >= 1.0\n"); + "mcpp:floor=widget.driver >= 1.0\n" + "mcpp:pack-format=appimage\n"); std::ostringstream os; dirs::serialize(os, d); @@ -258,6 +259,44 @@ TEST(BuildDirectives, FactsAndFloorsAreClaimsThatFoldIntoRuntimeDeclarations) { EXPECT_EQ(m.runtimeConfig.requirements[0].phase, "build"); } +TEST(BuildDirectives, APackFormatIsANameCarriedAndNotInterpreted) { + auto d = parse("mcpp:pack-format=appimage\n" + "mcpp:pack-format=msi\n"); + EXPECT_EQ(d.at(dirs::Slot::PackFormats), + (std::vector{"appimage", "msi"})); + + mcpp::manifest::Manifest m; + dirs::apply(m, d); + EXPECT_EQ(m.buildConfig.packFormats, + (std::vector{"appimage", "msi"})); + // The engine holds the DISPATCH and no format, so a name reaches neither + // the compile line nor the link line. A row that leaked into either would + // put dpkg's or WiX's vocabulary on a command line. + EXPECT_TRUE(m.buildConfig.cflags.empty()); + EXPECT_TRUE(m.buildConfig.cxxflags.empty()); + EXPECT_TRUE(m.buildConfig.ldflags.empty()); + EXPECT_TRUE(m.buildConfig.sources.empty()); + EXPECT_TRUE(m.buildConfig.actions.empty()); +} + +TEST(BuildDirectives, APackFormatDeclarationIsPersisted) { + // THE HALF THAT WOULD OTHERWISE BE MISSED. A build program's result is + // cached and a hit does not re-run it, and the pass that READS this set is + // `mcpp pack`, which is never a project's first build. A declaration that + // was not persisted would therefore be present exactly once and absent + // every time it mattered, and `--format ` would refuse naming + // nothing. + // + // `SerializeDeserializeRoundTrip` above asserts the round trip for every + // tagged row, so this only has to hold the field that puts the row in that + // set -- which is the field a new row is most likely to be added without. + const auto* def = dirs::find_by_wire("pack-format"); + ASSERT_NE(def, nullptr); + EXPECT_FALSE(def->tag.empty()) << "a tag-less row is not replayed on a cache hit"; + EXPECT_EQ(def->slot, dirs::Slot::PackFormats); + EXPECT_EQ(def->scope, dirs::Scope::Claim); +} + TEST(BuildDirectives, AClaimReachesNeitherCompileNorLink) { auto d = parse("mcpp:fact=widget.driver=1.2\n" "mcpp:floor=widget.driver >= 2.0\n"); diff --git a/tests/unit/test_graph_shape.cpp b/tests/unit/test_graph_shape.cpp index 40a75ae9d..88c63f554 100644 --- a/tests/unit/test_graph_shape.cpp +++ b/tests/unit/test_graph_shape.cpp @@ -3,10 +3,21 @@ import std; import mcpp.build.graph_shape; // The header line build.ninja carries is what the fast paths read before any -// plan exists. Three facts ride it -- the graph's shape, the module-edge -// schedule, and (2026.9.5.3+) whether a `--accel` / `--no-accel` override -// chose the device variant -- and the fast paths decline a graph that says -// anything other than "plain build, manifest's own variant". +// plan exists. Four facts ride it -- the graph's shape, the module-edge +// schedule, (2026.9.5.3+) whether a `--accel` / `--no-accel` override chose +// the device variant, and (2026.9.11.1+) which distribution format the graph +// was generated for -- and the fast paths decline a graph that says anything +// other than "plain build, manifest's own variant, no distribution edge". +// +// THE FOURTH FIELD IS VERIFIED HERE AND NOT ONLY END TO END, deliberately. +// `mcpp pack --format ` prepares twice and the second pass writes its +// graph into the SAME fingerprint directory a plain build uses, because the +// format is not in the fingerprint. Measured on 2026-09-11: a plain build +// after that pass regenerates the graph even with this field ignored, so some +// earlier freshness condition already declines -- which means an end-to-end +// assertion would pass whether or not the field works, and would keep passing +// if the field were deleted. A read-side invariant only has to hold once +// (see the module header); this is where it is held. namespace { @@ -26,11 +37,16 @@ std::filesystem::path write_graph(const std::string& first) { } // namespace -TEST(GraphShape, TheHeaderNamesShapeScheduleAndSelection) { +TEST(GraphShape, TheHeaderNamesShapeScheduleSelectionAndFormat) { EXPECT_EQ(mcpp::build::header_line(mcpp::build::GraphShape::Normal, "none", false), - "# mcpp:graph=normal;schedule=none;accel=default"); + "# mcpp:graph=normal;schedule=none;accel=default;dist=none"); EXPECT_EQ(mcpp::build::header_line(mcpp::build::GraphShape::WithTests, "two-phase", true), - "# mcpp:graph=test;schedule=two-phase;accel=override"); + "# mcpp:graph=test;schedule=two-phase;accel=override;dist=none"); + // An empty format reads as "none" rather than as an empty field: the value + // has to be a word, because `read_pack_format` returning "" already means + // "this file predates the field", and the two must not collide. + EXPECT_EQ(mcpp::build::header_line(mcpp::build::GraphShape::Normal, "none", false, "appimage"), + "# mcpp:graph=normal;schedule=none;accel=default;dist=appimage"); } TEST(GraphShape, OnlyAPlainGraphWithTheManifestsVariantIsReplayed) { @@ -40,6 +56,19 @@ TEST(GraphShape, OnlyAPlainGraphWithTheManifestsVariantIsReplayed) { EXPECT_FALSE(is_plain_build_graph(write_graph(header_line(GraphShape::Normal, "none", true)))); // The test-shaped graph was already refused. EXPECT_FALSE(is_plain_build_graph(write_graph(header_line(GraphShape::WithTests, "none", false)))); + // A DISTRIBUTION EDGE IS NOT PART OF A PLAIN BUILD. `mcpp pack --format + // appimage` makes a build program submit an artifact action consuming the + // staged tree; replaying that graph for a plain build would produce a + // distributable as a side effect of `mcpp build`, from a staged tree that + // is no longer guaranteed to describe this build. + EXPECT_FALSE(is_plain_build_graph( + write_graph(header_line(GraphShape::Normal, "none", false, "appimage")))); + EXPECT_EQ(read_pack_format( + write_graph(header_line(GraphShape::Normal, "none", false, "appimage"))), + "appimage"); + EXPECT_EQ(read_pack_format( + write_graph(header_line(GraphShape::Normal, "none", false))), + "none"); } TEST(GraphShape, AGraphThatPredatesTheFieldIsAMiss) { @@ -50,5 +79,15 @@ TEST(GraphShape, AGraphThatPredatesTheFieldIsAMiss) { auto p = write_graph("# mcpp:graph=normal;schedule=none"); EXPECT_EQ(read_shape(p), GraphShape::Normal); EXPECT_EQ(read_accel_selection(p), ""); + EXPECT_EQ(read_pack_format(p), ""); EXPECT_FALSE(is_plain_build_graph(p)); + + // Written by a 2026.9.10.2 mcpp: shape, schedule and selection, no + // distribution field. Same rule one field later -- absent is a miss, and + // must not be read as "none", or the very first build after an upgrade + // would replay a graph this binary cannot describe. + auto q = write_graph("# mcpp:graph=normal;schedule=none;accel=default"); + EXPECT_EQ(read_accel_selection(q), "default"); + EXPECT_EQ(read_pack_format(q), ""); + EXPECT_FALSE(is_plain_build_graph(q)); } diff --git a/tests/unit/test_loader_contract.cpp b/tests/unit/test_loader_contract.cpp index 52643e75a..1862b219d 100644 --- a/tests/unit/test_loader_contract.cpp +++ b/tests/unit/test_loader_contract.cpp @@ -60,9 +60,18 @@ TEST(GraphShape, UnlabelledOrUnknownGraphIsNeverPlain) { }; EXPECT_TRUE(is_plain_build_graph( - write("normal.ninja", "# banner\n# mcpp:graph=normal;schedule=none;accel=default\nrule x\n"))); + write("normal.ninja", "# banner\n# mcpp:graph=normal;schedule=none;accel=default;dist=none\nrule x\n"))); EXPECT_FALSE(is_plain_build_graph( - write("test.ninja", "# banner\n# mcpp:graph=test;schedule=none;accel=default\nrule x\n"))); + write("test.ninja", "# banner\n# mcpp:graph=test;schedule=none;accel=default;dist=none\nrule x\n"))); + + // A plain-shaped graph that `mcpp pack --format ` wrote (2026.9.11.1+): + // it carries an artifact edge consuming a staged tree, which a plain build + // must not have. These lines are spelled by hand here rather than through + // `header_line`, which is the point of this copy -- a reader of build.ninja + // sees the text, and a field added to the producer without being added to + // the reader would still pass a test that only compared the two. + EXPECT_FALSE(is_plain_build_graph( + write("dist.ninja", "# mcpp:graph=normal;schedule=none;accel=default;dist=appimage\n"))); // A plain-shaped graph an `--accel` / `--no-accel` build wrote (2026.9.5.3+): // the variant a flag chose is not the variant a plain build produces. @@ -74,6 +83,14 @@ TEST(GraphShape, UnlabelledOrUnknownGraphIsNeverPlain) { EXPECT_FALSE(is_plain_build_graph( write("no-selection.ninja", "# banner\n# mcpp:graph=normal\nrule x\n"))); + // A graph from 2026.9.10.2: shape, schedule and selection, no distribution + // field. Same rule one field later. This assertion is what caught the + // second copy of these lines when the field was added -- the line above it + // was the CURRENT spelling in one file and became the LEGACY spelling in + // both, and only a test that spells it out could say so. + EXPECT_FALSE(is_plain_build_graph( + write("no-dist.ninja", "# banner\n# mcpp:graph=normal;schedule=none;accel=default\nrule x\n"))); + // A build.ninja from before the marker existed. It MUST read as a miss: // treating it as plain is precisely the replay #407 is about. EXPECT_FALSE(is_plain_build_graph( diff --git a/tests/unit/test_pack_stage_tree.cpp b/tests/unit/test_pack_stage_tree.cpp new file mode 100644 index 000000000..5bf709694 --- /dev/null +++ b/tests/unit/test_pack_stage_tree.cpp @@ -0,0 +1,141 @@ +#include + +import std; +import mcpp.pack.stage_tree; + +// The staged tree is what `mcpp pack` computes and, until `${mcpp.stage_dir}`, +// then threw away. Two things about it are the engine's contract with a +// distribution member, and both are asserted here rather than end to end, +// because the end-to-end criterion for either is "the distributable is +// rebuilt", which a wrong answer also satisfies. + +namespace { + +struct Tmp { + std::filesystem::path path; + Tmp() { + path = std::filesystem::temp_directory_path() + / std::format("mcpp_stage_tree_{}", std::random_device{}()); + std::filesystem::create_directories(path); + } + ~Tmp() { + std::error_code ec; + std::filesystem::remove_all(path, ec); + } +}; + +void write_file(const std::filesystem::path& p, std::string_view body) { + std::filesystem::create_directories(p.parent_path()); + std::ofstream os(p, std::ios::binary); + os << body; +} + +std::string read_file(const std::filesystem::path& p) { + std::ifstream is(p, std::ios::binary); + return std::string{std::istreambuf_iterator(is), {}}; +} + +} // namespace + +TEST(PackStageTree, TheManifestIsASiblingAndNeverAMember) { + Tmp t; + auto stage = t.path / "app-1.0.0-x86_64-linux-gnu"; + std::filesystem::create_directories(stage); + + auto manifest = mcpp::pack::stage_manifest_path(stage); + EXPECT_EQ(manifest.parent_path(), stage.parent_path()); + // A file INSIDE the tree would be collected by every format that packages + // the directory wholesale, and would then ship inside the user's + // installer. Asserted as a path relationship because that is the property, + // and a spelling change that broke it would otherwise only show up as an + // extra file in a released package. + EXPECT_FALSE(manifest.string().starts_with(stage.string() + "/")); + EXPECT_FALSE(manifest.string().starts_with(stage.string() + "\\")); + + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage)); + std::error_code ec; + for (auto const& e : std::filesystem::recursive_directory_iterator(stage, ec)) + FAIL() << "the manifest landed inside the tree: " << e.path().string(); +} + +TEST(PackStageTree, TheManifestChangesWhenTheStagedSetDoes) { + Tmp t; + auto stage = t.path / "app"; + write_file(stage / "bin" / "app", "0123456789"); + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage)); + const auto first = read_file(mcpp::pack::stage_manifest_path(stage)); + EXPECT_NE(first.find("bin/app"), std::string::npos); + EXPECT_NE(first.find("10 "), std::string::npos); + + // A DEPENDENCY'S SHARED LIBRARY JOINING THE CLOSURE. This is the case the + // manifest exists for: the program's own bytes need not have changed, so an + // edge that depended only on the link output would report the previous + // distributable as up to date. + write_file(stage / "lib" / "libdep.so.1", "xx"); + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage)); + const auto second = read_file(mcpp::pack::stage_manifest_path(stage)); + EXPECT_NE(first, second); + EXPECT_NE(second.find("lib/libdep.so.1"), std::string::npos); + + // A staged file whose LENGTH changed, with no entry added or removed. + write_file(stage / "bin" / "app", "0123456789abcdef"); + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage)); + EXPECT_NE(read_file(mcpp::pack::stage_manifest_path(stage)), second); +} + +TEST(PackStageTree, StagingTheSameTreeTwiceLeavesTheManifestUntouched) { + Tmp t; + auto stage = t.path / "app"; + write_file(stage / "bin" / "app", "same"); + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage)); + auto manifest = mcpp::pack::stage_manifest_path(stage); + const auto before = std::filesystem::last_write_time(manifest); + + // Rewriting identical bytes would move the mtime, and a moved mtime on an + // input is indistinguishable from a changed input: a pack that staged the + // same tree twice would rebuild the distributable both times. Same rule + // `mcpp.build.stage` states at length for a staged BMI. + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage)); + EXPECT_EQ(std::filesystem::last_write_time(manifest), before); +} + +TEST(PackStageTree, TheOrderOfADirectoryWalkIsNotAPromise) { + // Two trees with the same contents produce the same bytes. Without the + // sort, an iteration order that differed between runs would make the dist + // edge dirty on every pack for no reason -- which reads as "packaging is + // slow" rather than as a defect. + Tmp a, b; + for (auto const& root : {a.path, b.path}) { + write_file(root / "s" / "bin" / "app", "aa"); + write_file(root / "s" / "lib" / "z.so", "bbb"); + write_file(root / "s" / "share" / "doc" / "readme", "c"); + ASSERT_TRUE(mcpp::pack::write_stage_manifest(root / "s")); + } + EXPECT_EQ(read_file(mcpp::pack::stage_manifest_path(a.path / "s")), + read_file(mcpp::pack::stage_manifest_path(b.path / "s"))); +} + +TEST(PackStageTree, TheEngineOwnsExactlyTwoFormatNames) { + // `tar` and `dir` are the archive shapes `mcpp pack` owns; every other + // value of `--format` is a name a package provides. The list lives beside + // the staged tree because two layers need the same answer -- the parser, + // which decides whether a value is a built-in or a dispatch, and the + // declaration check, which refuses a package that claims one of them and + // would therefore be silently unreachable. + EXPECT_TRUE(mcpp::pack::is_builtin_pack_format("tar")); + EXPECT_TRUE(mcpp::pack::is_builtin_pack_format("dir")); + EXPECT_FALSE(mcpp::pack::is_builtin_pack_format("appimage")); + EXPECT_FALSE(mcpp::pack::is_builtin_pack_format("msi")); + EXPECT_FALSE(mcpp::pack::is_builtin_pack_format("")); + EXPECT_EQ(mcpp::pack::kBuiltinPackFormats.size(), 2u); +} + +TEST(PackStageTree, AMissingTreeIsRefusedRatherThanDescribedAsEmpty) { + Tmp t; + // An empty manifest for a directory that does not exist would say "nothing + // is staged", which is what a correct pack of an empty bundle also says. + // The two must not be spelled alike. + EXPECT_FALSE(mcpp::pack::write_stage_manifest(t.path / "never-staged")); + EXPECT_FALSE(std::filesystem::exists( + mcpp::pack::stage_manifest_path(t.path / "never-staged"))); +} From 26ec4c93ed56962d3485f4f00ef7a7276f4a72e4 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Fri, 11 Sep 2026 02:15:59 +0800 Subject: [PATCH 2/5] fix(pack): an unsupported `--format` value writes nothing to stdout `mcpp pack --format bogus` must write NOTHING to stdout and exit 2. That is the machine-output contract, and 202_machine_output_contract.sh asserts it for exactly this command, because it is the path a client hits when it probes an mcpp for a capability -- the most common machine-facing failure, and the one that used to print to stdout. Moving the refusal from the CLI parser to after `prepare_build` broke it. It had to move: the set of valid values is a property of the RESOLVED GRAPH, so a refusal written in the parser could only compare against a constant, which is the coupling this whole mechanism exists to remove. But prepare narrates what it resolves, so the refusal now arrived after three lines on stdout. FAIL: unsupported value (pack) wrote to stdout: Resolving toolchain Measured on macos-arm64, and it would have failed on every platform -- the macOS shard is simply the one that reached it first. The fix is to be quiet until the value is validated, and only then. Nothing is lost when the value IS valid: the dispatch pass prepares a second time and prints the same lines, so a successful `pack --format ` narrates once rather than twice. `--format tar` and `--format dir` are untouched, because their values were never in question. The musl re-prepare is quieted on the same grounds: it also runs before the format has been validated. The assertion is added to 638_pack_format_dispatch.sh as well as living in 202. That is deliberate duplication: the tension is local to this feature -- the valid set needs the graph, and the graph narrates -- so the test for the feature should fail when the contract does, rather than only the general contract test noticing. --- src/pack/pipeline.cppm | 20 ++++++++++++++++++++ tests/e2e/638_pack_format_dispatch.sh | 14 ++++++++++++++ 2 files changed, 34 insertions(+) diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index 65608f2d4..08482ee8d 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -57,8 +57,24 @@ export int build_and_pack(Options opts, bool modeFromUser, ov.profile = opts.profile; ov.profile_fallback = "release"; + // QUIET FOR A DISPATCHED FORMAT, AND ONLY UNTIL THE VALUE IS VALIDATED. + // + // `mcpp pack --format bogus` must write NOTHING to stdout and exit 2 -- + // the machine-output contract, asserted by + // tests/e2e/202_machine_output_contract.sh, because this is the path a + // client hits when it probes an mcpp for a capability. The set of valid + // values is a property of the resolved graph, so the refusal cannot be + // decided until prepare has run, and prepare narrates what it resolves. + // + // Nothing is lost when the value IS valid: the dispatch pass prepares a + // second time and prints the same lines, so a successful + // `pack --format ` narrates once rather than twice. + const bool quietUntilValidated = + opts.format == mcpp::pack::Format::Dispatched && !mcpp::ui::is_quiet(); + if (quietUntilValidated) mcpp::ui::set_quiet(true); auto ctx = mcpp::build::prepare_build(/*print_fp=*/false, /*includeDevDeps=*/false, /*extraTargets=*/{}, ov); + if (quietUntilValidated) mcpp::ui::set_quiet(false); if (!ctx) { mcpp::ui::error(ctx.error()); return 2; @@ -91,7 +107,11 @@ export int build_and_pack(Options opts, bool modeFromUser, // overrides object left behind here would make that pass differ from // this build in a way nothing states. ov.target_triple = "x86_64-linux-musl"; + // Quiet on the same grounds as the first prepare: this one also runs + // before `--format` has been validated. + if (quietUntilValidated) mcpp::ui::set_quiet(true); auto ctx2 = mcpp::build::prepare_build(false, false, {}, ov); + if (quietUntilValidated) mcpp::ui::set_quiet(false); if (!ctx2) { mcpp::ui::error(ctx2.error()); return 2; } ctx = std::move(ctx2); } diff --git a/tests/e2e/638_pack_format_dispatch.sh b/tests/e2e/638_pack_format_dispatch.sh index 3ef21d1b7..fa68b2c03 100755 --- a/tests/e2e/638_pack_format_dispatch.sh +++ b/tests/e2e/638_pack_format_dispatch.sh @@ -130,6 +130,20 @@ grep -q "available in this build: tar, dir, zap" b2.log \ # Nothing was compiled to find that out. grep -q "Compiling app" b2.log \ && { cat b2.log; echo "FAIL: the refusal arrived after a compile"; exit 1; } +# AND NOTHING REACHED STDOUT. This is the path a client hits when it probes an +# mcpp for a capability, so the machine-output contract +# (202_machine_output_contract.sh) requires an empty stdout, a non-empty stderr +# and exit 2. It is asserted here as well because the tension is local to this +# feature: the valid set is a property of the resolved graph, so the refusal +# cannot be decided until prepare has run -- and prepare narrates what it +# resolves. Deciding it later is what broke the contract once. +set +e +so=$("$MCPP" pack --format bogus 2>/dev/null); rc=$? +se=$("$MCPP" pack --format bogus 2>&1 >/dev/null) +set -e +[ -z "$so" ] || { echo "FAIL: the refusal wrote to stdout: $(echo "$so" | head -1)"; exit 1; } +[ -n "$se" ] || { echo "FAIL: the refusal said nothing on stderr"; exit 1; } +[ "$rc" = 2 ] || { echo "FAIL: the refusal exited $rc, expected 2"; exit 1; } # ── 3. the dispatched format produces a file, and it saw the staged tree ─── "$MCPP" pack --format zap > b3.log 2>&1 || { cat b3.log; echo "FAIL: pack --format zap failed"; exit 1; } From efba58583339a55fd506c71dde532830d2cf5692 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Fri, 11 Sep 2026 02:29:53 +0800 Subject: [PATCH 3/5] feat(target): Android, iOS and wasm are rows, and the object format is an axis Section 3 of the design record draws the boundary exactly: a package can add a language, a tool, an action, a payload and a generated module, and IT CANNOT ADD A TRIPLE. Identity is three strings and `kKnownTargets` is compiled into the binary, so every layer below the first -- the `.apk` step, the `.app` step, the `.html`+`.wasm` step, the runner, the signing, the non-C++ glue -- waits on a row here and on nothing else in the engine. Registering the rows is what turns each of those from a plugin with nowhere to attach into a plugin that can be written. aarch64-linux-android x86_64-linux-android aarch64-ios wasm32-emscripten ALL FOUR ARE `planned`, WHICH IS A REFUSAL AND NOT A GAP. The tier gate answers `tier-planned` naming the row: error: target 'aarch64-linux-android' is registered but not yet supported (planned) -- no toolchain is published for it yet error: target 'aarch64-linux-androideabi' (which resolves to 'aarch64-linux-android') is registered but not yet supported rather than `unknown target`, which was false, or a build that resolves and produces nothing, which section 3.1 argues would be worse than the row's absence. What each row still needs is a PAYLOAD in every case and never engine work: `xim:android-ndk`, `xim:emsdk`, and for iOS a licence reading before a packaging decision. THE OBJECT FORMAT IS NOW ONE ANSWER, AND THAT IS WHAT #597 ACTUALLY NEEDED. The binary format was never a field. It was re-derived from `os` wherever it was needed -- `is_pe()` asked `os == "windows"`, artifact naming asked again, the packer asked a third time -- which is affordable only while the answer has two values. `wasm32` is the first target in mcpp's vocabulary whose format is neither, and a THIRD value turns those derivations into an addition at every such site. A site that is missed does not fail: it silently answers ELF, because ELF is what every `else` branch in the tree assumes. `ObjectFormat` is that addition made once, with `is_pe()` / `is_mach_o()` / `is_wasm()` reading it. It is deliberately NOT the same question as `is_freestanding()`. A bare-metal RISC-V image is ELF with no OS; a wasm module has an OS-like layer (Emscripten's POSIX emulation) and is not ELF. Merging the two axes is the mistake this replaces. ANDROID'S PLACEMENT IS THE MODELLING DECISION: `env = "android"` on a `linux` OS, not `os = "android"`. The kernel IS Linux, so ELF, the `unix` family and `nasm -f elf64` are already right; an OS value would have made every one of them wrong by default and required a new answer at each site. What differs from `gnu` is bionic, the loader path and the SDK -- which is what an `env` value is for. `androideabi` resolves to the same env: the EABI half is the ARM calling convention, which the arch segment already carries. `is_apple()` exists because a site that means "Apple" and asks "macOS" gets iOS wrong in the direction that still links. iOS shares the object format, the linker, the `arm64` spelling and `codesign` with macOS, and differs in the SDK and the deployment-target flag. No deployment target is baked into `llvm_triple()` for it, unlike the macOS branch: `-miphoneos-version-min` belongs to the layer that owns the SDK and the bundle, and a default here would be a second place that answers it. A DISPLAY DEFECT THE ROWS EXPOSED. `x86_64-linux-android` showed no `cross` tag, because that test compared arch and OS only -- and this target agrees with an x86_64 Linux host on both. An Android artifact needs bionic's loader at `/system/bin/linker64`, which no ordinary Linux host has, so it cannot run there. Spelled as a property rather than by adding `env != env`, which would have taken `x86_64-linux-musl` with it -- that one is static and does run here. 48 new cells in tests/matrix/expected.tsv, one per (mode, host, compiler) the table declares, all `unsupported / tier-planned`; the declared per-host counts move with them, because compare.sh checks the total before it checks a cell. The day a row is wired, its cells go red and say so. Verified locally: 50/50 payload and 24/24 graph on linux-x86_64. Docs: `docs/21`'s segment tables gain the new values, a section states the object-format axis and why it is not the freestanding question, and the host/target matrix gains four rows. Both READMEs' platform tables record what each row waits on. Both languages. --- ...tion-plugins-and-platform-decomposition.md | 71 ++++-- README.md | 3 + README.zh-CN.md | 3 + docs/21-the-target-triple.md | 45 +++- docs/zh/21-the-target-triple.md | 41 +++- modules/toolchain-model/src/triple.cppm | 217 +++++++++++++++++- src/toolchain/lifecycle.cppm | 13 +- tests/matrix/expected.tsv | 74 +++++- tests/unit/test_toolchain_triple.cpp | 136 +++++++++++ 9 files changed, 564 insertions(+), 39 deletions(-) diff --git a/.agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md b/.agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md index a50896e96..723dca4f0 100644 --- a/.agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md +++ b/.agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md @@ -710,24 +710,59 @@ requests **no** format and asserts the set is still non-empty. | A dist member's tool is absent | The tool is a host lookup, and an empty path becomes an argv token | Refuse in the build program, naming the tool and where it was looked for | | The produced distributable is valid and empty | Section 2's measured 52 KB installer | Each member asserts a floor on its own output, on the **success** path, through `mcpp::warning` | -### 9.7 What sections 4 to 7 promise that this pass does not deliver - -Stated here rather than discovered later. Each is blocked on something no -amount of engine work supplies: - -- **`xim:android-ndk`, `xim:emsdk`** (§7 step 5) and the **Android row** - (step 6). Both recipes are measured working in a scratch directory (§3.1); - turning either into a payload means fetching and republishing a - multi-gigabyte vendor toolchain with a derived 133-file module surface. That - is its own release, not a side effect of this one. -- **The iOS row** (step 7) and `dist-apple`'s iOS half. Not measurable on a - Linux host, and §3.1 states the two possibilities rather than choosing. -- **Web** (step 8). [#597](https://github.com/mcpp-community/mcpp/issues/597) - is a target-model change. -- **`dist-android`, `dist-web`**. Each waits on its row. - -The engine additions in §2.2 are what make each of those a single verifiable -change when it comes. None of them is a prerequisite for the others. +### 9.7 The engine's half of section 3, and what is left after it + +Section 3's boundary is exact: a package can add a language, a tool, an action, +a payload and a generated module, and it **cannot add a triple**. Every layer +below the first — the `.apk` step, the `.app` step, the `.html`+`.wasm` step, +the runner, the signing, the non-C++ glue — therefore waits on a row in +`kKnownTargets` and on nothing else in the engine. So the rows are engine work +and belong in the same release as §2.2: + +| Row | Tier | What it still needs | +|---|---|---| +| `aarch64-linux-android`, `x86_64-linux-android` | `planned` | `xim:android-ndk` | +| `aarch64-ios` | `planned` | the iPhoneOS SDK, and a licence reading first (§9.8) | +| `wasm32-emscripten` | `planned` | `xim:emsdk` | + +**`planned` is a refusal, not a gap.** The tier gate answers `tier-planned` +naming the row, so `mcpp build --target aarch64-linux-android` says the +vocabulary has this target and nothing is wired yet — rather than `unknown +target`, which was false, or a build that resolves and produces nothing, which +§3.1 argues would be worse than the row's absence. Each cell is declared in +`tests/matrix/expected.tsv`, so the day a row is wired the matrix goes red and +says so. + +**Web needed one thing the other two did not, and it was not a table row.** The +binary format was never a field: it was re-derived from `os` at each site that +needed it, which is affordable while the answer has two values. `wasm32` is the +first target whose format is neither, and a third value turns those derivations +into an addition at every such site — where a missed site does not fail, it +silently answers ELF. `ObjectFormat` is that addition made once. This is the +substance of [#597](https://github.com/mcpp-community/mcpp/issues/597)'s +"changes the target model rather than extending a table", and with §3.1 having +answered the standard-library half, #597 is now one problem rather than two. + +**Android's placement is the modelling decision.** `env = "android"` on a +`linux` OS, not `os = "android"`: the kernel is Linux, so ELF, the `unix` +family and `nasm -f elf64` are already right, and an OS value would have made +every one of them wrong by default and required a new answer at each site. What +differs from `gnu` is bionic, the loader path and the SDK, which is what an +`env` value is for. + +What remains outside this pass, each blocked on something no engine work +supplies: + +- **`xim:android-ndk`, `xim:emsdk`** (§7 step 5). Both recipes are measured + working in a scratch directory (§3.1); turning either into a payload means + fetching and republishing a multi-gigabyte vendor toolchain with a derived + 133-file module surface. That is its own release, not a side effect of this + one — and the rows landing first is exactly §7's ordering argument, which + said doing the payloads first would make each row small. The rows turned out + to be the cheap half either way. +- **The iPhoneOS SDK.** Not measurable on a Linux host, and §9.8 gives the + decision procedure rather than the measurement. +- **`dist-android`, `dist-web`**. Each waits on its payload, not on its row. ### 9.8 The iOS SDK: a three-tier policy rather than an open question diff --git a/README.md b/README.md index 578ec150e..49ae4ca9a 100644 --- a/README.md +++ b/README.md @@ -419,6 +419,9 @@ list` reports for this machine): | `aarch64-none-elf` · `x86_64-none-elf` | llvm 22 — bare metal, no C library by default ² | preview | | `thumbv7em-none-eabi` · `thumbv8m.base-none-eabi` · `thumbv8m.main-none-eabihf` | llvm 22 — Cortex-M4/M7 soft float, M23, M33F/M55F ² | preview | | `riscv64-linux-musl` · `aarch64-linux-gnu` · `x86_64-macos` | — | planned | +| `aarch64-linux-android` · `x86_64-linux-android` | needs `xim:android-ndk`; `import std` measured working on the NDK's clang | planned | +| `aarch64-ios` | needs the iPhoneOS SDK, which is a licence question before it is a packaging one | planned | +| `wasm32-emscripten` | needs `xim:emsdk`; `import std` measured working on `em++`, and the target model is [#597](https://github.com/mcpp-community/mcpp/issues/597) | planned | `verified` an image has been built **and run** for the row, qemu and wine included · `preview` it builds and links, and no emulator run has been recorded diff --git a/README.zh-CN.md b/README.zh-CN.md index c029f5435..075de37b7 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -406,6 +406,9 @@ mcpp 的身份模型是两条正交轴:**工具链** = `family@version`(family | `aarch64-none-elf` · `x86_64-none-elf` | llvm 22——裸机,默认不带 C 库 ² | preview | | `thumbv7em-none-eabi` · `thumbv8m.base-none-eabi` · `thumbv8m.main-none-eabihf` | llvm 22——Cortex-M4/M7 软浮点、M23、M33F/M55F ² | preview | | `riscv64-linux-musl` · `aarch64-linux-gnu` · `x86_64-macos` | — | planned | +| `aarch64-linux-android` · `x86_64-linux-android` | 待 `xim:android-ndk`;`import std` 在 NDK 自带的 clang 上已实测可用 | planned | +| `aarch64-ios` | 待 iPhoneOS SDK,而它先是一个许可问题再是一个打包问题 | planned | +| `wasm32-emscripten` | 待 `xim:emsdk`;`import std` 在 `em++` 上已实测可用,目标模型见 [#597](https://github.com/mcpp-community/mcpp/issues/597) | planned | `verified` 该行的镜像已被构建**并运行**过,qemu 与 wine 都算 · `preview` 可构建 可链接,未记录过模拟器运行 · `planned` 已登记在词表中,尚未接线 —— 面向这类目标 diff --git a/docs/21-the-target-triple.md b/docs/21-the-target-triple.md index b6a74e998..ea6d18354 100644 --- a/docs/21-the-target-triple.md +++ b/docs/21-the-target-triple.md @@ -38,19 +38,26 @@ and the build reports what it resolved. | Segment | Content | Example | |---|---|---| -| `arch` | instruction set | `x86_64`, `aarch64`, `riscv64` | -| `os` | operating system, or `none` | `linux`, `windows`, `macos`, `none` | -| `env` | see below — it is a different axis per platform | `gnu`, `musl`, `msvc`, `elf` | +| `arch` | instruction set | `x86_64`, `aarch64`, `riscv64`, `wasm32` | +| `os` | operating system, or `none` | `linux`, `windows`, `macos`, `ios`, `emscripten`, `none` | +| `env` | see below — it is a different axis per platform | `gnu`, `musl`, `msvc`, `android`, `elf` | The third segment is the one that repays attention, because it does not name the same kind of thing everywhere: | Platform | `env` names | Values | |---|---|---| -| `linux` | the **C library** | `gnu` (glibc), `musl` | +| `linux` | the **C library** | `gnu` (glibc), `musl`, `android` (bionic) | | `windows` | the **object ABI** | `gnu` (Itanium C++ ABI), `msvc` (Microsoft's) | | `none` | the **object format** | `elf` | -| `macos` | nothing; the platform carries no segment | — | +| `macos`, `ios`, `emscripten` | nothing; the platform carries no segment | — | + +`android` is a **C library** and therefore sits where `musl` sits, on a `linux` +OS. That placement is the whole of the modelling decision: the kernel *is* +Linux, so ELF, the `unix` family and `nasm -f elf64` are already right, and an +`os = "android"` would have made every one of them wrong by default and needed +a new answer at each site. What differs from `gnu` is bionic, the loader path +and the SDK — which is exactly what an `env` value is for. On Windows the segment is frequently misread, because the word `gnu` suggests a C library that is not there. Measured on an artefact built for @@ -68,6 +75,30 @@ openkal. `gnu` is LLVM's label for the non-MSVC ABI, inherited from MinGW, and clang requires that spelling to select the right internal toolchain. mcpp cannot rename it. +### The object format is an axis, not a derivation + +A triple's binary format used to be nothing at all: it was re-derived from `os` +wherever it was needed. `is_pe()` asked `os == "windows"`, artifact naming asked +again, the packer asked a third time. That is affordable while the answer has +two values. + +`wasm32` is the first target in mcpp's vocabulary whose format is neither, and a +third value turns those derivations into an addition **at every such site** — and +a site that is missed does not fail. It silently answers ELF, because ELF is +what every `else` branch in the tree assumes. So the format is now one answer: + +| target | format | +|---|---| +| `x86_64-linux-gnu`, `aarch64-linux-android`, `riscv64-none-elf` | ELF | +| `aarch64-macos`, `aarch64-ios` | Mach-O | +| `x86_64-windows-gnu`, `x86_64-windows-msvc` | PE | +| `wasm32-emscripten` | wasm | + +It is **not** the same question as "is there an operating system to link +against". A bare-metal RISC-V image is ELF with no OS; a wasm module has an +OS-like layer (Emscripten's POSIX emulation) and is not ELF. Merging the two +axes is the mistake this replaces. + ## Declining The Third Segment `-` is a complete target on every platform: @@ -444,6 +475,10 @@ other's rows. | `thumbv8m.base-none-eabi` | preview | `llvm@22.1.8` | payload | payload | payload | payload | | `thumbv8m.main-none-eabi` | verified | `llvm@22.1.8` | payload | payload | payload | payload | | `thumbv8m.main-none-eabihf` | preview | `llvm@22.1.8` | payload | payload | payload | payload | +| `aarch64-linux-android` | planned | — | planned | planned | planned | planned | +| `x86_64-linux-android` | planned | — | planned | planned | planned | planned | +| `aarch64-ios` | planned | — | planned | planned | planned | planned | +| `wasm32-emscripten` | planned | — | planned | planned | planned | planned | `payload` a toolchain payload here produces it · `graph` no payload, but a dependency can supply the system · `system` located on the machine, not diff --git a/docs/zh/21-the-target-triple.md b/docs/zh/21-the-target-triple.md index 9707ca39a..ef7dde21e 100644 --- a/docs/zh/21-the-target-triple.md +++ b/docs/zh/21-the-target-triple.md @@ -32,18 +32,24 @@ C 库。选中 `x86_64-linux-musl` 就是选中 musl-gcc 载荷,选中 | 段 | 内容 | 例 | |---|---|---| -| `arch` | 指令集 | `x86_64`、`aarch64`、`riscv64` | -| `os` | 操作系统,或 `none` | `linux`、`windows`、`macos`、`none` | -| `env` | 见下 —— 它在每个平台上是不同的轴 | `gnu`、`musl`、`msvc`、`elf` | +| `arch` | 指令集 | `x86_64`、`aarch64`、`riscv64`、`wasm32` | +| `os` | 操作系统,或 `none` | `linux`、`windows`、`macos`、`ios`、`emscripten`、`none` | +| `env` | 见下 —— 它在每个平台上是不同的轴 | `gnu`、`musl`、`msvc`、`android`、`elf` | 第三段值得留意,因为它在各处命名的并不是同一类东西: | 平台 | `env` 命名 | 取值 | |---|---|---| -| `linux` | **C 库** | `gnu`(glibc)、`musl` | +| `linux` | **C 库** | `gnu`(glibc)、`musl`、`android`(bionic) | | `windows` | **对象 ABI** | `gnu`(Itanium C++ ABI)、`msvc`(微软的) | | `none` | **对象格式** | `elf` | -| `macos` | 无;该平台不带这一段 | — | +| `macos`、`ios`、`emscripten` | 无;该平台不带这一段 | — | + +`android` 是一个 **C 库**,所以它落在 `musl` 落的那个位置上,OS 段仍是 `linux`。 +这个位置就是这处建模决定的全部:内核**就是** Linux,所以 ELF、`unix` family、 +`nasm -f elf64` 全都已经是对的;而一个 `os = "android"` 会让这三样默认全错,并 +且要求在每一处站点给出一个新答案。它与 `gnu` 的差别是 bionic、加载器路径和 SDK +—— 而这恰好就是 `env` 这一段存在的意义。 在 Windows 上这一段经常被读错,因为 `gnu` 这个词暗示了一个并不在场的 C 库。 对一份按构建期体系为 `x86_64-windows-gnu` 构建的产物实测: @@ -59,6 +65,27 @@ compiler-rt,C 库是 musl,C++ 运行时是 libc++,平台是 openkal。`gnu` 是 LLVM 词表里「非 MSVC 的那套 ABI」的标签,继承自 MinGW,而 clang 需要这个 拼写来选中正确的内部工具链。mcpp 改不了它。 +### 对象格式是一个轴,不是一处推导 + +一个三元组的二进制格式过去根本不是任何东西:它在每一处需要它的地方从 `os` 重新 +推导一遍。`is_pe()` 问 `os == "windows"`,产物命名再问一遍,打包器问第三遍。答案 +只有两个取值时,这是负担得起的。 + +`wasm32` 是 mcpp 词表里第一个格式不属于那两个的目标,而第三个取值会把那些推导变成 +**在每一处这样的站点上的一次添加** —— 而漏掉的那一处不会报错。它会静默地答 ELF, +因为 ELF 正是这棵树里每一个 `else` 分支所假设的东西。于是格式现在是一个答案: + +| 目标 | 格式 | +|---|---| +| `x86_64-linux-gnu`、`aarch64-linux-android`、`riscv64-none-elf` | ELF | +| `aarch64-macos`、`aarch64-ios` | Mach-O | +| `x86_64-windows-gnu`、`x86_64-windows-msvc` | PE | +| `wasm32-emscripten` | wasm | + +它与「有没有一个操作系统可供链接」**不是**同一个问题。一个裸机 RISC-V 映像是 ELF +且没有 OS;一个 wasm 模块有一层类 OS 的东西(Emscripten 的 POSIX 模拟)而不是 +ELF。把这两个轴并成一个,正是这处改动要消除的那个错误。 + ## 省略第三段 `-` 在每个平台上都是一个完整的目标: @@ -398,6 +425,10 @@ CRT;图供给时是 `musl`。一个目标字符串,两个不同的 C 库 —— | `thumbv8m.base-none-eabi` | preview | `llvm@22.1.8` | 载荷 | 载荷 | 载荷 | 载荷 | | `thumbv8m.main-none-eabi` | verified | `llvm@22.1.8` | 载荷 | 载荷 | 载荷 | 载荷 | | `thumbv8m.main-none-eabihf` | preview | `llvm@22.1.8` | 载荷 | 载荷 | 载荷 | 载荷 | +| `aarch64-linux-android` | planned | — | planned | planned | planned | planned | +| `x86_64-linux-android` | planned | — | planned | planned | planned | planned | +| `aarch64-ios` | planned | — | planned | planned | planned | planned | +| `wasm32-emscripten` | planned | — | planned | planned | planned | planned | `载荷` 这里有工具链载荷产出它 · `图` 没有载荷,但依赖可以供给系统 · `系统` 在机器上被找到,不是 mcpp 装的 · `SDK` 平台自己的 · diff --git a/modules/toolchain-model/src/triple.cppm b/modules/toolchain-model/src/triple.cppm index 2a605a65b..795c629ce 100644 --- a/modules/toolchain-model/src/triple.cppm +++ b/modules/toolchain-model/src/triple.cppm @@ -29,6 +29,37 @@ import mcpp.platform; export namespace mcpp::toolchain::triple { +// WHAT A TARGET PRODUCES, AS ONE ANSWER RATHER THAN A DERIVATION AT EACH SITE. +// +// The binary format used not to be anything: it was re-derived from `os` +// wherever it was needed -- `is_pe()` asked `os == "windows"`, artifact naming +// asked again, the packer asked a third time -- and that is affordable only +// while the answer has two values. A THIRD produces an addition at every such +// site, and a site that was missed does not fail: it silently answers "ELF", +// because ELF is what every `else` branch in the tree assumes. +// +// That is why this exists before wasm needs it rather than after. `wasm32` is +// the first target in mcpp's vocabulary whose object format is neither of the +// two the tree was written around, and #597 is a target-model change for +// exactly this reason -- not because a table row is hard. +// +// IT IS NOT THE SAME QUESTION AS `is_freestanding()`, and merging them would be +// the mistake this replaces. "Which container do objects come in" and "is there +// an operating system to link against" are different axes: a bare-metal +// RISC-V image is ELF with no OS, and a wasm module has an OS-like layer +// (Emscripten's POSIX emulation) and is not ELF. +enum class ObjectFormat { Elf, MachO, Pe, Wasm }; + +std::string_view to_string(ObjectFormat f) { + switch (f) { + case ObjectFormat::Elf: return "ELF"; + case ObjectFormat::MachO: return "Mach-O"; + case ObjectFormat::Pe: return "PE"; + case ObjectFormat::Wasm: return "wasm"; + } + return "ELF"; +} + struct Triple { std::string arch; // "x86_64" | "aarch64" | "riscv64" | ... (GNU spelling) std::string os; // "linux" | "macos" | "windows" @@ -116,7 +147,29 @@ struct Triple { if (is_msvc_env()) return arch + "-pc-windows-msvc"; return arch + "-w64-windows-gnu"; } + // APPLE'S OTHER OS. Same `arm64` spelling and the same vendor segment; + // what differs is the SDK and the deployment-target flag. + // + // NO VERSION IS BAKED IN, unlike the macOS branch above, and that is a + // decision rather than an omission. `-miphoneos-version-min` belongs to + // the layer that also owns the SDK path and the `.app` bundle -- a + // distribution plugin -- and a default written here would be a second + // place that answers it. clang picks its own when nothing says. + if (os == "ios") { + const std::string a = (arch == "aarch64") ? "arm64" : arch; + return a + "-apple-ios"; + } + // ANDROID IS LINUX, AND THE ENV SEGMENT IS WHERE IT SAYS SO. clang also + // accepts an API level fused onto the OS segment + // (`aarch64-linux-android24`), which selects which bionic symbols are + // visible; it is omitted here for the reason the iOS version is -- + // the minimum platform version is the project's statement, and clang + // has a default. if (os == "linux") return arch + "-unknown-linux-" + (env.empty() ? "gnu" : env); + // Emscripten's own effective triple. The vendor segment is `unknown` + // and the OS segment is the platform layer rather than a kernel, which + // is why `object_format()` reads the ARCH for this row. + if (os == "emscripten") return arch + "-unknown-emscripten"; if (os == "none") return str(); // freestanding: already LLVM's form return str(); } @@ -124,7 +177,39 @@ struct Triple { bool is_musl() const { return env == "musl"; } bool is_msvc_env() const { return env == "msvc"; } bool is_windows_gnu() const { return os == "windows" && env == "gnu"; } - bool is_pe() const { return os == "windows"; } + + // THE SINGLE DERIVATION. Every question about the container objects come in + // is answered here and nowhere else -- see `ObjectFormat` for why a third + // value makes that a requirement rather than a preference. + // + // The arch test precedes the ELF fallback because a wasm target's OS + // segment names a platform layer (`emscripten`), not a format, and the + // fallback would otherwise claim ELF for it -- the silent wrong answer this + // whole axis exists to remove. + ObjectFormat object_format() const { + if (os == "windows") return ObjectFormat::Pe; + if (os == "macos" || os == "ios") return ObjectFormat::MachO; + if (arch.starts_with("wasm")) return ObjectFormat::Wasm; + return ObjectFormat::Elf; + } + + // Kept as its own name because it is what 30-odd sites already ask, and now + // reads the single answer rather than re-deriving one. + bool is_pe() const { return object_format() == ObjectFormat::Pe; } + bool is_mach_o() const { return object_format() == ObjectFormat::MachO; } + bool is_wasm() const { return object_format() == ObjectFormat::Wasm; } + + // APPLE, AS ONE QUESTION. `os == "macos"` was the whole of it while macOS + // was the only Apple row; iOS shares the object format, the linker, the + // `arm64` spelling and `codesign`, and differs in the SDK and the + // deployment-target flag. A site that means "Apple" and asks "macOS" gets + // iOS wrong in the direction that still links. + bool is_apple() const { return os == "macos" || os == "ios"; } + // Android is Linux with a different C library and a different loader path. + // `os` stays `linux` for that reason -- it is the kernel, and every + // Linux-shaped decision in the tree is right about it -- and the env + // segment carries what differs. + bool is_android() const { return env == "android"; } // Bare metal: there is no OS to link against. THE predicate every // freestanding decision keys off, spelled once here so no consumer @@ -154,9 +239,18 @@ struct Triple { bool pin_is_capability() const { return is_freestanding() || (is_pe() && is_musl()); } // cfg() `family` dimension: unix | windows. + // + // iOS and Android are unix for the reason macOS and Linux are: the + // predicate answers about the API surface a source can assume, and both are + // POSIX. Emscripten is unix on the same test rather than on a claim about + // wasm -- it supplies a POSIX emulation, and a source guarded by + // `cfg(unix)` compiles against it. A target with no OS still answers + // nothing, unchanged: `cfg(unix)` on bare metal would be false in a way no + // source could act on. std::string family() const { if (os == "windows") return "windows"; - if (os == "linux" || os == "macos") return "unix"; + if (os == "linux" || os == "macos" || os == "ios" + || os == "emscripten") return "unix"; return {}; } @@ -413,6 +507,86 @@ inline constexpr TargetInfo kKnownTargets[] = { // library for these targets arrives from the dependency graph. { "armv7a-none-eabi", "verified", "bare","llvm@22.1.8","", true }, { "armv7a-none-eabihf", "verified", "bare","llvm@22.1.8","", true }, + + // ── The three platforms a package cannot add ──────────────────────────── + // + // A package can add a language, a tool, an action, a payload and a + // generated module. IT CANNOT ADD A TRIPLE: identity is these three + // strings and this table is compiled into the binary, so every layer above + // -- the `.apk` step, the `.app` step, the `.html`+`.wasm` step, the + // runner, the signing -- waits on a row here and on nothing else in the + // engine. Registering the rows is what turns each of those into a plugin + // that can be written rather than a plugin that has nowhere to attach. + // + // ALL FOUR ARE `planned`, WHICH IS A REFUSAL AND NOT A GAP. The tier gate + // refuses a planned row with `tier-planned` naming the row, so + // `mcpp build --target aarch64-linux-android` says the vocabulary has this + // target and nothing is wired yet -- rather than `unknown target`, which + // was false, or a build that resolves and produces nothing, which would be + // worse than either. What each row still needs is recorded in + // .agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md + // section 3, and it is a payload in every case, never engine work. + // + // THE PREREQUISITE NOBODY LISTS IS ANSWERED FOR TWO OF THE THREE. mcpp is + // module-first, so a row whose toolchain cannot compile a module interface + // unit would be worse than its absence. Measured 2026-09-11: `import std` + // works on both the NDK's clang 18 and Emscripten's, and neither needs a + // fork or a compiler upgrade -- what both need is the generated module + // surface their vendor chose not to install (133 files, 620 KB, taken from + // the libc++ revision matching `_LIBCPP_VERSION`, which for Emscripten is + // NOT the version its clang reports). Apple's half is not measurable on a + // Linux host and is the one genuinely open question of the three. + + // ANDROID IS THE SMALLEST OF THE THREE, and the ranking is the opposite of + // the demand ranking. `aarch64` is already an arch, ELF is already the + // object format, and Linux is already the OS: what was missing is an `env` + // value and a sysroot that points at an NDK. No `pin`, because no cross + // payload exists yet -- `xim:android-ndk` is the row's whole remaining + // cost, and until it lands `[target.].sysroot` is the escape hatch + // for a machine that has an NDK already. + { "aarch64-linux-android", "planned", "", "", "", false }, + // The emulator's row. Not a convenience: x86_64 is what an Android + // emulator image runs, so a row for the device without one for the + // emulator describes a target nothing in CI can execute. + { "x86_64-linux-android", "planned", "", "", "", false }, + + // iOS IS NEXT. `aarch64-macos` is `verified`, so Mach-O, `arm64`, the + // linker and the Apple half of the toolchain model all exist; what is + // missing is an `os` value and the iPhoneOS SDK. + // + // THE SDK IS A LICENCE QUESTION AND NOT A PACKAGING ONE, which is why this + // row carries no `sysroot`. The NDK is Apache-2.0 and Emscripten is MIT, + // both redistributable; the iPhoneOS SDK is neither. The recipe should + // reach for the lowest of three tiers its licence allows -- redistribute, + // fetch from upstream without a mirror, or locate what the machine already + // has -- and say which tier it took, because a consumer reading "locator" + // needs to know that is a licence conclusion rather than an unfinished + // recipe. `msvc@system` is the shape of the third tier and mcpp already + // has it. + // + // The simulator is deliberately not a row. It has its own SDK and produces + // its own object, so folding it in would make two targets share an + // identity -- the mistake `x86_64-windows-musl` was added to undo. + { "aarch64-ios", "planned", "", "", "", false }, + + // WEB IS THE OUTLIER, AND IT IS THE ONLY ONE OF THE THREE THAT CHANGES THE + // MODEL RATHER THAN EXTENDING A TABLE. A new arch (`wasm32`), a new os + // (`emscripten`), and -- the sharp part -- a new OBJECT FORMAT, which + // before `ObjectFormat` existed was not a field at all but a derivation + // repeated at every site that needed it. That is why this is + // https://github.com/mcpp-community/mcpp/issues/597 and not a table row. + // + // IT IS NOW ONLY THAT. The standard-library half is answered: measured + // 2026-09-11, `em++` compiles and links `import std` with NO additional + // flags once the module surface from llvm 20.1.7 is present -- the release + // matching Emscripten's `_LIBCPP_VERSION` of 200100, not the 22.0.0git its + // clang reports -- and `node app.js` printed the expected output. So #597 + // is one problem rather than two. + // + // `defaultStatic` is true because wasm has no dynamic loader in the sense + // the other rows mean: an Emscripten link produces one module plus its + // JavaScript, and there is no shared object for a search path to find. + { "wasm32-emscripten", "planned", "wasm","", "", true }, }; inline std::span known_targets() { return kKnownTargets; } @@ -762,6 +936,19 @@ std::optional parse(std::string_view s) { // "mingw32" is the GNU os segment for ALL MinGW targets (64-bit // included — historical residue); it means windows + gnu env. if (starts_with(k, "mingw")) { t.os = "windows"; sawOs = true; t.env = "gnu"; t.envExplicit = true; continue; } + // APPLE'S SECOND OS. `starts_with` for the same reason the macOS + // branch above uses it: an effective triple carries the deployment + // target on this segment (`arm64-apple-ios17.0`). The simulator is a + // different row and is deliberately not spelled here -- it has a + // different SDK and a different object, so folding it into this one + // would make two targets share an identity. + if (starts_with(k, "iphoneos") || starts_with(k, "ios")) + { t.os = "ios"; sawOs = true; t.env.clear(); continue; } + // EMSCRIPTEN IS AN OS SEGMENT, NOT AN ENV. It names the platform layer + // a wasm module is compiled against -- its POSIX emulation, its + // filesystem shim, its `main` loop -- which is the same kind of thing + // `linux` names and not the same kind of thing `musl` names. + if (starts_with(k, "emscripten")) { t.os = "emscripten"; sawOs = true; t.env.clear(); continue; } // Bare-metal object-format / ABI segments. Only meaningful with // os=none: `riscv64-none-elf`, `arm-none-eabi`, `arm-none-eabihf`. @@ -773,20 +960,38 @@ std::optional parse(std::string_view s) { } if (t.os != "macos") { + // ANDROID IS AN ENV SEGMENT ON A LINUX OS, and that placement is + // the whole of the modelling decision. The kernel IS Linux, so + // every Linux-shaped answer in the tree -- ELF, the `unix` family, + // `nasm -f elf64` -- is already right; what differs is the C + // library (bionic), the loader path and the SDK. An `os = "android"` + // would have made all three of those wrong by default and required + // a new answer at each site. + // + // `androideabi` is the 32-bit ARM spelling and resolves to the same + // env: the EABI half is the ARM calling convention, which `armv7a` + // already carries in the arch segment. + if (k == "android" || starts_with(k, "androideabi")) { + t.env = "android"; t.envExplicit = true; continue; + } if (k == "musl" || starts_with(k, "musleabi")) { t.env = "musl"; t.envExplicit = true; continue; } if (k == "gnu" || starts_with(k, "gnueabi")) { t.env = "gnu"; t.envExplicit = true; continue; } // starts_with: clang effective triples can carry a version suffix // on the env segment ("…-windows-msvc19.44.35211"). if (starts_with(k, "msvc")) { t.env = "msvc"; t.envExplicit = true; continue; } } - // Unrecognized segment (androideabi, wasi, …): not in mcpp's target - // language — treat as unparseable rather than guessing. + // Unrecognized segment (wasi, …): not in mcpp's target language — + // treat as unparseable rather than guessing. return std::nullopt; } if (!sawOs) return std::nullopt; - // macOS carries no env segment at all, so nothing was declined there. - if (t.os == "macos") { t.env.clear(); t.envExplicit = false; } + // macOS carries no env segment at all, so nothing was declined there. iOS + // and Emscripten are the same shape: the platform layer is the whole of the + // identity past the arch, and there is no C-library axis to decline. + if (t.os == "macos" || t.os == "ios" || t.os == "emscripten") { + t.env.clear(); t.envExplicit = false; + } // THE FILL STAYS, AND THE FACT THAT IT WAS A FILL IS NOW RECORDED. // `x86_64-linux` is the canonical identity `x86_64-linux-gnu` — every // directory name and cache key downstream depends on that — but it is NOT diff --git a/src/toolchain/lifecycle.cppm b/src/toolchain/lifecycle.cppm index dc7fa6277..638014f6e 100644 --- a/src/toolchain/lifecycle.cppm +++ b/src/toolchain/lifecycle.cppm @@ -623,7 +623,18 @@ export int toolchain_list(const mcpp::config::GlobalConfig& cfg, if (!info->note.empty()) tags.emplace_back(info->note); if (info->defaultStatic) tags.push_back("static"); } - if (t != hostT && (t.os != hostT.os || t.arch != hostT.arch)) + // A TARGET THIS HOST CANNOT EXECUTE. Arch and OS are the usual answer, + // and deliberately not env: an `x86_64-linux-musl` artifact is static + // and runs here, so calling it cross would be false. + // + // The env axis matters for exactly one row today. An Android artifact + // needs bionic's loader at `/system/bin/linker64`, which no ordinary + // Linux host has -- so `x86_64-linux-android` agrees with this host on + // both segments the test looked at and cannot run on it. Spelled as a + // property rather than by adding `env != env`, which would have taken + // musl with it. + if (t != hostT && (t.os != hostT.os || t.arch != hostT.arch + || t.is_android())) tags.push_back("cross"); std::string out; for (auto& tag : tags) { if (!out.empty()) out += ", "; out += tag; } diff --git a/tests/matrix/expected.tsv b/tests/matrix/expected.tsv index 8fb38b869..0badcac48 100644 --- a/tests/matrix/expected.tsv +++ b/tests/matrix/expected.tsv @@ -46,10 +46,14 @@ # 那台机器。 # # 各台格数不同,而这是事实不是遗漏: -# linux-x86_64 40 gcc + llvm × 12 目标(payload 24 / graph 16) -# linux-aarch64 16 只有 musl-gcc —— llvm 在非 x86_64 Linux 上被显式延缓 -# macos-arm64 20 只有 llvm × 12 目标(payload 12 / graph 8) -# windows-x86_64 40 llvm + msvc@system +# linux-x86_64 74 gcc + llvm(payload 50 / graph 24) +# linux-aarch64 33 只有 musl-gcc —— llvm 在非 x86_64 Linux 上被显式延缓 +# macos-arm64 37 只有 llvm(payload 25 / graph 12) +# windows-x86_64 74 llvm + msvc@system +# +# 这几个数字是**声明**,和表里的行一样参与比对(compare.sh 先比总数再比每一格), +# 所以加了目标行就必须同时改它们。2026-09-11 加入方案 §3 的四个平台行时,四台 +# 宿主各 +12 格:两种体系 × 该宿主声明的编译器 × 4 个目标。 graph linux-aarch64 aarch64-linux-gnu gcc@16.1.0 - - - - - unsupported tier-planned graph linux-aarch64 aarch64-linux-musl gcc@16.1.0 - - - - - unsupported layer-requirement graph linux-aarch64 riscv64-linux-musl gcc@16.1.0 - - - - - unsupported tier-planned @@ -220,3 +224,65 @@ payload windows-x86_64 x86_64-windows-msvc llvm@22.1.8 x86_64-pc-windows-msvc no payload windows-x86_64 x86_64-windows-msvc msvc@system x86_64-pc-windows-msvc none msvc(payload) (payload) - ok none payload windows-x86_64 x86_64-windows-musl llvm@22.1.8 - - - - - unsupported host-cannot-serve payload windows-x86_64 x86_64-windows-musl msvc@system - - - - - unsupported capability-pin + +# ── 方案 §3 的三个平台:词表里有,还没有任何东西接线 ────────────────────── +# +# 四行全部 `planned`,于是全部十二格(两种体系 × 四台宿主的编译器轴)都是 +# `unsupported / tier-planned`。这些格子不是占位:它们断言的是**拒绝的形状**—— +# 一个 planned 目标必须报「词表里有这一行,还没有接线」,而不是 `unknown target` +# (那是假的),也不是一次解析通过却什么都没建出来的构建(那更糟)。 +# +# 每一行接上线的时候,这里对应的那一格会从 tier-planned 变成别的东西,而这张表 +# 会因此变红 —— 那正是它该做的事。缺的东西每一处都是**载荷**,不是引擎: +# aarch64-linux-android / x86_64-linux-android xim:android-ndk +# aarch64-ios iPhoneOS SDK(先要一次许可判断) +# wasm32-emscripten xim:emsdk,以及 #597 的目标模型 +# +graph linux-aarch64 aarch64-linux-android gcc@16.1.0 - - - - - unsupported tier-planned +graph linux-x86_64 aarch64-linux-android gcc@16.1.0 - - - - - unsupported tier-planned +graph linux-x86_64 aarch64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +graph macos-arm64 aarch64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +graph windows-x86_64 aarch64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +graph windows-x86_64 aarch64-linux-android msvc@system - - - - - unsupported tier-planned +payload linux-aarch64 aarch64-linux-android gcc@16.1.0 - - - - - unsupported tier-planned +payload linux-x86_64 aarch64-linux-android gcc@16.1.0 - - - - - unsupported tier-planned +payload linux-x86_64 aarch64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +payload macos-arm64 aarch64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +payload windows-x86_64 aarch64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +payload windows-x86_64 aarch64-linux-android msvc@system - - - - - unsupported tier-planned +graph linux-aarch64 x86_64-linux-android gcc@16.1.0 - - - - - unsupported tier-planned +graph linux-x86_64 x86_64-linux-android gcc@16.1.0 - - - - - unsupported tier-planned +graph linux-x86_64 x86_64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +graph macos-arm64 x86_64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +graph windows-x86_64 x86_64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +graph windows-x86_64 x86_64-linux-android msvc@system - - - - - unsupported tier-planned +payload linux-aarch64 x86_64-linux-android gcc@16.1.0 - - - - - unsupported tier-planned +payload linux-x86_64 x86_64-linux-android gcc@16.1.0 - - - - - unsupported tier-planned +payload linux-x86_64 x86_64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +payload macos-arm64 x86_64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +payload windows-x86_64 x86_64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +payload windows-x86_64 x86_64-linux-android msvc@system - - - - - unsupported tier-planned +graph linux-aarch64 aarch64-ios gcc@16.1.0 - - - - - unsupported tier-planned +graph linux-x86_64 aarch64-ios gcc@16.1.0 - - - - - unsupported tier-planned +graph linux-x86_64 aarch64-ios llvm@22.1.8 - - - - - unsupported tier-planned +graph macos-arm64 aarch64-ios llvm@22.1.8 - - - - - unsupported tier-planned +graph windows-x86_64 aarch64-ios llvm@22.1.8 - - - - - unsupported tier-planned +graph windows-x86_64 aarch64-ios msvc@system - - - - - unsupported tier-planned +payload linux-aarch64 aarch64-ios gcc@16.1.0 - - - - - unsupported tier-planned +payload linux-x86_64 aarch64-ios gcc@16.1.0 - - - - - unsupported tier-planned +payload linux-x86_64 aarch64-ios llvm@22.1.8 - - - - - unsupported tier-planned +payload macos-arm64 aarch64-ios llvm@22.1.8 - - - - - unsupported tier-planned +payload windows-x86_64 aarch64-ios llvm@22.1.8 - - - - - unsupported tier-planned +payload windows-x86_64 aarch64-ios msvc@system - - - - - unsupported tier-planned +graph linux-aarch64 wasm32-emscripten gcc@16.1.0 - - - - - unsupported tier-planned +graph linux-x86_64 wasm32-emscripten gcc@16.1.0 - - - - - unsupported tier-planned +graph linux-x86_64 wasm32-emscripten llvm@22.1.8 - - - - - unsupported tier-planned +graph macos-arm64 wasm32-emscripten llvm@22.1.8 - - - - - unsupported tier-planned +graph windows-x86_64 wasm32-emscripten llvm@22.1.8 - - - - - unsupported tier-planned +graph windows-x86_64 wasm32-emscripten msvc@system - - - - - unsupported tier-planned +payload linux-aarch64 wasm32-emscripten gcc@16.1.0 - - - - - unsupported tier-planned +payload linux-x86_64 wasm32-emscripten gcc@16.1.0 - - - - - unsupported tier-planned +payload linux-x86_64 wasm32-emscripten llvm@22.1.8 - - - - - unsupported tier-planned +payload macos-arm64 wasm32-emscripten llvm@22.1.8 - - - - - unsupported tier-planned +payload windows-x86_64 wasm32-emscripten llvm@22.1.8 - - - - - unsupported tier-planned +payload windows-x86_64 wasm32-emscripten msvc@system - - - - - unsupported tier-planned diff --git a/tests/unit/test_toolchain_triple.cpp b/tests/unit/test_toolchain_triple.cpp index 3e609f611..49472dcee 100644 --- a/tests/unit/test_toolchain_triple.cpp +++ b/tests/unit/test_toolchain_triple.cpp @@ -515,3 +515,139 @@ TEST(Triple, EveryTableRowIsItsOwnCanonicalForm) { << info.canonical << " does not round-trip"; } } + +// ── The object-format axis, and the three platforms it exists for ─────────── +// +// The binary format used to be re-derived from `os` at every site that needed +// it, which is affordable only while the answer has two values. These hold the +// single derivation, because a site that misses a third value does not fail -- +// it silently answers ELF, which is what every `else` branch in the tree +// assumes. + +TEST(Triple, TheObjectFormatIsOneAnswerAndNotADerivation) { + EXPECT_EQ(parse("x86_64-linux-gnu")->object_format(), ObjectFormat::Elf); + EXPECT_EQ(parse("x86_64-linux-musl")->object_format(), ObjectFormat::Elf); + EXPECT_EQ(parse("aarch64-linux-android")->object_format(), ObjectFormat::Elf); + EXPECT_EQ(parse("riscv64-none-elf")->object_format(), ObjectFormat::Elf); + EXPECT_EQ(parse("x86_64-windows-gnu")->object_format(), ObjectFormat::Pe); + EXPECT_EQ(parse("x86_64-windows-msvc")->object_format(), ObjectFormat::Pe); + EXPECT_EQ(parse("aarch64-macos")->object_format(), ObjectFormat::MachO); + EXPECT_EQ(parse("aarch64-ios")->object_format(), ObjectFormat::MachO); + EXPECT_EQ(parse("wasm32-emscripten")->object_format(), ObjectFormat::Wasm); +} + +TEST(Triple, TheFormatQuestionIsNotTheOperatingSystemQuestion) { + // The two axes were conflated before `ObjectFormat` existed, and merging + // them is the mistake it replaces: a bare-metal image is ELF with no OS, + // and a wasm module has an OS-like layer and is not ELF. + auto bare = parse("riscv64-none-elf"); + EXPECT_TRUE(bare->is_freestanding()); + EXPECT_EQ(bare->object_format(), ObjectFormat::Elf); + + auto web = parse("wasm32-emscripten"); + EXPECT_FALSE(web->is_freestanding()); + EXPECT_TRUE(web->is_wasm()); +} + +TEST(Triple, IsPeAndIsMachOReadTheSingleAnswer) { + EXPECT_TRUE (parse("x86_64-windows-gnu")->is_pe()); + EXPECT_FALSE(parse("x86_64-windows-gnu")->is_mach_o()); + EXPECT_TRUE (parse("aarch64-ios")->is_mach_o()); + EXPECT_FALSE(parse("aarch64-ios")->is_pe()); + // The row that used to answer this by `os == "windows"` and would have + // answered ELF for wasm. + EXPECT_FALSE(parse("wasm32-emscripten")->is_pe()); + EXPECT_FALSE(parse("wasm32-emscripten")->is_mach_o()); +} + +TEST(Triple, AndroidIsAnEnvOnALinuxOs) { + auto t = parse("aarch64-linux-android"); + ASSERT_TRUE(t.has_value()); + EXPECT_EQ(t->arch, "aarch64"); + // THE PLACEMENT IS THE MODELLING DECISION. The kernel is Linux, so ELF, + // the `unix` family and `nasm -f elf64` are all already right; an + // `os = "android"` would have made every one of them wrong by default. + EXPECT_EQ(t->os, "linux"); + EXPECT_EQ(t->env, "android"); + EXPECT_TRUE(t->is_android()); + EXPECT_EQ(t->family(), "unix"); + EXPECT_EQ(t->str(), "aarch64-linux-android"); + EXPECT_EQ(t->llvm_triple(), "aarch64-unknown-linux-android"); + + // `androideabi` is the 32-bit ARM spelling of the same env: the EABI half + // is the calling convention, which the arch segment already carries. + auto eabi = parse("armv7a-linux-androideabi"); + ASSERT_TRUE(eabi.has_value()); + EXPECT_EQ(eabi->env, "android"); + + // The env fill must not reach an Android request. `x86_64-linux` is still + // `gnu`; `x86_64-linux-android` is not. + EXPECT_EQ(parse("x86_64-linux")->env, "gnu"); + EXPECT_EQ(parse("x86_64-linux-android")->env, "android"); +} + +TEST(Triple, IosIsAppleWithoutBeingMacos) { + auto t = parse("aarch64-ios"); + ASSERT_TRUE(t.has_value()); + EXPECT_EQ(t->os, "ios"); + EXPECT_TRUE(t->env.empty()); + EXPECT_EQ(t->family(), "unix"); + EXPECT_EQ(t->str(), "aarch64-ios"); + // Apple's own spelling of the architecture, as the macOS branch already + // produces. No deployment target is baked in: that flag belongs to the + // layer that owns the SDK, and a default here would be a second answer. + EXPECT_EQ(t->llvm_triple(), "arm64-apple-ios"); + + // A SITE THAT MEANS "APPLE" AND ASKS "macOS" GETS iOS WRONG IN THE + // DIRECTION THAT STILL LINKS, which is why the predicate exists. + EXPECT_TRUE(parse("aarch64-macos")->is_apple()); + EXPECT_TRUE(parse("aarch64-ios")->is_apple()); + EXPECT_FALSE(parse("aarch64-linux-musl")->is_apple()); + + // An effective triple carries the deployment target on this segment. + auto eff = parse("arm64-apple-ios17.0"); + ASSERT_TRUE(eff.has_value()); + EXPECT_EQ(eff->os, "ios"); +} + +TEST(Triple, EmscriptenIsAnOsSegmentAndNotAnEnv) { + auto t = parse("wasm32-emscripten"); + ASSERT_TRUE(t.has_value()); + EXPECT_EQ(t->arch, "wasm32"); + // It names the platform layer a module is compiled against -- the POSIX + // emulation, the filesystem shim, the main loop -- which is the kind of + // thing `linux` names and not the kind of thing `musl` names. + EXPECT_EQ(t->os, "emscripten"); + EXPECT_TRUE(t->env.empty()); + EXPECT_EQ(t->str(), "wasm32-emscripten"); + EXPECT_EQ(t->llvm_triple(), "wasm32-unknown-emscripten"); + // `unix` on the test the predicate actually applies -- what API surface a + // source may assume -- rather than on a claim about wasm. + EXPECT_EQ(t->family(), "unix"); + // NASM is x86-family by construction and must decline rather than choose. + EXPECT_FALSE(t->nasm_format().has_value()); +} + +TEST(Triple, TheThreePlatformsAreRegisteredAndPlanned) { + // A row here is what every layer above waits on: the `.apk` step, the + // `.app` step, the `.html`+`.wasm` step, the runner and the signing all + // attach to a triple, and a package cannot add one. + // + // `planned` is a REFUSAL and not a gap -- the tier gate answers + // `tier-planned` naming the row, rather than `unknown target`, which would + // be false, or a build that resolves and produces nothing, which is worse. + for (auto name : {"aarch64-linux-android", "x86_64-linux-android", + "aarch64-ios", "wasm32-emscripten"}) { + auto t = parse(name); + ASSERT_TRUE(t.has_value()) << name; + EXPECT_EQ(t->str(), name); + auto* info = find_known_target(*t); + ASSERT_NE(info, nullptr) << name; + EXPECT_EQ(info->tier, "planned") << name; + // No pin and no sysroot: what each row still needs is a PAYLOAD, and + // naming a compiler that cannot serve the target would be a claim the + // row cannot keep. + EXPECT_TRUE(info->pin.empty()) << name; + EXPECT_TRUE(info->sysroot.empty()) << name; + } +} From 6163f86f8ce6ca1832e97e56a9eb4b7dfff54eb1 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Fri, 11 Sep 2026 02:56:45 +0800 Subject: [PATCH 4/5] fix(pack): the distributable is what the request introduced, not what names the staged tree `mcpp pack --format ` reported as the distributable only those artifact actions that named `${mcpp.stage_dir}`, on the assumption that a distributable consumes the staged closure. NOT EVERY FORMAT DOES, and the one that does not is the one the guidance recommends. An `.msi` built from ONE NAMED PROGRAM takes `${mcpp.target_file:}` and never looks at the tree. That is `docs/31`'s "name the input, do not harvest a directory", written after a bind path that resolved to nothing produced a valid, empty, 52 KB installer with no diagnostic. So a member following that guidance failed the check, and `mcpp pack --format msi` reported no action claimed --format 'msi' after a `wix build` that had succeeded. The workaround available to the member was to declare the placeholder as an extra, otherwise-unused input purely to satisfy the engine -- which also gave it a dependency on a tree its one `File` row never reads. The property actually wanted is presence in the dispatch pass. An artifact action present whether or not a format was asked for -- a codesign stamp, a size budget -- existed before anyone asked, and reporting one as the package would be a wrong answer that looks like a right one. So the first pass's artifact actions are collected, and the dispatch reports the difference. Identity is (package, id): an id is unique within the package that declared it and nothing more. `consumesStageDir` keeps its one real job -- the implicit dependency on the staged tree's manifest, so an action that reads the tree is dirty when the staged SET changes and not only when a link output does. Its comment now says what it is not. `638_pack_format_dispatch.sh` gains the case that pins this from both sides: a fixture submitting an UNGATED artifact stamp and a GATED action that names no staged tree at all. The gated one must be reported and the stamp must not, and the stamp must still have been built -- it was simply not the answer. Verified load-bearing by removing the guard, which reports `size.stamp` as the package and fails the test. Found while porting `mcpp.dist.wix`, which is to say by writing the second member. The first one consumed the staged tree and agreed with the check by accident. --- docs/30-build-mcpp.md | 8 ++++ docs/zh/30-build-mcpp.md | 6 +++ modules/manifest/src/types.cppm | 19 +++++++--- src/pack/pipeline.cppm | 42 +++++++++++++++++---- tests/e2e/638_pack_format_dispatch.sh | 53 +++++++++++++++++++++++++++ 5 files changed, 114 insertions(+), 14 deletions(-) diff --git a/docs/30-build-mcpp.md b/docs/30-build-mcpp.md index a7dd89f5c..43844a8ab 100644 --- a/docs/30-build-mcpp.md +++ b/docs/30-build-mcpp.md @@ -625,6 +625,14 @@ is that format's knowledge and not the engine's. `mcpp pack --format dir` writes the same tree to a path and stops, which is how a person inspects what a member will be handed. +**`mcpp pack` reports the artifact actions the request introduced.** An action +present whether or not a format was asked for — a codesign stamp, a size budget +— is not the distributable, and naming one would be a wrong answer that looks +like a right one. Nothing about the criterion is a property of the member: a +format that packages one named program and never reads the staged tree is +recognised exactly as one that consumes the whole closure. A format nothing +submitted for is refused by name. + **An action that names `${mcpp.stage_dir}` gains a dependency on the tree's manifest.** mcpp writes `.stage-manifest` — a sibling, never a member, so it does not travel inside anyone's installer — listing each staged diff --git a/docs/zh/30-build-mcpp.md b/docs/zh/30-build-mcpp.md index e541d651f..2552d352f 100644 --- a/docs/zh/30-build-mcpp.md +++ b/docs/zh/30-build-mcpp.md @@ -534,6 +534,12 @@ pass 和那次暂存已经回答过的。 `mcpp pack --format dir` 把同一棵树写到一个路径上就停下,这是人去查看一个成员将会拿到 什么的方式。 +**`mcpp pack` 报告的是这次请求**引入**的那些 artifact action。** 一条无论有没有人 +要格式都在场的 action —— 一个签名 stamp、一次 size budget —— 不是可分发物,点名它会 +是一个看起来像对的错答案。这个判据里没有任何一项是成员的性质:一个只打包一个具名程序、 +从不读暂存树的格式,与一个消费整个闭包的格式被同等识别。对一个谁都没为之提交的格式, +会被点名拒绝。 + **写了 `${mcpp.stage_dir}` 的 action 会自动获得一条对这棵树的 manifest 的依赖。** mcpp 会写出 `<暂存树>.stage-manifest` —— 一个兄弟文件,永不是成员,所以它不会跑进任何 人的安装包里 —— 逐条列出每个已暂存文件的大小与相对路径。这条依赖由引擎添加,因为「用 diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index cb2a9ef3f..d39f7c792 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -388,12 +388,19 @@ struct BuildAction { // Set by the engine, never by the build program: this action's command or // inputs named `${mcpp.stage_dir}`. // - // It is what makes the distributable ATTRIBUTABLE. `mcpp pack --format - // ` reports the file the pass produced, and the alternative -- taking - // every artifact action's output -- would name a codesign stamp or a size - // budget alongside it. It is also the flag the dispatch checks to refuse a - // member that declared a format and then submitted nothing for it, which - // would otherwise be a pack that succeeds and produces no package. + // ITS ONE JOB IS THE IMPLICIT DEPENDENCY. An action that names the staged + // tree gains an edge to that tree's manifest, so it is dirty when the + // staged SET changes and not only when a link output does. The dependency + // is implied by the use, so a member author cannot forget it. + // + // IT IS NOT HOW `mcpp pack --format ` DECIDES WHICH ACTION IS THE + // DISTRIBUTABLE, and briefly was. Not every format consumes the closure: an + // `.msi` built from ONE NAMED PROGRAM takes `${mcpp.target_file:}` + // and never looks at the tree -- which is the shape the guidance + // recommends, after a bind path that resolved to nothing produced a valid, + // empty, 52 KB installer. So the member that followed the guidance was the + // member that check refused. The dispatch asks instead which artifact + // actions the REQUEST INTRODUCED; see mcpp.pack.pipeline. bool consumesStageDir = false; std::vector inputs; // absolute or package-relative std::vector outputs; // ditto; declared, see INV-D diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index 08482ee8d..8739e805a 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -17,6 +17,7 @@ import mcpp.build.ninja; import mcpp.build.plan; import mcpp.config; import mcpp.fetcher.progress; +import mcpp.manifest; import mcpp.pack; import mcpp.pack.stage_tree; import mcpp.pack.strip; @@ -318,18 +319,44 @@ export int build_and_pack(Options opts, bool modeFromUser, // host triple, say -- is the shape where two derivations of one value agree // on every machine the author has and disagree on one they do not. if (opts.format == mcpp::pack::Format::Dispatched) { + // WHICH ARTIFACT ACTIONS THIS BUILD ALREADY HAD, before a format was + // requested. The dispatch below reports what the REQUEST introduced, + // and this is the other half of that subtraction. + std::set> preexistingArtifacts; + for (auto const& a : ctx->plan.actions) + if (a.role == mcpp::manifest::BuildAction::Role::Artifact) + preexistingArtifacts.emplace(a.packageName, a.id); + ov.pack_format = opts.formatName; ov.pack_stage_dir = plan->stagingRoot; auto distCtx = mcpp::build::prepare_build(false, false, {}, ov); if (!distCtx) { mcpp::ui::error(distCtx.error()); return 2; } - // WHICH ACTIONS ARE THE DISTRIBUTABLE. Only those that named - // `${mcpp.stage_dir}`: a codesign stamp or a size budget is also an - // artifact action, and reporting one as the package would be a wrong - // answer that looks like a right one. + // WHICH ACTIONS ARE THE DISTRIBUTABLE: the artifact actions the REQUEST + // INTRODUCED. An action present in both passes existed before anyone + // asked for a format -- a codesign stamp, a size budget -- and + // reporting one as the package would be a wrong answer that looks like + // a right one. + // + // THE FIRST VERSION ASKED A NARROWER QUESTION AND GOT IT WRONG. It + // collected only actions naming `${mcpp.stage_dir}`, on the assumption + // that a distributable consumes the staged closure. Not every format + // does: an `.msi` built from ONE named program takes + // `${mcpp.target_file:}` and never looks at the tree, which is + // the shape section 6 of the design record recommends -- "name the + // input, do not harvest a directory", after a bind path that resolved + // to nothing produced a valid, empty, 52 KB installer. So the member + // that followed the guidance was the member the check refused, and the + // workaround was to name the placeholder as an unused input purely to + // satisfy it. Presence-in-this-pass is the property actually wanted, + // and it needs nothing of the member. + // + // Identity is (package, id): an id is unique within the package that + // declared it and nothing more. std::vector distOutputs; for (auto const& a : distCtx->plan.actions) { - if (!a.consumesStageDir) continue; + if (a.role != mcpp::manifest::BuildAction::Role::Artifact) continue; + if (preexistingArtifacts.contains({a.packageName, a.id})) continue; for (auto const& o : a.outputs) distOutputs.push_back(o); } // DECLARED AND THEN SUBMITTED NOTHING. The half of the contract a @@ -340,9 +367,8 @@ export int build_and_pack(Options opts, bool modeFromUser, mcpp::ui::error(std::format( "no action claimed --format '{}'.\n" " A package declared it provides this format, and no build " - "program submitted an\n" - " artifact action referencing ${{mcpp.stage_dir}} when it was " - "asked for.\n" + "program submitted a new\n" + " `role = \"artifact\"` action when it was asked for.\n" " The provider must gate on the request and not on anything " "else:\n" " mcpp::provides_pack_format(\"{}\"); " diff --git a/tests/e2e/638_pack_format_dispatch.sh b/tests/e2e/638_pack_format_dispatch.sh index fa68b2c03..b452ea581 100755 --- a/tests/e2e/638_pack_format_dispatch.sh +++ b/tests/e2e/638_pack_format_dispatch.sh @@ -256,4 +256,57 @@ set -e grep -q "no action claimed --format 'zap'" b7.log \ || { cat b7.log; echo "FAIL: the refusal does not name the unclaimed format"; exit 1; } +# ── 8. an artifact action that predates the request is not the package ───── +# THE CRITERION IS "WHAT THE REQUEST INTRODUCED", and this is what distinguishes +# it from "any artifact action". A codesign stamp or a size budget is also an +# artifact action and is present whether or not a format was asked for; naming +# one as the distributable would be a wrong answer that looks like a right one. +# +# It is also what an earlier revision got wrong from the other side: the check +# collected only actions naming ${mcpp.stage_dir}, which refused a member that +# packages ONE NAMED PROGRAM and never reads the tree -- the shape the design +# record recommends, after a bind path that resolved to nothing produced a +# valid, empty, 52 KB installer. So this fixture submits both shapes: an +# ungated stamp that must be ignored, and a gated action that names no staged +# tree at all and must still be reported. +cd "$TMP" +cp -r app twoshapes +cd twoshapes +cat > build.mcpp <<'EOF' +import mcpp; +#include +#include +int main() { + mcpp::provides_pack_format("zap"); + const std::string root = mcpp::manifest_dir(); + + // Ungated: present in both passes, so it is not the distributable. + const std::string stamp = std::string(mcpp::out_dir()) + "/size.stamp"; + mcpp::action s; + s.id = "size-budget"; s.role = "artifact"; + s.arg((root + "/dist.sh").c_str()).arg("stamp").arg(root.c_str()).arg(stamp.c_str()) + .input("${mcpp.target_file:app}").output(stamp.c_str()); + s.submit(); + + if (std::string_view(mcpp::pack_format()) != "zap") return 0; + // Gated, and it names NO staged tree: the program arrives through + // ${mcpp.target_file:...} exactly as an MSI's one File row does. + const std::string out = std::string(mcpp::out_dir()) + "/app.zap"; + mcpp::action a; + a.id = "zap"; a.role = "artifact"; + a.arg((root + "/dist.sh").c_str()).arg("named").arg(root.c_str()).arg(out.c_str()) + .input("${mcpp.target_file:app}").output(out.c_str()); + a.submit(); + return 0; +} +EOF +"$MCPP" pack --format zap > b8.log 2>&1 || { cat b8.log; echo "FAIL: a member that reads no staged tree was refused"; exit 1; } +grep -q "app.zap" b8.log \ + || { cat b8.log; echo "FAIL: the gated action was not reported as the package"; exit 1; } +grep -q "size.stamp" b8.log \ + && { cat b8.log; echo "FAIL: an action predating the request was reported as the package"; exit 1; } +# Both files exist -- the stamp was built, it was simply not the answer. +[ -n "$(find target -name 'size.stamp' 2>/dev/null)" ] \ + || { echo "FAIL: the ungated artifact action did not run at all"; exit 1; } + echo "PASS: 638_pack_format_dispatch" From 5097ccf44a3effeffd372b638c581ee8adda98d7 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Fri, 11 Sep 2026 03:38:15 +0800 Subject: [PATCH 5/5] fix(target): every object-format question reads the single answer, and two copies were searching for a vendor name MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An audit of the 35 sites that derived the binary format from `os`. Twelve of them asked "which object format" and were converted to read `object_format()`; the rest ask a different question -- which payload to install, which loader variable a platform reads, which flag spelling a compiler wants -- and are unchanged. The classification mattered more than the count: a site that means "which OS" and is converted becomes wrong in a new way. WHAT EACH CONVERTED SITE ANSWERED FOR `aarch64-ios` BEFORE: -femulated-tls, -fvisibility-hidden not passed (Mach-O needs both) shared_library_link_flags $ORIGIN, which ld64 rejects shared_soname_flag -Wl,-soname, a BFD-only flag exports_file_contents / exports_flag a GNU version script debug_info_is_in_band true; Mach-O splits to dSYM dist::format_for the BUILD HOST's format Every one of those is the ELF branch reached by falling off the end of a two-valued test, which is the failure `ObjectFormat` was introduced to make impossible. Verified for every row in `kKnownTargets` that the answer changes only for `aarch64-ios` and `wasm32-emscripten`, and only toward correctness. TWO COPIES WERE SEARCHING FOR A VENDOR NAME IN THE WRONG STRING, AND ONE OF THEM AFFECTS AN ALREADY-VERIFIED ROW. `compute_flags`'s `linkIntentFlavor` and `resolution.json`'s `format` both derived the object format by looking for "apple" / "darwin" / "windows" / "mingw" in `plan.toolchain.targetTriple`. That string is mcpp's CANONICAL spelling, and `aarch64-macos` contains none of those words. The words live in the LLVM spelling, which is a different string -- the build report prints both, either side of an arrow: Target aarch64-macos → arm64-apple-macos14.0 ^ the identity ^ what clang is given So an explicit `--target aarch64-macos` linked and recorded as ELF. A NATIVE macOS build was right by a different branch -- an empty triple reaching the `needs_explicit_libcxx` rescue -- which is why nothing caught it: two paths through one function disagreed and only the exercised one was correct. Both now ask the parsed triple, and the substring test survives only for a spelling `parse` REJECTS, which is the `[target.]` escape hatch where an LLVM-shaped string is what an author actually wrote. `test_toolchain_triple.cpp` states this as a fact about the vocabulary rather than as a comment elsewhere: for `aarch64-macos`, `x86_64-macos` and `aarch64-ios` it asserts the canonical spelling contains neither "apple" nor "darwin", that the format is Mach-O anyway because it is asked of the fields, and that the LLVM spelling is where the vendor name lives. A second test takes its denominator from the table, so a row added without an answer cannot be covered by a test whose name says every row is. TWO GAPS ARE NAMED RATHER THAN GUESSED. `dist::Format` and `LinkIntentFlavor` have no `Wasm` member, so `wasm32-emscripten` still resolves to `Elf` in both. What "self-contained" and "link_lib" mean for an Emscripten link is a distribution-contract decision and the open half of #597, not a rename; the switch names the case so the gap is visible instead of reached by falling through. `mcpp::pack::run` now refuses a file that is neither ELF, PE nor Mach-O by name, where it previously handed anything not-PE-not-Mach-O to `LD_TRACE_LOADED_OBJECTS`. Docs: the design record's section 3.1 is corrected twice, because two of its guesses were measured wrong in the same direction -- a vendor had already done the work and nobody looked. NDK r30 ships the 133-file module surface itself (r27 shipped none), so for Android there is nothing to derive; and Apple's libc++ IS a build of a public revision (210106 -> llvmorg-21.1.6), the SDK ships no surface, and an `import std` Mach-O arm64 binary was linked from a Linux host. Section 8 names the pattern rather than only listing the facts. 109 unit tests pass. --- ...tion-plugins-and-platform-decomposition.md | 114 +++++++++++++++--- modules/toolchain-model/src/model.cppm | 18 ++- src/build/distribution.cppm | 15 ++- src/build/flags.cppm | 34 ++++++ src/build/ninja_backend.cppm | 24 ++-- src/build/plan.cppm | 22 +++- src/build/prepare.cppm | 33 +++-- src/pack/pack.cppm | 23 ++++ src/pack/strip.cppm | 7 +- tests/unit/test_distribution.cpp | 7 ++ tests/unit/test_hostflags.cpp | 12 ++ tests/unit/test_pack_relocate.cpp | 2 + tests/unit/test_toolchain_triple.cpp | 61 ++++++++++ 13 files changed, 328 insertions(+), 44 deletions(-) diff --git a/.agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md b/.agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md index 723dca4f0..92b625649 100644 --- a/.agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md +++ b/.agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md @@ -253,10 +253,39 @@ than the shipped state of either toolchain suggests: **`import std` works on both Android and Emscripten today, and neither needs a fork or a compiler upgrade.** What both need is a directory their vendor chose not to install. -### The gap, stated once +### The gap, stated once — and closed by one vendor since An LLVM installation that supports `import std` carries a generated module -surface beside its headers. Neither vendor ships it: +surface beside its headers. + +**The NDK ships one as of r30.** Measured 2026-09-11 against +`android-ndk-r30-linux.zip` (`Pkg.Revision 30.0.16248370`, the current LTS): +`share/libc++/v1/` holds `std.cppm`, `std.compat.cppm`, 110 `std/*.inc` and 21 +`std.compat/*.inc` — the same 133 files this section measured as **absent** on +r27, generated by Google from the same AOSP checkout as the compiler. So the +table below is r27's state, and for Android the remedy it describes is no +longer needed: there is nothing to fetch from an `llvmorg-*` tag, nothing to +substitute, and no second asset to publish. + +Two things about r30 did **not** change, and both matter more than the file +count: + +- **`-D__BIONIC_CTYPE_INLINE=` is still required.** The bionic ctype defect + reproduces verbatim on r30 / clang 21: 28 errors, confined to `cctype.inc` + and `locale.inc`, `using declaration referring to 'isalnum' with internal + linkage cannot be exported`. Google ships the surface without shipping a + build of it that works out of the box. +- **The version to match is still not derivable from the compiler.** r30's + `_LIBCPP_VERSION` is `210000` from clang `21.0.0 (based on r574158c)` — an + AOSP `toolchains/llvm-project` mirror revision, **not** a build of any public + `llvmorg-21.1.x` tag. So the pairing check below cannot compare against an + upstream tag for Android; it can only pin the value and refuse a change. + +That is the second time a guess in this section was wrong in the same +direction: **a vendor had already done the work and nobody looked.** The first +was Apple (below). The table is kept as the r27 measurement it was, because the +argument it supports — that a target row for a toolchain without the surface +would be worse than its absence — is what motivated looking at all. | | `xim:llvm` 20.1.7 | NDK r27 | Emscripten 4.0.19 | |---|---|---|---| @@ -389,14 +418,61 @@ $ node app.js half is open: **the standard library story is answered**, and what remains is [#597](https://github.com/mcpp-community/mcpp/issues/597)'s target model. -### Apple: not measured, and the recipe may not transfer +### Apple: measured after all, and every guess in this section was wrong -iOS cannot be measured on a Linux host. Unlike the other two, the remedy may not -exist: Apple's libc++ is not a build of a public revision, so there is no -matching `libcxx/modules/` to take a surface from, and the `_LIBCPP_VERSION` -check that makes the other two recipes safe has nothing to compare against. -Two honest possibilities, and which holds is a measurement not taken — Xcode's -toolchain already ships the surface, or iOS waits on Apple. +This section first said iOS could not be measured on a Linux host, that Apple's +libc++ is not a build of a public revision, and that the remedy might therefore +not exist. **All three are false**, measured 2026-09-11 against +`iPhoneOS26.5.sdk` (49 MB, from a public mirror) and `xim:llvm@22.1.8`: + +| | | +|---|---| +| `_LIBCPP_VERSION` | **210106** — a public revision, and `llvmorg-21.1.6` exists upstream | +| `_LIBCPP_ABI_NAMESPACE` | `__1` — upstream's own default, not a vendor-private namespace like the NDK's `__ndk1` | +| `_LIBCPP_HAS_NO_STD_MODULES` | `/* #undef */` — Apple does **not** disable std modules, unlike the NDK | +| `std.cppm` in the SDK | **0 files**, exactly as on Android and Emscripten | + +So the recipe is the same recipe: take `libcxx/modules/` from the tag matching +`_LIBCPP_VERSION`, perform the `@LIBCXX_MODULE_STD_INCLUDE_SOURCES@` +substitution, and get 133 files and 620 KB — the same count and size the other +two produce, and the same the vendor ships where it ships one at all. + +Carried end to end, from a Linux host: + +``` +$ clang++ --no-default-config --target=arm64-apple-ios18.0 -isysroot \ + -nostdinc++ -isystem /usr/include/c++/v1 -std=c++23 \ + --precompile surface/std.cppm -o std.pcm # 34 MB BMI +$ clang++ ... -fmodule-file=std=std.pcm -c app.cpp # import std; OK +$ clang++ ... -fuse-ld=lld app.o std.pcm -lc++ -o app +$ file app +app: Mach-O 64-bit arm64 executable, flags: +``` + +The binary was not executed — no device and no simulator on a Linux host — so +this is "compiles and links", which is what the `preview` tier means. + +**`--no-default-config` is load-bearing, and the reason generalises past iOS.** +`xim:llvm`'s payload ships a `bin/clang++.cfg` that injects the host's glibc and +libc++ unconditionally: + +``` +-isystem /include/c++/v1 +-isystem /include +-Wl,--dynamic-linker=/lib64/ld-linux-x86-64.so.2 +``` + +Those apply to **every** target the driver is pointed at, so a cross target that +is not Linux/glibc silently gets the host's C and C++ standard library ahead of +its own. `-nostdinc++` does not displace them — measured: with `-isysroot` and +`-nostdinc++` both given, the search list still began with the host's +`include/c++/v1` and the compile failed inside the host's `stdint.h` on +`gnu/stubs-32.h`. The criterion is `clang -v`'s search list, not whether the +compile succeeds, because a compile that reads the wrong standard library +usually succeeds. + +That is a property of the payload rather than of iOS, and it applies to any +non-Linux target driven through it. ### What this means for xim-pkgindex @@ -574,14 +650,18 @@ answered it by adding a separate package whose `tests/` selects instead of discovering. This is unrelated to plugins and is noted because it is the second thing an ecosystem library hits. -**Apple was not measured, and its recipe may not exist.** Android and -Emscripten were carried to a running or linking artifact; iOS cannot be -measured on a Linux host, and 3.1 states the two possibilities rather than -choosing one. The other two results are the argument for not guessing: every -intermediate guess along the way was wrong — that the gap was one file (it is -133), that a mismatched surface fails obscurely (it names the missing header), -that the blocker was libc++ (it was bionic's `static inline` ctype), and that -Emscripten's libc++ version follows its clang version (it does not). +**Every guess along the way was wrong, including the ones this document made +about Apple.** That the gap was one file (it is 133); that a mismatched surface +fails obscurely (it names the missing header); that the blocker was libc++ (it +was bionic's `static inline` ctype); that Emscripten's libc++ version follows +its clang version (it does not); that iOS could not be measured from Linux (it +can); that Apple's libc++ is not a public revision (it is, 210106); and that +Xcode might already ship the surface (it does not, and the SDK carries none). + +The pattern is worth naming rather than just recording: **each wrong guess was +about what a vendor had done, and each was cheap to check and was not checked.** +The section that guessed least — Emscripten, where the version was read rather +than inferred — is the only one that needed no correction. **Neither working recipe was carried to a released payload.** Both were built and exercised in a scratch directory. Turning each into an xim recipe — the diff --git a/modules/toolchain-model/src/model.cppm b/modules/toolchain-model/src/model.cppm index 8229fa7db..448357758 100644 --- a/modules/toolchain-model/src/model.cppm +++ b/modules/toolchain-model/src/model.cppm @@ -497,10 +497,15 @@ std::vector graph_runtime_compile_flags(const Toolchain& tc) { // defect was invisible until a second architecture was built. if (t->arch == "aarch64") out.emplace_back("--rtlib=compiler-rt"); if (t->is_pe()) out.emplace_back("-fdwarf-exceptions"); - if (t->is_pe() || t->os == "macos") out.emplace_back("-femulated-tls"); + // OBJECT FORMAT, NOT OS: `is_mach_o()` covers iOS along with macOS, which + // `os == "macos"` used to miss. Both need the emulated-TLS model for the + // same reason PE does -- `_tlv_bootstrap` is loader-bootstrapped there + // exactly as `_tls_index` is on PE. + if (t->is_pe() || t->is_mach_o()) out.emplace_back("-femulated-tls"); // MACH-O ONLY, AND THE REASON IS THAT WEAK-DEF IS A RUN-TIME MECHANISM - // THERE. See the note on this function for the measurement. - if (t->os == "macos") { + // THERE. See the note on this function for the measurement. `is_mach_o()` + // rather than `os == "macos"`: the mechanism is ld64's, which iOS shares. + if (t->is_mach_o()) { out.emplace_back("-fvisibility=hidden"); out.emplace_back("-fvisibility-inlines-hidden"); } @@ -524,8 +529,11 @@ bool target_supports_full_static(std::string_view targetTriple, bool hostCapabil // false is what keeps the two mechanisms from both emitting the flag. if (t->is_pe()) return false; - // macOS cannot fully static-link: libSystem must stay dynamic. - if (t->os == "macos") return false; + // Mach-O cannot fully static-link: libSystem (macOS) / the Apple + // equivalent (iOS) must stay dynamic. `is_mach_o()`, paired with `is_pe()` + // above, so this reads as "for each object format" rather than leaving + // iOS to fall through to the `linux` line below by accident. + if (t->is_mach_o()) return false; // Linux ELF — glibc or musl, native or cross. This is the line that was // previously gated on the HOST being Linux. diff --git a/src/build/distribution.cppm b/src/build/distribution.cppm index 52ca0d694..aef02adcf 100644 --- a/src/build/distribution.cppm +++ b/src/build/distribution.cppm @@ -129,9 +129,22 @@ enum class Format { Elf, MachO, Pe }; Format format_for(std::string_view targetTriple, Format hostFallback) { if (auto parsed = mcpp::toolchain::triple::parse(targetTriple)) { if (parsed->is_pe()) return Format::Pe; - if (parsed->os == "macos") return Format::MachO; + // `is_mach_o()`, not `os == "macos"`: the latter answered the + // opposite-hosts defect above for macOS and would still get iOS + // wrong the same way, since iOS's `os` is `ios`. + if (parsed->is_mach_o()) return Format::MachO; if (parsed->os == "linux" || parsed->os == "none") return Format::Elf; + // THIS `Format` HAS NO FOURTH MEMBER YET. A wasm32-emscripten + // triple parses here and falls out of every branch above (`is_pe()` + // and `is_mach_o()` are both false, and its `os` is `emscripten`, + // neither `linux` nor `none`) to the substring fallback below, which + // also does not name it, and then to `hostFallback` -- so today this + // function still answers the machine's own format for wasm rather + // than the target's, the same defect class its own header measured + // for macOS. Adding `Format::Wasm` is deferred to whoever gives this + // module a Mach-O-shaped mechanism for it (see `resolve`'s `switch`), + // not attempted here. } if (targetTriple.find("windows") != std::string_view::npos || targetTriple.find("mingw") != std::string_view::npos) diff --git a/src/build/flags.cppm b/src/build/flags.cppm index 7c2a64c18..c2adf9181 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -691,6 +691,40 @@ CompileFlags compute_flags(const BuildPlan& plan) { const auto linkIntentFlavor = [&] { if (isMingwTc) return LinkIntentFlavor::PeGnu; if (isMsvcDialect) return LinkIntentFlavor::PeMsvc; + // THE OBJECT FORMAT IS ASKED OF THE PARSED TRIPLE, AND THE SUBSTRING + // TEST BELOW IS NOW ONLY THE ESCAPE HATCH. + // + // `plan.toolchain.targetTriple` is mcpp's CANONICAL spelling, and + // `aarch64-macos` contains neither "apple" nor "darwin" -- so an + // explicit `--target aarch64-macos`, a verified row, fell through to + // `Elf`. Only a NATIVE macOS build was right, and by a different + // branch: an empty triple reaching the `needs_explicit_libcxx` rescue + // below. That is why nothing caught it -- the two paths through this + // function disagreed and only one of them was exercised. + // + // The substring test is kept for a triple `parse` REJECTS, which is + // the `[target.]` escape hatch: an author may name a spelling + // outside the canonical vocabulary, and it is then an LLVM-shaped + // string where "apple" and "windows" do appear. + if (auto t = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple)) { + switch (t->object_format()) { + case mcpp::toolchain::triple::ObjectFormat::MachO: + return LinkIntentFlavor::MachO; + case mcpp::toolchain::triple::ObjectFormat::Pe: + return LinkIntentFlavor::PeGnu; + case mcpp::toolchain::triple::ObjectFormat::Wasm: + // No `LinkIntentFlavor::Wasm` exists, and inventing one + // here would be a link-contract decision rather than a + // format question -- what `link_lib` and a search path + // even mean for an Emscripten link is the open half of + // #597. `Elf` is the wrong answer and is the one this + // returns; it is named here so the gap is visible rather + // than reached by falling off the end of a switch. + return LinkIntentFlavor::Elf; + case mcpp::toolchain::triple::ObjectFormat::Elf: + return LinkIntentFlavor::Elf; + } + } auto triple = plan.toolchain.targetTriple; std::ranges::transform(triple, triple.begin(), [](unsigned char c) { return std::tolower(c); }); diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 4f88f54af..cdebeeef0 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -260,15 +260,18 @@ std::string pe_link_flag(const BuildPlan& plan, bool sep, std::string shared_soname_flag(const LinkUnit& lu, const BuildPlan& plan) { if (lu.kind != LinkUnit::SharedLibrary) return ""; const auto t = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple); - const std::string os = t ? t->os - : (mcpp::platform::is_macos ? "macos" - : mcpp::platform::is_windows ? "windows" : "linux"); + // WHICH FLAG SPELLING, ASKED OF THE OBJECT FORMAT DIRECTLY rather than of + // an `os` string: the prior `os == "macos"` fell through to the ELF + // branch (`-Wl,-soname`, a GNU ld/BFD flag ld64 does not accept) for + // iOS, whose `os` is `ios`. + const bool pe = t ? t->is_pe() : bool(mcpp::platform::is_windows); + const bool macho = t ? t->is_mach_o() : bool(mcpp::platform::is_macos); // PE records no such name: a DLL is found by the filename in the importing // module's import table, and there is nothing to override. - if (os == "windows") return ""; + if (pe) return ""; const std::string name = lu.soname.empty() ? lu.output.filename().string() : lu.soname; - if (os == "macos") return "-Wl,-install_name,@rpath/" + name; + if (macho) return "-Wl,-install_name,@rpath/" + name; return lu.soname.empty() ? "" : "-Wl,-soname," + lu.soname; } @@ -291,7 +294,11 @@ std::string shared_soname_flag(const LinkUnit& lu, const BuildPlan& plan) { // script's syntax is not what the author wrote. std::string exports_file_contents(const LinkUnit& lu, std::string_view os) { std::string out; - if (os == "macos") { + // Which SYMBOL-TABLE SYNTAX, which is a property of the object format: + // iOS links with the same ld64 and the same leading-underscore Mach-O + // symbol table as macOS, so it takes this branch too rather than the + // GNU version-script one below, which ld64 does not parse. + if (os == "macos" || os == "ios") { // One symbol per line. Mach-O symbols carry a leading underscore that // the C++ source never writes, so it is added here -- the author names // the symbol, not the object format's spelling of it. @@ -316,7 +323,10 @@ std::string exports_flag(const LinkUnit& lu, std::string_view os, const std::filesystem::path& file) { if (lu.kind != LinkUnit::SharedLibrary || lu.exportPatterns.empty()) return ""; if (os == "windows") return ""; - if (os == "macos") + // iOS alongside macOS, matching `exports_file_contents`: same linker, + // same flag. Leaving it out sent an iOS shared-library link a GNU + // `--version-script` for a Mach-O symbol list, which ld64 rejects. + if (os == "macos" || os == "ios") return "-Wl,-exported_symbols_list," + file.generic_string(); return "-Wl,--version-script=" + file.generic_string(); } diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 73dadccee..0fe1db43d 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -555,8 +555,12 @@ std::vector shared_library_link_flags( const mcpp::toolchain::triple::Triple& target) { std::vector flags; const bool pe = n.sharedNeedsImportLib; + // WHICH RPATH SYNTAX, ASKED OF THE OBJECT FORMAT. `target.os == "macos"` + // used to answer this and missed iOS, which links with the same ld64 and + // wants the same `@loader_path` -- `os == "ios"` would otherwise take the + // ELF branch below and hand `$ORIGIN` to a linker that has no such token. const bool macho = target.empty() ? bool(mcpp::platform::is_macos) - : target.os == "macos"; + : target.is_mach_o(); if (pe) { flags.push_back(import_library_for(t, n).generic_string()); } else { @@ -810,9 +814,16 @@ std::vector runtime_search_closure( auto t = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple); return t ? *t : mcpp::toolchain::triple::Triple{}; }(); + // `!= "macos" && != "windows"` READ AS "ELF" BY EXCLUSION, WHICH IS THE + // ONE ANSWER THAT MUST NEVER BE REACHED BY EXCLUDING EVERYTHING ELSE: + // iOS and wasm32-emscripten both satisfy that double negative (their `os` + // is `ios` / `emscripten`, neither string), so this line called an iOS + // Mach-O and a wasm module ELF and would have handed both a DT_RPATH + // mechanism neither format has. Asked of `object_format()` directly, the + // single derivation, instead. const bool elfTarget = triple.empty() ? bool(mcpp::platform::is_linux) - : (triple.os != "macos" && triple.os != "windows"); + : (triple.object_format() == mcpp::toolchain::triple::ObjectFormat::Elf); // THE ARTIFACT'S OWN DIRECTORY — `$ORIGIN` (#415). // @@ -1059,10 +1070,13 @@ make_plan(const mcpp::manifest::Manifest& manifest, // The loader-tag contract exists only where DT_RPATH/DT_RUNPATH do. // Mach-O and PE have neither, so they get no flag rather than a branch in - // every consumer. + // every consumer. Asked of `object_format()`: the exclusion form this + // used to be (`!= "macos" && != "windows"`) answers "ELF" for any `os` it + // does not name, which is wrong the same way for iOS and for + // wasm32-emscripten — see the sibling derivation earlier in this file. const bool elfTarget = targetTriple.empty() ? bool(mcpp::platform::is_linux) - : (targetTriple.os != "macos" && targetTriple.os != "windows"); + : (targetTriple.object_format() == mcpp::toolchain::triple::ObjectFormat::Elf); auto loader_tag_flag = [&](LinkUnit::Kind kind) -> std::string { if (!elfTarget) return {}; using mcpp::build::loader::Form; diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index ec0cc7c93..8f8f9cf13 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -11522,14 +11522,29 @@ prepare_build(bool print_fingerprint, ctx.plan.runtimeBinding), nullptr, false); if (binding.is_discarded()) binding = nlohmann::json::object(); - auto triple = ctx.tc.targetTriple; - std::ranges::transform(triple, triple.begin(), - [](unsigned char c) { return std::tolower(c); }); - const bool pe = triple.find("windows") != std::string::npos - || triple.find("mingw") != std::string::npos; - const bool macho = triple.find("darwin") != std::string::npos - || triple.find("apple") != std::string::npos; - std::string format = pe ? "pe" : macho ? "macho" : "elf"; + // ASKED OF THE PARSED TRIPLE, with the substring test kept only for a + // spelling `parse` rejects. This field is the SECOND copy of a + // derivation `mcpp.build.dist::format_for` already owns, and it had + // the same defect: mcpp's canonical `aarch64-macos` contains neither + // "apple" nor "darwin", so an explicit `--target aarch64-macos` + // recorded `"elf"` while the native build on the same machine recorded + // `"macho"` -- one report contradicting the other about one machine. + std::string format = "elf"; + if (auto t = mcpp::toolchain::triple::parse(ctx.tc.targetTriple)) { + format = std::string(mcpp::toolchain::triple::to_string(t->object_format())); + std::ranges::transform(format, format.begin(), + [](unsigned char c) { return std::tolower(c); }); + if (format == "mach-o") format = "macho"; + } else { + auto triple = ctx.tc.targetTriple; + std::ranges::transform(triple, triple.begin(), + [](unsigned char c) { return std::tolower(c); }); + const bool pe = triple.find("windows") != std::string::npos + || triple.find("mingw") != std::string::npos; + const bool macho = triple.find("darwin") != std::string::npos + || triple.find("apple") != std::string::npos; + format = pe ? "pe" : macho ? "macho" : "elf"; + } // The ORDERED run-time search closure with provenance. Order is // semantics here, not presentation: it is what the loader will walk, // and the mutable SubOS farm sitting last is the invariant that keeps @@ -11548,7 +11563,7 @@ prepare_build(bool print_fingerprint, } nlohmann::json search = { {"format", format}, - {"link_library", pe ? "libpath" : "library_path"}, + {"link_library", format == "pe" ? "libpath" : "library_path"}, {"transitive_needed", format == "elf" ? "rpath_link" : "none"}, {"runtime", format == "pe" ? "deploy" : format == "macho" ? "loader_rpath" : "runpath"}, diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index 6b62d2919..b2437b1f8 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -1171,6 +1171,29 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) // it and nothing has modified either one at this point (patchelf runs // further down). What changes is only which directory `$ORIGIN` // expands to while the loader is looking. + // + // A NON-ELF ARTIFACT REACHING THIS POINT ASSUMED ELF BY EXCLUSION. + // PE and Mach-O are refused by name above `run()`'s `#else`; nothing + // between there and here asks what is LEFT actually is ELF, because + // ELF used to be the only format left once those two were excluded. + // wasm32-emscripten is the first target where that assumption is + // false: `binfmt::identify` reports `Format::Unknown` for a `.wasm` + // module (it carries none of the three magics), and `ldd_parse` + // below runs the file through the same LD_TRACE_LOADED_OBJECTS + // mechanism the Mach-O branch above refuses by name rather than + // risk — on a host where `.wasm` is registered in `binfmt_misc`, + // that does not fail, it RUNS the module. + if (auto fmt = mcpp::pack::binfmt::identify(plan.builtBinary).format; + fmt != mcpp::pack::binfmt::Format::Elf) { + return std::unexpected(Error{std::format( + "cannot package the {} artifact '{}' yet.\n" + " Its dependency closure is resolved by running the " + "artifact under its own\n" + " dynamic linker, and this file is neither ELF, PE nor " + "Mach-O -- there is no\n" + " such linker to ask.", + mcpp::pack::binfmt::format_name(fmt), plan.binaryName)}); + } auto deps = ldd_parse(plan.builtBinary); if (!deps) return std::unexpected(Error{std::format( "ldd failed on {}: {}", plan.builtBinary.string(), deps.error())}); diff --git a/src/pack/strip.cppm b/src/pack/strip.cppm index ab2143732..6707ebb51 100644 --- a/src/pack/strip.cppm +++ b/src/pack/strip.cppm @@ -166,7 +166,12 @@ bool debug_info_is_in_band(std::string_view canonicalTriple) { seg.push_back(canonicalTriple.substr(i, j - i)); i = j + 1; } - if (seg.size() >= 2 && seg[1] == "macos") return false; // debug map + .dSYM + // Mach-O, not "macOS": iOS carries the same debug map + out-of-band + // .dSYM as macOS (same ld64, same object format). This module takes a + // string rather than a `Triple` on purpose (see the note on + // `debug_info_is_in_band` above), so the grouping `is_mach_o()` states is + // spelled out here instead of asked of it. + if (seg.size() >= 2 && (seg[1] == "macos" || seg[1] == "ios")) return false; if (seg.size() >= 3 && seg[2] == "msvc") return false; // separate .pdb return true; } diff --git a/tests/unit/test_distribution.cpp b/tests/unit/test_distribution.cpp index 83ede0dc4..db33297ae 100644 --- a/tests/unit/test_distribution.cpp +++ b/tests/unit/test_distribution.cpp @@ -645,6 +645,13 @@ TEST(Distribution, FormatIsTakenFromTheTargetAndNotTheFallback) { for (auto fb : {dist::Format::Elf, dist::Format::MachO, dist::Format::Pe}) { EXPECT_EQ(dist::format_for("aarch64-macos", fb), dist::Format::MachO); EXPECT_EQ(dist::format_for("x86_64-macos", fb), dist::Format::MachO); + // iOS shares macOS's Mach-O format. Before `format_for` asked + // `is_mach_o()` instead of `os == "macos"`, this triple matched none + // of the branches inside the parsed-triple block and fell through to + // `fb` itself -- so the assertion below would have failed for two of + // the three fallbacks in this loop, the exact "decided by the + // machine doing the building" defect this test exists to catch. + EXPECT_EQ(dist::format_for("aarch64-ios", fb), dist::Format::MachO); EXPECT_EQ(dist::format_for("x86_64-windows-gnu", fb), dist::Format::Pe); EXPECT_EQ(dist::format_for("x86_64-linux-gnu", fb), dist::Format::Elf); EXPECT_EQ(dist::format_for("aarch64-linux-musl", fb), dist::Format::Elf); diff --git a/tests/unit/test_hostflags.cpp b/tests/unit/test_hostflags.cpp index 34a9d7188..07a214cf4 100644 --- a/tests/unit/test_hostflags.cpp +++ b/tests/unit/test_hostflags.cpp @@ -342,6 +342,18 @@ TEST(GraphRuntimeFlags, MachOTakesEmulatedTlsAndHiddenVisibilityButNotDwarf) { EXPECT_TRUE(has(f, "-fvisibility-inlines-hidden")); } +// iOS shares macOS's object format (Mach-O, ld64), so it must share this +// exact set of flags. Before `is_mach_o()` replaced `os == "macos"` here, an +// iOS triple matched neither the PE nor the macOS branch and this function +// silently returned no flags at all for it. +TEST(GraphRuntimeFlags, IosTakesTheSameFlagsAsMacOS) { + auto f = mcpp::toolchain::graph_runtime_compile_flags(graph_tc("aarch64-ios")); + EXPECT_FALSE(has(f, "-fdwarf-exceptions")); + EXPECT_TRUE(has(f, "-femulated-tls")); + EXPECT_TRUE(has(f, "-fvisibility=hidden")); + EXPECT_TRUE(has(f, "-fvisibility-inlines-hidden")); +} + // ELF takes NONE of them, and that is a decision rather than an omission. // There a `thread_local` is a fixed offset from the thread pointer, which the // C library establishes itself; adding the flag would work, cost an diff --git a/tests/unit/test_pack_relocate.cpp b/tests/unit/test_pack_relocate.cpp index 518819287..c1146ea70 100644 --- a/tests/unit/test_pack_relocate.cpp +++ b/tests/unit/test_pack_relocate.cpp @@ -325,6 +325,8 @@ TEST(PackStrip, WhetherStrippingAppliesIsAskedOfTheTargetNotTheCompiler) { EXPECT_FALSE(mcpp::pack::debug_info_is_in_band("x86_64-windows-msvc")); EXPECT_FALSE(mcpp::pack::debug_info_is_in_band("aarch64-macos")); EXPECT_FALSE(mcpp::pack::debug_info_is_in_band("x86_64-macos")); + // iOS carries the same Mach-O debug map + out-of-band .dSYM as macOS. + EXPECT_FALSE(mcpp::pack::debug_info_is_in_band("aarch64-ios")); // Segment-wise, not substring: mcpp has been bitten by a triple predicate // that answered on a substring before. EXPECT_TRUE(mcpp::pack::debug_info_is_in_band("macos64-linux-gnu")); diff --git a/tests/unit/test_toolchain_triple.cpp b/tests/unit/test_toolchain_triple.cpp index 49472dcee..3302b3a89 100644 --- a/tests/unit/test_toolchain_triple.cpp +++ b/tests/unit/test_toolchain_triple.cpp @@ -651,3 +651,64 @@ TEST(Triple, TheThreePlatformsAreRegisteredAndPlanned) { EXPECT_TRUE(info->sysroot.empty()) << name; } } + +TEST(Triple, TheCanonicalSpellingIsNotSEARCHABLEForAVENDORNAME) { + // WHY A SUBSTRING TEST ON THE CANONICAL TRIPLE IS WRONG, stated as a fact + // about the vocabulary rather than as a comment somewhere else. + // + // Two sites derived the object format by looking for "apple" / "darwin" / + // "windows" / "mingw" in `plan.toolchain.targetTriple`. That string is + // mcpp's CANONICAL spelling, and `aarch64-macos` contains none of those + // words -- so an explicit `--target aarch64-macos`, a `verified` row, was + // recorded and linked as ELF. A NATIVE macOS build was right by a + // different branch (an empty triple), which is why the two paths through + // one function disagreed and only the exercised one was correct. + // + // The words appear in the LLVM spelling, which is a different string and + // the reason the mistake is easy to make: + // + // aarch64-macos -> arm64-apple-macos14.0 + // ^ the identity ^ what clang is given + for (auto name : {"aarch64-macos", "x86_64-macos", "aarch64-ios"}) { + auto t = parse(name); + ASSERT_TRUE(t.has_value()) << name; + const std::string canonical = t->str(); + EXPECT_EQ(canonical.find("apple"), std::string::npos) << canonical; + EXPECT_EQ(canonical.find("darwin"), std::string::npos) << canonical; + // And the format is right anyway, because it is asked of the fields. + EXPECT_EQ(t->object_format(), ObjectFormat::MachO) << canonical; + // The LLVM spelling is where the vendor name lives. + EXPECT_NE(t->llvm_triple().find("apple"), std::string::npos) + << t->llvm_triple(); + } + // The one family the substring test got right, and only by luck: the + // canonical spelling happens to carry the OS name. + EXPECT_NE(std::string(parse("x86_64-windows-gnu")->str()).find("windows"), + std::string::npos); +} + +TEST(Triple, EveryKnownRowHasAnObjectFormatAndNoneFallsThrough) { + // THE DENOMINATOR IS THE TABLE. A row added without an answer here would + // otherwise be covered by a test whose name says every row is -- and the + // answer it would get is ELF, because ELF is what every `else` branch in + // the tree assumes. + std::size_t elf = 0, macho = 0, pe = 0, wasm = 0; + for (auto const& row : known_targets()) { + auto t = parse(row.canonical); + ASSERT_TRUE(t.has_value()) << row.canonical; + EXPECT_EQ(t->str(), row.canonical) << "a row that is not its own canonical form"; + switch (t->object_format()) { + case ObjectFormat::Elf: ++elf; break; + case ObjectFormat::MachO: ++macho; break; + case ObjectFormat::Pe: ++pe; break; + case ObjectFormat::Wasm: ++wasm; break; + } + } + // Each format has at least one row, which is what makes the axis worth + // having: a fourth value with no row would be an enum nothing produces. + EXPECT_GT(elf, 0u); + EXPECT_GT(macho, 0u); + EXPECT_GT(pe, 0u); + EXPECT_EQ(wasm, 1u) << "wasm32-emscripten is the only wasm row today"; + EXPECT_EQ(elf + macho + pe + wasm, known_targets().size()); +}