diff --git a/.agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md b/.agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md new file mode 100644 index 000000000..92b625649 --- /dev/null +++ b/.agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md @@ -0,0 +1,916 @@ +--- +subject: plugins +status: active +--- + +# The category the plugin taxonomy does not name, and what a platform actually decomposes into + +Date: 2026-09-11. Base: mcpp `2d102817`, `mcpp-plugins` `b4f7590` (0.2.6). + +**How to read this document.** Sections 1 to 3 are analysis of the base commits +above and can be checked against them. Sections 4 to 7 are a proposal and are +not implemented. Section 8 is what the proposal does not solve. + +The thesis is one sentence: **the engine should own the mechanism by which a +distributable is produced, and no format should live in the engine at all.** + +The occasion is a framework outside this project — HuxerUI, a declarative UI +library with a CMake build and a six-platform reach — being ported to mcpp, and +its ecosystem library `Lib-Live2D` being examined as the case that a package +manager either serves or does not. Two things came out of it that are about +mcpp rather than about that framework, and this document is those two. + +## 1. What is already true + +`mcpp:plugins` 0.2.6 is one package whose members are selected by features: + +```toml +[features] +default = [] +rules-ascendc = { sources = ["rules/ascendc.cppm"] } +rules-cuda = { sources = ["rules/cuda.cppm"] } +rules-hip = { sources = ["rules/hip.cppm"] } +rules-spirv = { sources = ["rules/spirv.cppm"] } +rules-sycl = { sources = ["rules/sycl.cppm"] } +tools-embed = { sources = ["tools/embed.cppm"] } +``` + +The taxonomy behind the two prefixes is stated in `docs/30-build-mcpp.md:789`: + +> **A tool is not a rule.** A rule states how a translation unit is compiled by +> a compiler mcpp does not drive: it submits an action and the engine schedules +> it. A tool states something the build program needs that no compiler +> performs, and does it while the program runs. + +And the reserved prefix, from the same file: + +> `warning: build rule 'mcpplibs.plugins' declares the module 'mcpp.rules.spirv'; +> the 'mcpp.' prefix is reserved for rules maintained by the mcpp project.` + +Five extension points do the extending, and `docs/31-authoring-a-rule-package.md` +says what they buy: **the engine holds no name that comes through any of them**. +Slang is the evidence — "the first language mcpp supports without naming it in +the engine". + +## 2. The category the taxonomy does not name + +A third kind of work fits neither definition. It does not compile a translation +unit, so it is not a rule; and it does not run while the build program runs, so +it is not a tool. It consumes **link outputs** and produces something a user +installs: + +| Work | Compiles a TU? | Runs in the build program? | +|---|---|---| +| SPIR-V from a shader | yes | no | +| A data file into a header | no | yes | +| **An MSI from the linked program** | **no** | **no** | +| **codesign on a `.app`** | **no** | **no** | +| **A `.deb`, an AppImage, an `.apk`** | **no** | **no** | + +The engine already has the mechanism: `role = "artifact"`, whose inputs are link +outputs, so ninja sequences it after the link. `docs/30-build-mcpp.md:464` names +the intended uses — "codesign, packaging, size budgets". What is missing is a +name for the packages that ship such actions, and therefore a place for them. + +**This is not hypothetical.** The HuxerUI port implements a Windows installer +this way and it is green on Windows CI: one `role = "artifact"` action running +`wix build`, with the program passed as `-d Executable=${mcpp.target_file:}` +and the definition naming it with ``. The +resulting MSI is 1572 KB and its `File` table carries one row, the 6.9 MB +application. Nothing in that action is specific to a UI framework except a +default icon path. + +An earlier revision of the same action bound a directory (`-bindpath +Application=bin`) and harvested it. When the path resolved to nothing on +Windows, WiX produced a **valid, empty, 52 KB installer with no diagnostic**. +That failure is worth recording because it is the argument for a shared package +rather than a per-project action: the mistake is not obvious, it is silent, and +every project that writes its own installer step gets to make it once. + +### 2.1 `mcpp pack` keeps the universal format and dispatches the rest + +`mcpp pack` is the right home, and the question is which half of it is the +engine's. `docs/10-pack-and-release.md:389` documents `[pack]` as declarative +configuration over a fixed set of modes (`static`, `bundle-project`, +`bundle-all`, `system`), and lists `.deb`, `.rpm` and AppImage under **Planned +Support**. + +Those three should not land there, and the split that keeps them out is: + +> **`mcpp pack` owns the mechanism and the one universal format. Every other +> format lives in a package, and `mcpp pack` dispatches to it.** + +The universal format is what it already produces: an archive that extracts and +runs. It is universal in the only sense that matters here — it needs no +knowledge of anyone else's release. Everything past it does. dpkg's control +fields, AppImage's runtime, WiX's schema, Apple's notarisation, Android's +signing scheme v3: each one bound into the engine couples an mcpp release to a +release mcpp does not control. The project already made this argument for +languages — Slang is supported "without naming it in the engine" — and a +distribution format has less claim to a name in the engine than a language +does. + +Dispatch is what keeps this from fragmenting into "packaging is a thing you do +outside `mcpp pack`" — and **the flag for it already exists**: + +``` +mcpp pack --format tar (default; .zip for a Windows target) | dir +``` + +`tar` and `dir` answer the same question `msi` and `appimage` answer — what +shape does the output take — so they belong on one axis, and the proposal is +not a new flag but a wider set of values for this one: + +```bash +mcpp pack --format msi +mcpp pack --format appimage +``` + +exactly as `--target` reaches a triple the engine did not have to know about +individually. Two consequences follow from the values no longer being a fixed +list: `--help` says "plus any format the resolved graph provides", and an +unknown value names what *is* available rather than a constant. + +### 2.2 What the engine must expose for that to work + +Three additions, and each is **format-neutral** — which is the test for whether +something belongs in the engine at all. + +**(a) A staged tree an artifact action can consume.** `mcpp pack` already +computes one: the dependency closure, the strip policy, the debug-symbol split, +`include`/`exclude`. It then compresses it and the directory is gone. A `.deb`, +an AppImage, a `.app` and an `.msi` all want exactly that directory and today +must each rebuild it. Exposing it — a `${mcpp.stage_dir}` placeholder, or a +`--stage-only` mode whose output path an action can name — is one addition that +serves every format, present and future, and encodes no format's knowledge. + +**(b) Package metadata in the build program.** A build program is told +`package_name()` and `package_namespace()` and not the version, description, +license or authors. Every installer needs the version; the HuxerUI +implementation therefore asks the project to restate it in the rule's options, +where it can drift from `[package] version` with nothing to detect it. This is +a read of data mcpp already parsed. + +**(c) `--format` resolves its value through the graph.** A package declares that +it provides a format name; `--format ` finds the provider among the +resolved dependencies and hands it the staged tree. The engine holds the +*dispatch*, not the format — the same shape as `device_extensions`, where the +engine classifies a source it cannot compile and a package supplies the +compiler. This is the smallest of the three additions, because the flag, its +parsing and its position in the command are already there. + +With (a) to (c), `[pack]`'s built-in modes become one provider among several +rather than the model, and nothing new needs to join them. + +## 3. Where a platform actually decomposes + +The second question was whether Android, iOS and Web can be reached from +packages. The answer differs per platform, and the difference is not a matter +of degree. + +A target is **not** extensible from a package. `modules/toolchain-model/src/triple.cppm` +declares the identity as three strings — + +```cpp +struct Triple { + std::string arch; // "x86_64" | "aarch64" | "riscv64" | ... + std::string os; // "linux" | "macos" | "windows" + std::string env; // "gnu" | "musl" | "msvc" | "" +}; +``` + +— with `os == "none"` added for freestanding, and a **23-row +`kKnownTargets` table compiled into the binary**: + +```cpp +{ "x86_64-windows-msvc", "verified", "PE", "", "", false }, +{ "aarch64-macos", "verified", "", "", "", false }, +{ "aarch64-linux-gnu", "planned", "", "", "", false }, +{ "riscv64-none-elf", "verified", "bare", "llvm@22.1.8", "xim:picolibc-riscv@1.8.12", true }, +``` + +Bare-metal targets are **rows in that table**, not something a board-support +package introduced; a BSP supplies the runtime for an already-known target. So +the engine boundary is exact: a package can add a language, a tool, an action, +a payload and a generated module. It cannot add a triple. + +There is no separate object-format model to extend either, and that is the +sharper form of the same fact. The binary format is not a field; it is derived +from `os` at each site that needs it — `is_pe()` is `os == "windows"`, +`is_freestanding()` is `os == "none"`, and `family()`, the `cfg()` dimension, +answers `windows` or `unix` and **nothing at all** for anything else. A fourth +answer to "what does this produce" is therefore not one +addition but an addition at every such site, which is why wasm is a different +size of change from a table row. (`nasm_format()` looks like an object-format +switch and is not one: it is NASM's `-f` flag, x86-only by construction, and it +hard-errors off x86 rather than choosing.) + +With that boundary fixed, each platform decomposes into layers, and only the +first is engine work: + +| Layer | Android | iOS | Web | Owner | +|---|---|---|---|---| +| **Toolchain does modules** | **yes, with a supplied surface + one define** (3.1) | not measured | **yes, with a supplied surface** (3.1) | **prerequisite, and answered for two** | +| Target identity | `env = "android"`, one row | `os = "ios"`, one row | **new arch, new os, new object format** | **engine** | +| Toolchain payload | `xim:android-ndk` | `xim:iphoneos-sdk` | `xim:emsdk` | index | +| Sysroot | table column, or `[target.].sysroot` | same | same | engine / manifest | +| Compile and link flags | `-target aarch64-linux-android` | `-miphoneos-version-min` | `-sUSE_WEBGL2` etc. | **plugin** | +| Packaging | `.apk` | `.app` | `.html` + `.wasm` + `.js` | **plugin** | +| Signing | `apksigner` | `codesign` | — | **plugin** | +| Running | `adb install` | simulator | a browser | **plugin** (`mcpp::runner`) | +| Non-C++ glue | Gradle | Xcode project | JS bridge | **plugin** | + +The distances are unequal, and the ranking is the opposite of the usual demand +ranking: + +- **Android is the smallest.** `aarch64-linux-gnu` is already a `planned` row, + ELF is already an object format, and `aarch64` is already an arch. What is + missing is an `env` value and a sysroot that points at an NDK. +- **iOS is next.** `aarch64-macos` is `verified`, so Mach-O and the Apple half + of the toolchain model exist. What is missing is an `os` value and the + iPhoneOS SDK. +- **Web is the outlier.** A new arch (`wasm32`), a new os, a new object format, + and a driver that is a wrapper rather than a compiler mcpp drives directly. + This is [#597](https://github.com/mcpp-community/mcpp/issues/597), and it is + the only one of the three that changes the target model rather than extending + a table. + +**The consequence for planning:** every layer below the first is the same +mechanism on all three platforms. A `.apk` step, a `.app` step and an +`.html`+`.wasm` step are three members of the category section 2 names, and +none of them waits on the engine to be *written* — only to be *useful*. + +## 3.1 The prerequisite nobody lists: does the platform's toolchain do modules? + +Section 3 treats the target row as the engine's part of the work. That is true +and incomplete. mcpp is module-first — `import std` availability is one of the +eleven fingerprint inputs, and a package's interface is a BMI — so a target row +for a toolchain that cannot compile a module interface unit would resolve, build +nothing, and be worse than its absence. + +This was measured on this machine rather than assumed, and the result is better +than the shipped state of either toolchain suggests: **`import std` works on +both Android and Emscripten today, and neither needs a fork or a compiler +upgrade.** What both need is a directory their vendor chose not to install. + +### The gap, stated once — and closed by one vendor since + +An LLVM installation that supports `import std` carries a generated module +surface beside its headers. + +**The NDK ships one as of r30.** Measured 2026-09-11 against +`android-ndk-r30-linux.zip` (`Pkg.Revision 30.0.16248370`, the current LTS): +`share/libc++/v1/` holds `std.cppm`, `std.compat.cppm`, 110 `std/*.inc` and 21 +`std.compat/*.inc` — the same 133 files this section measured as **absent** on +r27, generated by Google from the same AOSP checkout as the compiler. So the +table below is r27's state, and for Android the remedy it describes is no +longer needed: there is nothing to fetch from an `llvmorg-*` tag, nothing to +substitute, and no second asset to publish. + +Two things about r30 did **not** change, and both matter more than the file +count: + +- **`-D__BIONIC_CTYPE_INLINE=` is still required.** The bionic ctype defect + reproduces verbatim on r30 / clang 21: 28 errors, confined to `cctype.inc` + and `locale.inc`, `using declaration referring to 'isalnum' with internal + linkage cannot be exported`. Google ships the surface without shipping a + build of it that works out of the box. +- **The version to match is still not derivable from the compiler.** r30's + `_LIBCPP_VERSION` is `210000` from clang `21.0.0 (based on r574158c)` — an + AOSP `toolchains/llvm-project` mirror revision, **not** a build of any public + `llvmorg-21.1.x` tag. So the pairing check below cannot compare against an + upstream tag for Android; it can only pin the value and refuse a change. + +That is the second time a guess in this section was wrong in the same +direction: **a vendor had already done the work and nobody looked.** The first +was Apple (below). The table is kept as the r27 measurement it was, because the +argument it supports — that a target row for a toolchain without the surface +would be worse than its absence — is what motivated looking at all. + +| | `xim:llvm` 20.1.7 | NDK r27 | Emscripten 4.0.19 | +|---|---|---|---| +| clang | 20.1.7 | 18.0.1 | 22.0.0git | +| `_LIBCPP_VERSION` | 200100 | 180000 | **200100** | +| `_LIBCPP_ABI_NAMESPACE` | `__1` | `__ndk1` | `__2` | +| `share/libc++/v1/std.cppm` | yes | — | — | +| `std/*.inc` + `std.compat/*.inc` | 110 + 21 | 0 | 0 | +| total | 133 files, 620 KB | **0** | **0** | + +The NDK additionally sets `_LIBCPP_HAS_NO_STD_MODULES` in its `__config_site`. +That macro appears **exactly once in the whole NDK — in `__config_site` +itself**; no header consults it. It disabled the *installation* of the module +files at libc++ build time and gates nothing in the library, which is what makes +supplying them legitimate rather than a workaround around a disabled feature. +Emscripten does not set it at all. + +### Why "just upgrade libc++ to 22" is not the answer + +Tried, and it fails for a reason worth recording. libc++'s headers are **not +portable across configurations**: `__config_site` is generated per build and +records the ABI, threading and locale decisions that build made. Pointing NDK +clang at llvm 22.1.8's headers gives + +``` +__config:13:10: fatal error: '__config_site' file not found +``` + +and the three `_LIBCPP_ABI_NAMESPACE` values in the table above are the deeper +form of the same fact: `__1`, `__ndk1` and `__2` are deliberately incompatible, +so replacing a vendor's libc++ renames every `std` symbol's ABI namespace. +"Upgrade to 22" therefore means *building* libc++ from source for that target +and accepting a platform ABI change — not a file swap. It is a real option for +a project that already static-links its C++ runtime, and it is not needed, +because matching the revision works. + +### Android: measured, works, one define + +Named modules work as shipped: + +``` +$ clang++ --target=aarch64-linux-android24 -std=c++20 --precompile m.cppm # OK +$ clang++ --target=aarch64-linux-android24 -std=c++20 -fmodule-file=m=m.pcm -c main.cpp # OK +``` + +`import std;` does not — `fatal error: module 'std' not found` — for the reason +the table gives. Supplying the surface is mechanical, because `std.cppm` is +generated: + +``` +// WARNING, this entire header is generated by utils/generate_libcxx_cppm_in.py +``` + +from `libcxx/modules/std.cppm.in` and one CMake substitution that fills +`@LIBCXX_MODULE_STD_INCLUDE_SOURCES@` with `#include` lines naming the +`std/*.inc` partitions. Taking `libcxx/modules/` from **llvmorg-18.1.8** — the +release matching `_LIBCPP_VERSION 180000` — and performing that substitution +reproduces the vendor's own 133 files. + +Compiling it against the NDK's own headers then failed, and the failure is the +interesting part: + +``` +std/cctype.inc:11: error: using declaration referring to 'isalnum' with + internal linkage cannot be exported +bionic ctype.h:127: __BIONIC_CTYPE_INLINE int isalnum(int __ch) { … } +``` + +28 errors, all of that one kind, confined to 2 of the 110 partitions +(`cctype.inc` and `locale.inc`). bionic defines the ctype functions +`static __inline`, and a using-declaration naming an internal-linkage entity +cannot be exported from a module. + +bionic anticipated this. The macro is overridable and says why: + +```c +/* All the functions in this file are trivial … we inline them by default. + * This macro is meant for internal use only, so that we can also provide + * actual symbols for any caller that needs them. */ +#if !defined(__BIONIC_CTYPE_INLINE) +#define __BIONIC_CTYPE_INLINE static __inline +#endif +``` + +With `-D__BIONIC_CTYPE_INLINE=` on the `std.cppm` compile, the surface builds +(27.5 MB BMI) and a program that uses it links: + +```cpp +import std; +int main() { + std::vector v{3,1,2}; + std::ranges::sort(v); + return std::format("{}-{}-{}", v[0], v[1], v[2]) == "1-2-3" ? 0 : 1; +} +``` + +``` +app: ELF 64-bit LSB pie executable, ARM aarch64, interpreter /system/bin/… +``` + +The binary was not executed — no device or emulator here — so this is "compiles +and links", not "runs". + +### Emscripten: measured, works, no define + +Named modules work as shipped. The module surface is absent as it is on +Android, and the version to match is **not** the one the compiler reports: +`em++` is clang 22.0.0git while its libc++ is `_LIBCPP_VERSION 200100`, LLVM +20.1. llvm 22.1.8's surface therefore fails on `'flat_set' file not found`; +llvm 20.1.7's builds with **no additional flags at all** (31 MB BMI). + +End to end, and this one did run: + +```cpp +import std; +int main() { + std::vector v{3,1,2}; + std::ranges::sort(v); + std::print("{}-{}-{}\n", v[0], v[1], v[2]); +} +``` + +``` +$ em++ -std=c++23 -fmodule-file=std=std.pcm -c app.cpp && em++ app.o std.pcm -o app.js +$ node app.js +1-2-3 +``` + +`app.wasm` is 470 KB. So the Web question splits cleanly in two, and only one +half is open: **the standard library story is answered**, and what remains is +[#597](https://github.com/mcpp-community/mcpp/issues/597)'s target model. + +### Apple: measured after all, and every guess in this section was wrong + +This section first said iOS could not be measured on a Linux host, that Apple's +libc++ is not a build of a public revision, and that the remedy might therefore +not exist. **All three are false**, measured 2026-09-11 against +`iPhoneOS26.5.sdk` (49 MB, from a public mirror) and `xim:llvm@22.1.8`: + +| | | +|---|---| +| `_LIBCPP_VERSION` | **210106** — a public revision, and `llvmorg-21.1.6` exists upstream | +| `_LIBCPP_ABI_NAMESPACE` | `__1` — upstream's own default, not a vendor-private namespace like the NDK's `__ndk1` | +| `_LIBCPP_HAS_NO_STD_MODULES` | `/* #undef */` — Apple does **not** disable std modules, unlike the NDK | +| `std.cppm` in the SDK | **0 files**, exactly as on Android and Emscripten | + +So the recipe is the same recipe: take `libcxx/modules/` from the tag matching +`_LIBCPP_VERSION`, perform the `@LIBCXX_MODULE_STD_INCLUDE_SOURCES@` +substitution, and get 133 files and 620 KB — the same count and size the other +two produce, and the same the vendor ships where it ships one at all. + +Carried end to end, from a Linux host: + +``` +$ clang++ --no-default-config --target=arm64-apple-ios18.0 -isysroot \ + -nostdinc++ -isystem /usr/include/c++/v1 -std=c++23 \ + --precompile surface/std.cppm -o std.pcm # 34 MB BMI +$ clang++ ... -fmodule-file=std=std.pcm -c app.cpp # import std; OK +$ clang++ ... -fuse-ld=lld app.o std.pcm -lc++ -o app +$ file app +app: Mach-O 64-bit arm64 executable, flags: +``` + +The binary was not executed — no device and no simulator on a Linux host — so +this is "compiles and links", which is what the `preview` tier means. + +**`--no-default-config` is load-bearing, and the reason generalises past iOS.** +`xim:llvm`'s payload ships a `bin/clang++.cfg` that injects the host's glibc and +libc++ unconditionally: + +``` +-isystem /include/c++/v1 +-isystem /include +-Wl,--dynamic-linker=/lib64/ld-linux-x86-64.so.2 +``` + +Those apply to **every** target the driver is pointed at, so a cross target that +is not Linux/glibc silently gets the host's C and C++ standard library ahead of +its own. `-nostdinc++` does not displace them — measured: with `-isysroot` and +`-nostdinc++` both given, the search list still began with the host's +`include/c++/v1` and the compile failed inside the host's `stdint.h` on +`gnu/stubs-32.h`. The criterion is `clang -v`'s search list, not whether the +compile succeeds, because a compile that reads the wrong standard library +usually succeeds. + +That is a property of the payload rather than of iOS, and it applies to any +non-Linux target driven through it. + +### What this means for xim-pkgindex + +The index has **253 recipes and none of the three**: no `android-ndk`, no +`emsdk`, no `iphoneos-sdk`. `llvm` is there and is the shape to copy. + +| Recipe | Beyond the archive it must carry | Blocked on | +|---|---|---| +| `xim:android-ndk` | `share/libc++/v1/` from llvmorg-18.1.8, a `_LIBCPP_VERSION` check, and `-D__BIONIC_CTYPE_INLINE=` on the std BMI | **nothing — measured working** | +| `xim:emsdk` | `share/libc++/v1/` from the release matching *libc++*'s version, not the compiler's | **nothing — measured working** | +| `xim:iphoneos-sdk` | licensing decides whether it installs or merely locates | a licence reading, and one measurement | + +Both writable recipes owe the same two things: **pin both halves to one +revision and refuse on mismatch** — the check is exact, `_LIBCPP_VERSION` in +`__config_site` against the release the surface came from — and **re-derive on +every vendor bump**, since the surface is a function of the vendor's libc++, +not a constant. + +The licensing asymmetry is worth stating plainly. The NDK is Apache-2.0 and +Emscripten is MIT, both redistributable; the iPhoneOS SDK is neither, which is +why cross-platform toolchains reach it through a locally installed Xcode. +`xim:iphoneos-sdk` may therefore have to be a *locator* — a recipe that finds +and pins what the machine already has, the way `msvc@system` does — rather than +an installer. mcpp already has that shape. + +## 4. The members this proposes + +Nothing here is a new repository. `mcpp-plugins` exists and its shape — one +package, one module interface unit per feature — already fits. + +| Feature | Module | Category | What it does | +|---|---|---|---| +| `dist-wix` | `mcpp.dist.wix` | dist | An MSI from a linked program and a definition it renders. Windows only. | +| `dist-appimage` | `mcpp.dist.appimage` | dist | An AppImage from a staged tree. Linux only. | +| `dist-apple` | `mcpp.dist.apple` | dist | A `.app` bundle, `Info.plist`, and `codesign`. macOS now; iOS when the row exists. | +| `dist-android` | `mcpp.dist.android` | dist | An `.apk`: Gradle invocation or direct `aapt2`/`d8`/`apksigner`. When the row exists. | +| `dist-web` | `mcpp.dist.web` | dist | The `.html`/`.js`/`.wasm` set and its loader. When the target model admits wasm. | + +Two of these can be written **today**, against the engine as it is: `dist-wix` +(a working implementation exists and would be a port, not a design) and +`dist-appimage`. `dist-apple` can be written for macOS today and gains iOS when +the row lands. The other two wait on section 3's first layer. + +### Why `dist-` and not `rules-` + +Because the taxonomy in section 1 is load-bearing and these are not rules. A +consumer reading `rules-wix` would expect a compiler it does not drive and a +translation unit; there is neither. The prefix should say which of the three +questions the member answers: + +``` +rules-* how is this translation unit compiled +tools-* what does the build program need to do itself +dist-* what comes out of the link, and in what form a user installs it +``` + +`mcpp.` stays reserved for members of this repository, unchanged, and +`mcpp.build.*` remains the engine's own module family — `mcpp.dist.*` collides +with neither. + +## 5. What `tools-embed` is missing + +`mcpp.tools.embed` covers one file at a time. `files()` writes **one header per +input**, deriving an identifier from each, and refuses `options::identifier` +because it "names one symbol and `files()` writes several". + +The case it does not cover, taken from Lib-Live2D's `cmake/EmbeddedShaders.cmake` +(34 lines of `file(READ)` and string concatenation, with a second 59-line +variant for Metal): **N inputs, one header, one table**, where each row carries +the file's name alongside its contents and the consumer iterates. + +```cpp +inline constexpr EmbeddedShader embedded_shaders[]{ + {"Standard.vert", R"(…)"}, + {"Standard.frag", R"(…)"}, +}; +``` + +This is not a new plugin. It is a `table()` entry point beside `file()` and +`files()`, with an option for the row type's name and for how the key is +derived from the path. The `write_if_different` behaviour already there is the +part that makes it cheap to call unconditionally, and it carries over. + +I have not measured how many projects want the table shape rather than the +per-file shape. One does, and its 93 lines of CMake are the evidence that the +shape is worth having; that is an argument for adding it, not a measurement of +demand. + +## 6. What a dist member owes its consumer + +The rules in `docs/30-build-mcpp.md` apply unchanged, and two of them bind +harder here than for a rule: + +**Expose a plan/submit pair.** A dist member's output is the last thing before +a user's hands, so it is the most likely to need a project-specific edit — a +different compression level, an extra file, a second signature. `generate_all(opt)` +being `submit(plan_all(opt))` is what keeps that edit from becoming a +reimplementation. + +**Failure and advice use different channels.** A packaging step that succeeds +while carrying nothing is the failure mode section 2 measured. Where a dist +member can detect an empty or implausible result, it must say so on a +*successful* build through `mcpp::warning`, because stderr on success is +discarded. + +**One `(name, version)` names one payload.** A dist member that wraps a signing +tool inherits that tool's compatibility surface. Versioning in lock-step with +the wrapped tool is legitimate and says something true. + +Two more that are specific to this category: + +**Name the input, do not harvest a directory.** Section 2's 52 KB installer is +the general case: a path that resolves to nothing is silent, and a named input +that is missing is an error. `${mcpp.target_file:}` is the mechanism — +a build program is told neither the triple nor the fingerprint, and an unknown +target name is refused rather than expanded to an empty path. + +**Declare the tool where it will be looked up.** `xpkg_dir` answers from +`MCPP_XPKG_*_DIR`, which mcpp sets for the *building* package. A dependency's +declaration provisions the payload — the log says +`Provisioning [xlings.workspace] entries (...)` — without making it visible to +a consumer's build program. A dist member that runs a payload tool therefore +declares it itself, and says so when the lookup returns empty rather than +pointing at a manifest the reader does not own. + +## 7. Staging + +The order is not the demand order, and the reason is that the lower layers are +shared: + +1. **`dist-wix`, then `dist-appimage`.** No engine change. `dist-wix` is a port + of a working implementation; `dist-appimage` is the same shape on the + platform where it is easiest to test. Both restage by hand, which is the + evidence for step 2 rather than a reason to delay them. +2. **The two engine additions of 2.1: a consumable staged tree, and package + metadata in the build program.** Both are format-neutral, and step 1 will + have shown what each costs to do without. After this, no format needs the + engine again. + +3. **`tools-embed`'s `table()`.** Independent of everything else, and the + smallest of these. +4. **`dist-apple` for macOS.** Establishes the bundle-and-sign shape on a target + that already exists, so that iOS later adds a row and not a design. +5. **`xim:android-ndk` and `xim:emsdk`, each carrying the module surface (3.1).** + Both are measured working and neither waits on the engine. Writable before + the rows, and doing them first turns each row into a small verifiable change + rather than a change plus an unknown. +6. **The Android row.** One `env` value, one table row, one payload. The + smallest of the three platform steps, and it makes `dist-android` writable. +7. **The iOS row**, then `dist-apple` extends to it — preceded by the same + measurement for Apple clang, and by whether `xim:iphoneos-sdk` can be a + payload or must be a locator. +8. **Web**, as [#597](https://github.com/mcpp-community/mcpp/issues/597) + describes. It is a target-model change and not a table row — but it is now + *only* that: 3.1 measured the standard-library half and it works, so #597 is + one problem rather than two. + +Steps 1 to 4 do not wait on a platform. Doing them first means that when a +platform row lands, the layer above it already exists — and that the row is the +only thing that had to land. + +## 8. What this does not solve + +**A published package still carries no consumer dependencies from the target +axis.** `mcpp emit xpkg` derives `xpm..deps` from the top-level +`[xlings.workspace]` only. A dist member declaring its tool on the target axis — +which is where a payload the produced code links against belongs — publishes +cleanly and fails in the consumer's link. The workaround is to declare on both +axes, and the second copy states a target fact on the host axis, which mcpp's +own guidance calls the wrong form. + +**`mcpp test` is not configurable.** A library with an existing test tree that +is not `tests/**/*.cpp` cannot use `mcpp test` at all; HuxerUI reached this and +answered it by adding a separate package whose `tests/` selects instead of +discovering. This is unrelated to plugins and is noted because it is the second +thing an ecosystem library hits. + +**Every guess along the way was wrong, including the ones this document made +about Apple.** That the gap was one file (it is 133); that a mismatched surface +fails obscurely (it names the missing header); that the blocker was libc++ (it +was bionic's `static inline` ctype); that Emscripten's libc++ version follows +its clang version (it does not); that iOS could not be measured from Linux (it +can); that Apple's libc++ is not a public revision (it is, 210106); and that +Xcode might already ship the surface (it does not, and the SDK carries none). + +The pattern is worth naming rather than just recording: **each wrong guess was +about what a vendor had done, and each was cheap to check and was not checked.** +The section that guessed least — Emscripten, where the version was read rather +than inferred — is the only one that needed no correction. + +**Neither working recipe was carried to a released payload.** Both were built +and exercised in a scratch directory. Turning each into an xim recipe — the +fetch, the substitution, the version check, the layout — is the next step and +is not done here. + +**Nothing here shortens the platform work itself.** Sections 3 and 7 say which +layer is engine and which is package; they do not make the engine layer +smaller. Android is small because the model nearly admits it already, not +because a plugin can stand in for the row. + +## 9. Review of sections 4 to 7, and what each risk is measured against + +The proposal survives review with one correction, one omission that would have +been a silent defect, and one boundary the sections above draw in the wrong +place. They are stated here before the task list because each one moves a task. + +### 9.1 The correction: `--stage-only` already shipped, under another name + +Section 2.2(a) asks for "a `${mcpp.stage_dir}` placeholder, or a `--stage-only` +mode whose output path an action can name". The second half exists. +`--format dir` sets `writeArchive = false`, and `mcpp pack` then reports +`plan.stagingRoot` — `target/dist//` — as the output. That tree +is the closure after the strip policy, the debug split and `include`/`exclude` +have run: the thing section 2.2(a) describes. + +So only the placeholder is new. `--stage-only` would be a second spelling of a +mode that ships, which is the shape this project refuses elsewhere +(`mcpp sbom` versus `mcpp emit sbom`). + +### 9.2 The omission: the requested format is a graph input, so it is graph state + +`mcpp pack --format appimage` changes what the build program submits. Anything +that changes what a build program submits changes the graph, and +`target///build.ninja` is shared mutable state that two fast paths +replay. `mcpp.build.graph_shape` exists because of exactly this class of +defect, and it already carries two such axes: `graph=` (a test graph replayed +for a plain build) and `accel=` (a device variant a flag chose, replayed for a +build that chose nothing). + +A `dist` axis with no entry on that line reproduces the same failure a third +time: `mcpp pack --format appimage`, then `mcpp build`, replays a graph +carrying a dist edge that a plain build must not have. The line therefore gains +a third field, and `is_plain_build_graph` requires it to read `none`. + +The format deliberately does **not** enter the build fingerprint. It would put +the packaging pass in its own directory and cost a full recompile to produce a +distributable from an already-built tree. The header line is the cheaper half +of that pair and is the half that answers the question the fast paths ask. + +### 9.3 The boundary in the wrong place: a staged tree is not a root filesystem + +Section 2.1 lists `.deb`, `.rpm` and AppImage as one group and 2.2(a) offers +one staged tree to all of them. Two shapes are being conflated: + +| Format | Wants | +|---|---| +| AppImage, `.app`, `.msi` | a **bundle** tree: `bin/`, `lib/`, relocatable, rooted anywhere | +| `.deb`, `.rpm` | an **FHS** tree: `usr/bin/`, `usr/lib//`, rooted at `/` | + +`mcpp pack` stages the first — it is what `--mode vendored` means, and the +`$ORIGIN` rewriting and the `run.sh` wrapper are what make it relocatable. A +`.deb` member consuming that tree must re-lay it out, and the re-layout is +`.deb`'s knowledge rather than the engine's, so this is not an argument for a +second staged tree in the engine. It is an argument about which member goes +first: **a bundle-shaped format exercises `${mcpp.stage_dir}` as it is, and an +FHS-shaped one exercises a re-layout step that would then be the thing under +test.** Section 7's choice of AppImage for step 1 is right, and the reason is +this rather than "the platform where it is easiest to test". + +### 9.4 The ordering problem sections 2.2 and 7 do not state + +A `role = "artifact"` action is a ninja edge. The staged tree is produced by +`mcpp::pack::run` in C++ **after** ninja has finished. So an artifact action +cannot depend on the staged tree in the pass that builds it, and +`${mcpp.stage_dir}` is not expressible in a single-pass pack. + +Two passes are, and every value the second one needs is already answered by the +first: + +1. `prepare_build`, with no format set. Build programs run and declare the + formats they provide; none submits a dist action, because none was asked + for. An unknown `--format` is refused **here**, before anything is compiled, + naming the set that was declared. +2. The ordinary build. The link outputs exist. +3. `make_plan` and `pack::run`. The staged tree exists at `plan.stagingRoot`. +4. `prepare_build` again, with `pack_format` and `pack_stage_dir` set to what + steps 1 and 3 answered — **not re-derived**. The claiming member submits its + action. Its command is what ninja runs. + +The re-derivation is what would have been the defect. `plan.stagingRoot` is a +function of the package name, the version, the resolved triple and the mode, +and the resolved triple is not known until prepare has run. Computing it a +second time before prepare — from `host_triple()`, say — is the shape where two +derivations of one value agree on every machine the author has and disagree on +one they do not. + +The second prepare is not free. It is bounded by the build program re-running: +the contract values are part of its re-run key unconditionally, so changing +`MCPP_PACK_FORMAT` invalidates exactly that one entry and nothing else. + +### 9.5 The declaration must not be gated on the request + +A member that emitted `mcpp:pack-format=appimage` only when +`pack_format() == "appimage"` would make the set unknowable: the engine could +never answer "which formats does this graph provide" and `--format bogus` could +name nothing. So the contract has two halves that must not be merged: + +> **Declare unconditionally. Submit conditionally.** + +This is the load-bearing rule of the whole dispatch, and it is the rule most +likely to be got wrong by a member author, because a member that gets it wrong +still works for the person who wrote it — they always pass their own format. +It therefore needs a test whose failure mode is the wrong half: a build that +requests **no** format and asserts the set is still non-empty. + +### 9.6 Risks, each with the measurement that would catch it + +| Risk | Why it would pass unnoticed | Criterion | +|---|---|---| +| The dist graph is replayed for a plain build | Both graphs live in one directory; the fast path predates any plan | `pack --format X`, then `build`, then assert no dist edge ran — the `A then B then A` shape | +| The second prepare replays a cached build program | A cached run re-emits the first pass's output, which submits nothing, so the pass succeeds and produces nothing | Assert the dist output **exists**, never that the command exited 0 | +| `${mcpp.stage_dir}` in a non-packing build | Expands to an empty string; the command then reads the build directory root, which exists | Refuse at expansion, and assert the refusal text | +| A member declares a built-in name (`tar`, `dir`) | The built-in wins and the member is silently unreachable | Refuse the collision, naming both | +| The dist action runs before staging | Only in a single-pass design; recorded so the two-pass ordering is not "simplified" away later | Assert the staged tree is non-empty **from inside the action** | +| A dist member's tool is absent | The tool is a host lookup, and an empty path becomes an argv token | Refuse in the build program, naming the tool and where it was looked for | +| The produced distributable is valid and empty | Section 2's measured 52 KB installer | Each member asserts a floor on its own output, on the **success** path, through `mcpp::warning` | + +### 9.7 The engine's half of section 3, and what is left after it + +Section 3's boundary is exact: a package can add a language, a tool, an action, +a payload and a generated module, and it **cannot add a triple**. Every layer +below the first — the `.apk` step, the `.app` step, the `.html`+`.wasm` step, +the runner, the signing, the non-C++ glue — therefore waits on a row in +`kKnownTargets` and on nothing else in the engine. So the rows are engine work +and belong in the same release as §2.2: + +| Row | Tier | What it still needs | +|---|---|---| +| `aarch64-linux-android`, `x86_64-linux-android` | `planned` | `xim:android-ndk` | +| `aarch64-ios` | `planned` | the iPhoneOS SDK, and a licence reading first (§9.8) | +| `wasm32-emscripten` | `planned` | `xim:emsdk` | + +**`planned` is a refusal, not a gap.** The tier gate answers `tier-planned` +naming the row, so `mcpp build --target aarch64-linux-android` says the +vocabulary has this target and nothing is wired yet — rather than `unknown +target`, which was false, or a build that resolves and produces nothing, which +§3.1 argues would be worse than the row's absence. Each cell is declared in +`tests/matrix/expected.tsv`, so the day a row is wired the matrix goes red and +says so. + +**Web needed one thing the other two did not, and it was not a table row.** The +binary format was never a field: it was re-derived from `os` at each site that +needed it, which is affordable while the answer has two values. `wasm32` is the +first target whose format is neither, and a third value turns those derivations +into an addition at every such site — where a missed site does not fail, it +silently answers ELF. `ObjectFormat` is that addition made once. This is the +substance of [#597](https://github.com/mcpp-community/mcpp/issues/597)'s +"changes the target model rather than extending a table", and with §3.1 having +answered the standard-library half, #597 is now one problem rather than two. + +**Android's placement is the modelling decision.** `env = "android"` on a +`linux` OS, not `os = "android"`: the kernel is Linux, so ELF, the `unix` +family and `nasm -f elf64` are already right, and an OS value would have made +every one of them wrong by default and required a new answer at each site. What +differs from `gnu` is bionic, the loader path and the SDK, which is what an +`env` value is for. + +What remains outside this pass, each blocked on something no engine work +supplies: + +- **`xim:android-ndk`, `xim:emsdk`** (§7 step 5). Both recipes are measured + working in a scratch directory (§3.1); turning either into a payload means + fetching and republishing a multi-gigabyte vendor toolchain with a derived + 133-file module surface. That is its own release, not a side effect of this + one — and the rows landing first is exactly §7's ordering argument, which + said doing the payloads first would make each row small. The rows turned out + to be the cheap half either way. +- **The iPhoneOS SDK.** Not measurable on a Linux host, and §9.8 gives the + decision procedure rather than the measurement. +- **`dist-android`, `dist-web`**. Each waits on its payload, not on its row. + +### 9.8 The iOS SDK: a three-tier policy rather than an open question + +Section 3.1 leaves `xim:iphoneos-sdk` as "licensing decides whether it installs +or merely locates", and §8 repeats it. The decision procedure is not a +measurement — it is a preference order, and stating it removes the open +question without taking the measurement: + +1. **Redistribute, if the licence permits it.** A published payload with a + GitCode mirror, like every other `xim` toolchain package. Publicly mirrored + SDK trees exist — `https://github.com/xybp888/iOS-SDKs` is one — and whether + this tier is reachable is a licence reading of the SDK itself, not of the + mirror. +2. **Fetch from upstream, without a CN mirror.** A recipe that downloads at + install time from the upstream URL and mirrors nothing. This is what a + licence that permits use but not redistribution allows, and declining the + mirror is the point: a mirrored copy *is* redistribution, so the tier is + defined by what it refuses to do. +3. **Locate what the machine already has.** The `msvc@system` shape: find, pin + and report an installed Xcode, install nothing. Correct under any licence, + and the only tier that cannot serve a machine without Xcode. + +Tier 3 always works and is therefore the floor, not the goal. The recipe should +reach for the lowest tier the licence allows and say in its description which +tier it took, because a consumer reading "locator" needs to know that is a +licence conclusion rather than an unfinished recipe. + +## 10. The task list + +Four repositories. One pull request each, in this order, because each depends +on the one above it being released rather than merely merged. + +### 10.1 `mcpp` — the three format-neutral additions + +| # | Task | Depends on | +|---|---|---| +| E1 | `MCPP_PKG_VERSION` / `_DESCRIPTION` / `_LICENSE` / `_AUTHORS` / `_REPO` in the build-program environment, with accessors | — | +| E2 | `MCPP_PACK_FORMAT` in that environment, and `mcpp::pack_format()` | — | +| E3 | The `mcpp:pack-format=` outlet, collected onto the plan | — | +| E4 | `${mcpp.stage_dir}`: expansion for Artifact actions, refusal elsewhere, the stage manifest as an implicit input | E2 | +| E5 | `--format` accepts a provided name; the refusal names the available set | E3 | +| E6 | The two-pass pack pipeline (§9.4) | E4, E5 | +| E7 | `dist=` on the graph header line; `is_plain_build_graph` requires `none` | E6 | +| E8 | `docs/10`, `docs/30`, `docs/31`, and a new `docs/35`; the `zh` mirror of each | E1–E7 | +| E9 | Unit tests for E3–E5, e2e for E6–E7, each with the §9.6 criterion | E1–E7 | + +### 10.2 `mcpp-plugins` — one member per verified platform + +| # | Task | Depends on | +|---|---|---| +| P1 | `tools-embed`: a `table()` entry point (§5) | — | +| P2 | `dist-appimage` → `mcpp.dist.appimage` | mcpp released, X1 | +| P3 | `dist-wix` → `mcpp.dist.wix` | mcpp released | +| P4 | `dist-apple` → `mcpp.dist.apple`, macOS half only | mcpp released | +| P5 | `MCPP_VERSION` in CI raised to the engine that carries E1–E7 | E1–E7 released | +| P6 | Plan-level tests on all three runners; end-to-end each on its own | P1–P4 | + +### 10.3 `xim-pkgindex` — the one payload this needs + +| # | Task | Depends on | +|---|---|---| +| X1 | `xim:appimagetool` | — | + +### 10.4 `mcpp-index` — publication + +| # | Task | Depends on | +|---|---|---| +| I1 | `mcpp:plugins@0.3.0` | P1–P6 released | + +The engine tasks are the only ones on the critical path. P1 and X1 do not wait +on anything. diff --git a/.agents/docs/README.md b/.agents/docs/README.md index 7fce20b7d..40d7d388e 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded --- ``` -274 records. +275 records. ## By subject @@ -40,10 +40,15 @@ Records that declare one. Everything else is listed by date below. - [Two answers and two silences: the scanner's second grammar, and the manifest keys nothing reads](2026-09-09-two-answers-and-two-silences.md) — active +### plugins + +- [The category the plugin taxonomy does not name, and what a platform actually decomposes into](2026-09-11-distribution-plugins-and-platform-decomposition.md) — active + ## By date ### 2026-09 +- [The category the plugin taxonomy does not name, and what a platform actually decomposes into](2026-09-11-distribution-plugins-and-platform-decomposition.md) — active - [Two answers and two silences: the scanner's second grammar, and the manifest keys nothing reads](2026-09-09-two-answers-and-two-silences.md) — active - [A dlopen surface no closure walks, and a process with two unwinders](2026-09-09-dlopen-surface-and-two-unwinders.md) — landed - [The documentation as a book: a chapter-by-chapter design](2026-09-08-the-documentation-as-a-book.md) — active diff --git a/CHANGELOG.md b/CHANGELOG.md index cf282c38f..035f6f3e8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,71 @@ ## [Unreleased] +## [2026.9.11.1] - 2026-09-11 + +### `mcpp pack --format ` 分派到包,而引擎里不再需要住进任何一种分发格式 + +`tar` 与 `dir` 回答的问题,和 `msi` 与 `appimage` 回答的问题是同一个 —— 输出取什么 +形状 —— 所以它们是一个 flag 的取值,而不是第二个 flag 的开端。分界是:`mcpp pack` +拥有机制,以及那一种通用格式(一个解开就能跑的归档,它不需要知道任何别人的发布); +其余每一种格式都住在包里。dpkg 的 control 字段、AppImage 的 runtime、WiX 的 schema、 +Apple 的公证,其中任何一个被绑进引擎,都会把一次 mcpp 的发布耦合到一次 mcpp 并不控制 +的发布上。这与本项目早已为语言做过的论证是同一个 —— Slang 被支持,而引擎里没有它的 +名字。 + +三样与格式无关的东西被加进引擎,「与格式无关」正是判断某样东西该不该进引擎的判据: + +- **一棵 `artifact` action 可以消费的暂存树。** `mcpp pack` 一直在算它 —— 依赖闭包, + 过了 strip 策略、调试信息拆分与 `include`/`exclude` —— 然后把它压掉,目录就没了。 + `${mcpp.stage_dir}` 把它暴露出来。它是一个 **bundle** 树(`bin/`、`lib/`、可重定位), + 这正是 AppImage、`.app`、`.msi` 要的形状;要一棵 FHS 树的格式(`.deb`、`.rpm`)自己 + 负责重排,因为一个文件该落在哪个目录是那个格式的知识。 +- **`[package]` 的其余字段进入构建程序。** `MCPP_PKG_VERSION` / `_DESCRIPTION` / + `_LICENSE` / `_AUTHORS` / `_REPO`,以及对应的 `mcpp::package_*()`。每一种安装包格式 + 都要写版本号;在这之前,项目只能把版本号在成员自己的 options 里再写一遍,而那份副本 + 会与 `[package]` 漂移,且没有任何东西能发现。 +- **`--format` 经图解析它的取值。** 包用 `mcpp::provides_pack_format("")` 声明, + `--format ` 在解析后的依赖里找到提供方并把暂存树交给它。未知取值点名**当下确实 + 可用**的那些,而不是一份固定清单,并且这次拒绝发生在任何东西被编译之前。 + +**无条件声明,有条件提交。** 这是整套分派最承重的一条规则,也是最容易被成员作者写错 +的一条 —— 因为写错了对作者自己仍然照常工作:他永远传的是自己那个格式。声明必须不加闸, +否则引擎永远回答不出「这张图提供哪些格式」;提交必须加闸,否则普通 `mcpp build` 会多出 +一条它不该有的边。对一个谁都没为之提交的格式,mcpp 会拒绝并点名,而不是报告一次「什么 +包都没产出」的成功打包。 + +**`mcpp pack --format ` 会 prepare 两次,而两次之间没有任何值被重新推导。** 一条 +`artifact` action 是一条 ninja 边,而暂存树是 mcpp 在链接**之后**产出的,所以这棵树不 +可能成为构建出它自己那一次 pass 的输入。第一趟收集声明并拒绝未知格式;随后是构建与暂存; +第二趟设上 `pack_format` 与 `pack_stage_dir` 并构建提供方提交的那条边。第二趟用的三元组 +与暂存路径,都是第一趟和 `make_plan` 已经回答过的 —— 重新推导一遍会得到那种「在作者所有 +机器上都一致、只在他没有的那台上不一致」的缺陷。 + +**build.ninja 的头行多了第四个字段 `dist=`,而快路径要求它读作 `none`。** 格式故意 +**不进指纹**:进了就要为「把一棵已经构建好的树打成包」付一次全量重编。于是两张图落在同 +一个目录里,而 `target///build.ninja` 是被两条快路径回放的共享可变状态 —— +这正是这一行上另外两个字段(`graph=`、`accel=`)已经各自记过一次的那种失败。判据放在 +单元测试里而不是端到端:实测(2026-09-11)即使忽略这个字段,pack 之后的普通构建也会因为 +更早的一个新鲜度条件而重新生成图,所以端到端断言无论字段是否生效都会通过,连字段被删掉 +都照样通过。 + +**`${mcpp.stage_dir}` 在两种位置上是拒绝而不是空展开:** 本次构建不在打包时,以及 +role 不是 `artifact` 时。一个空路径仍然是命令接受的 token,而工具随后读到的是构建目录 +根 —— 那个目录存在,所以这个错误会产出一个看起来合理的产物而不是一条诊断。实测过的原型 +是那个「有效的、空的、52 KB 的安装包,并且没有任何诊断」。 + +写了 `${mcpp.stage_dir}` 的 action 会自动获得一条对 `<暂存树>.stage-manifest` 的依赖 +(一个兄弟文件,永不是成员,所以它不会跑进任何人的安装包里)。依赖由引擎添加,因为「用 +了」本身就意味着「依赖」:没有它,这条边只在链接产物变化时才变脏,而一个闭包多出了某个 +依赖的共享库、同时程序自己的字节没变的情况,会把上一次的可分发物原地留下并报告为最新。 + +构建程序协议升到 v9(`mcpp:pack-format=`)。这条指令带非空 `tag`,因此会随构建程序的 +缓存记录一起被回放 —— 读它的那一趟是 `mcpp pack`,而那从来不是一个项目的第一次构建。 + +文档:`docs/10`(`--format` 那一个轴)、`docs/30`(三类成员的分类表、新占位符与新访问器)、 +`docs/31`(分发成员的六条约束),中英双份。 + + ## [2026.9.10.2] - 2026-09-10 ### dlopen 面检查:不适用的那一趟也会发布记录,并且不会盖掉已经量出来的答案 diff --git a/README.md b/README.md index 578ec150e..49ae4ca9a 100644 --- a/README.md +++ b/README.md @@ -419,6 +419,9 @@ list` reports for this machine): | `aarch64-none-elf` · `x86_64-none-elf` | llvm 22 — bare metal, no C library by default ² | preview | | `thumbv7em-none-eabi` · `thumbv8m.base-none-eabi` · `thumbv8m.main-none-eabihf` | llvm 22 — Cortex-M4/M7 soft float, M23, M33F/M55F ² | preview | | `riscv64-linux-musl` · `aarch64-linux-gnu` · `x86_64-macos` | — | planned | +| `aarch64-linux-android` · `x86_64-linux-android` | needs `xim:android-ndk`; `import std` measured working on the NDK's clang | planned | +| `aarch64-ios` | needs the iPhoneOS SDK, which is a licence question before it is a packaging one | planned | +| `wasm32-emscripten` | needs `xim:emsdk`; `import std` measured working on `em++`, and the target model is [#597](https://github.com/mcpp-community/mcpp/issues/597) | planned | `verified` an image has been built **and run** for the row, qemu and wine included · `preview` it builds and links, and no emulator run has been recorded diff --git a/README.zh-CN.md b/README.zh-CN.md index c029f5435..075de37b7 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -406,6 +406,9 @@ mcpp 的身份模型是两条正交轴:**工具链** = `family@version`(family | `aarch64-none-elf` · `x86_64-none-elf` | llvm 22——裸机,默认不带 C 库 ² | preview | | `thumbv7em-none-eabi` · `thumbv8m.base-none-eabi` · `thumbv8m.main-none-eabihf` | llvm 22——Cortex-M4/M7 软浮点、M23、M33F/M55F ² | preview | | `riscv64-linux-musl` · `aarch64-linux-gnu` · `x86_64-macos` | — | planned | +| `aarch64-linux-android` · `x86_64-linux-android` | 待 `xim:android-ndk`;`import std` 在 NDK 自带的 clang 上已实测可用 | planned | +| `aarch64-ios` | 待 iPhoneOS SDK,而它先是一个许可问题再是一个打包问题 | planned | +| `wasm32-emscripten` | 待 `xim:emsdk`;`import std` 在 `em++` 上已实测可用,目标模型见 [#597](https://github.com/mcpp-community/mcpp/issues/597) | planned | `verified` 该行的镜像已被构建**并运行**过,qemu 与 wine 都算 · `preview` 可构建 可链接,未记录过模拟器运行 · `planned` 已登记在词表中,尚未接线 —— 面向这类目标 diff --git a/docs/10-pack-and-release.md b/docs/10-pack-and-release.md index d99bb095e..b008014db 100644 --- a/docs/10-pack-and-release.md +++ b/docs/10-pack-and-release.md @@ -140,6 +140,7 @@ mcpp pack --mode self-contained # alias: --mode bundle-all mcpp pack --target x86_64-linux-musl # equivalent to --mode static mcpp pack --target aarch64-linux-musl # ARM64 equivalent mcpp pack --format dir # output as a directory, no tarball +mcpp pack --format appimage # a format a package in the graph provides mcpp pack -o myapp.tar.gz # filename only: lands at target/dist/myapp.tar.gz mcpp pack -o /abs/path/myapp.tar.gz # includes a directory: output to the literal path mcpp pack --profile dev # build with a different profile (default: release) @@ -147,6 +148,47 @@ mcpp pack --no-strip # ship the artifacts as built mcpp pack --debug-symbols dbg/ # write the separated *.debug files under dbg/ ``` +### `--format` owns one axis, and the engine owns two of its values + +`tar` and `dir` answer the same question `msi` and `appimage` answer — what +shape does the output take — so they are values of one flag rather than the +beginning of a second one. The split between what the engine holds and what a +package holds is: + +> **`mcpp pack` owns the mechanism and the one universal format. Every other +> format lives in a package, and `mcpp pack` dispatches to it.** + +The universal format is what it already produces: an archive that extracts and +runs. It is universal in the only sense that matters here — it needs no +knowledge of anyone else's release. Everything past it does. dpkg's control +fields, AppImage's runtime, WiX's schema, Apple's notarisation, Android's +signing scheme: each one bound into the engine would couple an mcpp release to +a release mcpp does not control. The same argument the project already made for +languages, where Slang is supported without being named in the engine. + +So the value set is open (mcpp 2026.9.11.1+). `--format ` finds the +package in the resolved graph that declares `` and hands it the staged +tree; an unknown value names what *is* available rather than a fixed list: + +``` +error: unknown --format 'bogus'. + available in this build: tar, dir, appimage + A format past `tar` and `dir` comes from a package in the resolved graph, which declares + it with `mcpp::provides_pack_format("")` in its build program. Add the package + that provides 'bogus' to [build-dependencies] and activate its feature. +``` + +The refusal arrives before anything is compiled. Writing such a package is +[Producing a distributable](30-build-mcpp.md#producing-a-distributable-pack_format--stage_dir-20269111); +the engine's three additions are a staged tree an `artifact` action can consume, +the rest of `[package]` in the build program, and this dispatch. Each is +format-neutral, which is the test for whether something belongs in the engine +at all. + +A dispatched format applies to a **program** target. A library package ships an +interface plus prebuilt binaries per triple and has no single staged tree, so +`mcpp pack --format ` is refused rather than ignored. + When `-o` is given a bare filename, the output is placed under `target/dist/`; when it includes a directory (relative or absolute), the literal path is used. @@ -421,8 +463,13 @@ macOS **program** bundling (the Mach-O dependency closure, via `otool -L` / `LC_LOAD_DYLIB`, and `install_name_tool` for relocation) is still on the roadmap; until it lands `mcpp pack ` refuses on that format rather than producing something that only looks like a bundle. Windows DLL bundling beyond -the current `.zip`, and distribution formats such as `.deb` / `.rpm` / AppImage, -are also on the roadmap. This document evolves alongside the -`mcpp pack` implementation; for the latest options, refer to -`mcpp pack --help`. +the current `.zip` is also on the roadmap. + +Distribution formats such as `.deb`, `.rpm`, AppImage and `.msi` are **not** on +this list, and that is a decision rather than an omission: they live in +packages and reach the user through `--format `, for the reason the +section above gives. Nothing further needs to join `[pack]`'s built-in modes. + +This document evolves alongside the `mcpp pack` implementation; for the latest +options, refer to `mcpp pack --help`. diff --git a/docs/21-the-target-triple.md b/docs/21-the-target-triple.md index b6a74e998..ea6d18354 100644 --- a/docs/21-the-target-triple.md +++ b/docs/21-the-target-triple.md @@ -38,19 +38,26 @@ and the build reports what it resolved. | Segment | Content | Example | |---|---|---| -| `arch` | instruction set | `x86_64`, `aarch64`, `riscv64` | -| `os` | operating system, or `none` | `linux`, `windows`, `macos`, `none` | -| `env` | see below — it is a different axis per platform | `gnu`, `musl`, `msvc`, `elf` | +| `arch` | instruction set | `x86_64`, `aarch64`, `riscv64`, `wasm32` | +| `os` | operating system, or `none` | `linux`, `windows`, `macos`, `ios`, `emscripten`, `none` | +| `env` | see below — it is a different axis per platform | `gnu`, `musl`, `msvc`, `android`, `elf` | The third segment is the one that repays attention, because it does not name the same kind of thing everywhere: | Platform | `env` names | Values | |---|---|---| -| `linux` | the **C library** | `gnu` (glibc), `musl` | +| `linux` | the **C library** | `gnu` (glibc), `musl`, `android` (bionic) | | `windows` | the **object ABI** | `gnu` (Itanium C++ ABI), `msvc` (Microsoft's) | | `none` | the **object format** | `elf` | -| `macos` | nothing; the platform carries no segment | — | +| `macos`, `ios`, `emscripten` | nothing; the platform carries no segment | — | + +`android` is a **C library** and therefore sits where `musl` sits, on a `linux` +OS. That placement is the whole of the modelling decision: the kernel *is* +Linux, so ELF, the `unix` family and `nasm -f elf64` are already right, and an +`os = "android"` would have made every one of them wrong by default and needed +a new answer at each site. What differs from `gnu` is bionic, the loader path +and the SDK — which is exactly what an `env` value is for. On Windows the segment is frequently misread, because the word `gnu` suggests a C library that is not there. Measured on an artefact built for @@ -68,6 +75,30 @@ openkal. `gnu` is LLVM's label for the non-MSVC ABI, inherited from MinGW, and clang requires that spelling to select the right internal toolchain. mcpp cannot rename it. +### The object format is an axis, not a derivation + +A triple's binary format used to be nothing at all: it was re-derived from `os` +wherever it was needed. `is_pe()` asked `os == "windows"`, artifact naming asked +again, the packer asked a third time. That is affordable while the answer has +two values. + +`wasm32` is the first target in mcpp's vocabulary whose format is neither, and a +third value turns those derivations into an addition **at every such site** — and +a site that is missed does not fail. It silently answers ELF, because ELF is +what every `else` branch in the tree assumes. So the format is now one answer: + +| target | format | +|---|---| +| `x86_64-linux-gnu`, `aarch64-linux-android`, `riscv64-none-elf` | ELF | +| `aarch64-macos`, `aarch64-ios` | Mach-O | +| `x86_64-windows-gnu`, `x86_64-windows-msvc` | PE | +| `wasm32-emscripten` | wasm | + +It is **not** the same question as "is there an operating system to link +against". A bare-metal RISC-V image is ELF with no OS; a wasm module has an +OS-like layer (Emscripten's POSIX emulation) and is not ELF. Merging the two +axes is the mistake this replaces. + ## Declining The Third Segment `-` is a complete target on every platform: @@ -444,6 +475,10 @@ other's rows. | `thumbv8m.base-none-eabi` | preview | `llvm@22.1.8` | payload | payload | payload | payload | | `thumbv8m.main-none-eabi` | verified | `llvm@22.1.8` | payload | payload | payload | payload | | `thumbv8m.main-none-eabihf` | preview | `llvm@22.1.8` | payload | payload | payload | payload | +| `aarch64-linux-android` | planned | — | planned | planned | planned | planned | +| `x86_64-linux-android` | planned | — | planned | planned | planned | planned | +| `aarch64-ios` | planned | — | planned | planned | planned | planned | +| `wasm32-emscripten` | planned | — | planned | planned | planned | planned | `payload` a toolchain payload here produces it · `graph` no payload, but a dependency can supply the system · `system` located on the machine, not diff --git a/docs/30-build-mcpp.md b/docs/30-build-mcpp.md index 56e7b9195..43844a8ab 100644 --- a/docs/30-build-mcpp.md +++ b/docs/30-build-mcpp.md @@ -463,6 +463,12 @@ attach: | `object` | join the **link** set | the link edge consumes them | a resource compiler, `objcopy` embedding a blob, a generated `.def`, a pre-built `.o` | | `artifact` | a new file | its *inputs* are link outputs, so it runs after the link | codesign, packaging, size budgets | +`artifact` is also the only role that may name `${mcpp.stage_dir}` — see +[Producing a distributable](#producing-a-distributable-pack_format--stage_dir-20269111) +below. The other three run before or alongside the link, so there is nothing +staged for them to read, and mcpp refuses the placeholder rather than expanding +it to a path that happens to exist. + No phase machinery is involved. `object` and `artifact` are sequenced by ninja's own file dependencies — which is also why an `artifact` action cannot double-apply itself the way a naive "post-build hook" would. `source` and a @@ -553,6 +559,89 @@ scan agrees with what the generator will emit — the same assertion-plus- verification trade `[modules].scan_overrides` makes, and the compiler's own P1689 output checks it at build time. +### Producing a distributable: `pack_format` / `stage_dir` (2026.9.11.1+) + +An `.msi`, a `.deb`, an AppImage and a signed `.app` are none of the four roles' +usual work and all of them are `artifact`: each consumes **link outputs** and +produces something a user installs. What the engine adds for them is a +mechanism and no format at all. + +`mcpp pack --format ` resolves `` through the resolved graph, the +same way `--target` reaches a triple the engine did not have to know +individually. `tar` and `dir` remain the archive shapes `mcpp pack` owns; +everything past them comes from a package. + +A provider has two halves, and they must not be merged: + +```cpp +import mcpp; +#include +#include + +int main() { + // Half one, unconditional. + mcpp::provides_pack_format("appimage"); + + // Half two, conditional. + if (std::string_view(mcpp::pack_format()) != "appimage") return 0; + + const std::string out = std::string(mcpp::out_dir()) + "/app.AppImage"; + mcpp::action a; + a.id = "appimage"; + a.role = "artifact"; + a.arg(tool).arg("${mcpp.stage_dir}").arg(out.c_str()) + .input("${mcpp.target_file:app}") + .output(out.c_str()) + .submit(); + return 0; +} +``` + +**Declare unconditionally, submit conditionally.** The declaration is what lets +the engine answer a question the requesting build cannot: `--format bogus` +names what *is* available, and `--help` says "any format the resolved graph +provides". Both read the set collected from a pass that asked for nothing. A +member that declared only when asked still works for its author — they always +pass their own format — and makes the set unknowable for everyone else. mcpp +refuses a format nothing submitted for, rather than reporting a pack that +produced no package. + +**`mcpp pack --format ` prepares twice.** An `artifact` action is a ninja +edge and the staged tree is produced by mcpp *after* the link, so the tree +cannot be an input of the pass that built it. The first pass collects the +declarations and refuses an unknown format before anything is compiled; the +build and the staging then happen; the second pass sets `pack_format` and +`pack_stage_dir` and builds the edge the provider submits. Nothing in the +second pass is re-derived — the triple and the staged path are what the first +pass and the staging already answered. + +**The staged tree is a bundle, not a root filesystem.** It is what +`--mode vendored` means: `bin/`, `lib/`, relocatable, rooted anywhere, after the +strip policy, the debug-symbol split and `include`/`exclude`. An AppImage, a +`.app` and an `.msi` want it as it stands. A format that wants an FHS tree +(`.deb`, `.rpm`) owns the re-layout, because which directory a file belongs in +is that format's knowledge and not the engine's. + +`mcpp pack --format dir` writes the same tree to a path and stops, which is how +a person inspects what a member will be handed. + +**`mcpp pack` reports the artifact actions the request introduced.** An action +present whether or not a format was asked for — a codesign stamp, a size budget +— is not the distributable, and naming one would be a wrong answer that looks +like a right one. Nothing about the criterion is a property of the member: a +format that packages one named program and never reads the staged tree is +recognised exactly as one that consumes the whole closure. A format nothing +submitted for is refused by name. + +**An action that names `${mcpp.stage_dir}` gains a dependency on the tree's +manifest.** mcpp writes `.stage-manifest` — a sibling, never a +member, so it does not travel inside anyone's installer — listing each staged +file's size and relative path. The dependency is added by the engine because +the use implies it: without it the edge is dirty only when a link output +changes, and a closure that grew a dependency's shared library while the +program's own bytes did not would leave the previous distributable in place, +reported as up to date. + Commands are an **argv, not a shell string** (no shell is assumed — Windows has none to rely on), and the only interpolations are a closed set: @@ -562,6 +651,7 @@ none to rely on), and the only interpolations are a closed set: | `${mcpp.bin_dir}` | where produced binaries land | | `${mcpp.compile_db}` | path to `compile_commands.json` (what clang-tidy's `-p` wants) | | `${mcpp.target_file:}` | the built file of target `` | +| `${mcpp.stage_dir}` *(2026.9.11.1+)* | the tree `mcpp pack` staged, absolute. `artifact` role only, and only under `mcpp pack --format ` | The raw stdout protocol above remains the low-level substrate; `import mcpp;` is the typed layer over it. @@ -671,6 +761,13 @@ The running program receives the build context as `MCPP_*` variables | `MCPP_LANGUAGE_MODULES` *(2026.9.7.1+)* | -- | `1` when the declaring package sets `[language] modules`, `0` otherwise. A rule that GENERATES a consumer-facing declaration reads it to choose between a module interface and a header, so a project states that once and never again. An older engine leaves it absent, which a rule reads as `0` -- the behaviour every consumer had before the variable existed | | `MCPP_PKG_NAME` *(2026.9.7.1+)* | -- | The `[package] name` of the package this program builds. Every name a rule generates is derived from it: the module a consumer imports, the namespace the accessors sit in, the symbols in a generated header. Before it existed the closest available answer was the leaf of `MCPP_MANIFEST_DIR`, which is a directory name -- so a package named `vulkan-saxpy` in a directory named `app` generated `app.shaders`, and every `/app/` in a workspace claimed the same module. Absent under an older engine, which a rule reads as a signal to keep its previous derivation | | `MCPP_PKG_NAMESPACE` *(2026.9.7.1+)* | -- | The `[package] namespace`. Empty when the package declares none. A rule that must produce a name unique across an index uses the pair rather than the name alone, because package identity is `(namespace, name)` | +| `MCPP_PKG_VERSION` *(2026.9.11.1+)* | `mcpp::package_version()` | The `[package] version`. Every installer format states a version; without this the project had to restate it in the member's own options, where the copy drifts from `[package]` with nothing able to detect it | +| `MCPP_PKG_DESCRIPTION` *(2026.9.11.1+)* | `mcpp::package_description()` | The `[package] description`. Empty when the package declares none | +| `MCPP_PKG_LICENSE` *(2026.9.11.1+)* | `mcpp::package_license()` | The `[package] license` | +| `MCPP_PKG_AUTHORS` *(2026.9.11.1+)* | `mcpp::package_authors()` | The `[package] authors`, joined with `;`. Not `,`: an author entry is conventionally `Name ` and a name may carry a comma, so a comma-joined list cannot be split back into the entries it was made from | +| `MCPP_PKG_REPO` *(2026.9.11.1+)* | `mcpp::package_repo()` | The `[package] repo` | +| `MCPP_PACK_FORMAT` *(2026.9.11.1+)* | `mcpp::pack_format()` | The `--format` value of the `mcpp pack` pass this program is part of; empty for every ordinary build. The empty value is the one that carries the meaning — a member gates its submission on this, so `mcpp build` has the graph it always had | +| `MCPP_PACK_STAGE_DIR` *(2026.9.11.1+)* | `mcpp::pack_stage_dir()` | Where `mcpp pack` has already staged the closure, absolute; empty when this build is not packaging. Read it to decide the shape of the work; write `${mcpp.stage_dir}` into the action, so the path in the graph and the path the program read cannot disagree | | `MCPP_DEVICE_SOURCES` *(2026.9.5.2+)* | `mcpp::device_sources()` | the device-kind sources (`.cu`, `.hip`, …) the package's effective `sources` match, package-root-relative, one per line; empty when there are none. The engine compiles none of them — the rule package this program imports turns each into an `mcpp::action`. Already narrowed: a `{ glob, accel }` entry the build does not cover contributes nothing, so `--no-accel` yields an empty list | | `MCPP_OUT_DIR` | `mcpp::out_dir()` | a writable scratch/output dir owned by mcpp | | `MCPP_MANIFEST_DIR` | `mcpp::manifest_dir()` | the package root (= CWD) | @@ -786,14 +883,30 @@ and warns when the two disagree — Nothing breaks; the name claims an origin the package does not have. A rule outside the project picks its own prefix. -**A tool is not a rule.** A rule states how a translation unit is compiled by a -compiler mcpp does not drive: it submits an action and the engine schedules it. -A tool states something the build program needs that no compiler performs, and -does it while the program runs. `mcpp.tools.embed` (feature `tools-embed`, +**A tool is not a rule, and a distributable is neither.** Three kinds of work, +and the member's prefix says which of the three questions it answers: + +| Prefix | Question | Compiles a TU | Runs in the build program | +|---|---|---|---| +| `rules-*` | how is this translation unit compiled | yes | no | +| `tools-*` | what does the build program need to do itself | no | yes | +| `dist-*` *(2026.9.11.1+)* | what comes out of the link, and in what form a user installs it | no | no | + +A rule states how a translation unit is compiled by a compiler mcpp does not +drive: it submits an action and the engine schedules it. A tool states +something the build program needs that no compiler performs, and does it while +the program runs. `mcpp.tools.embed` (feature `tools-embed`, mcpp 2026.9.5.4+) is the first: it writes a data file into a header the program compiles in, as a byte array or a 32-bit word array, and rewrites nothing when the content is unchanged, so calling it unconditionally costs no rebuild. +A `dist-*` member fits neither definition. It does not compile a translation +unit and it does not run while the build program runs: it consumes link outputs +and produces something a user installs, through an `artifact` action and +`mcpp pack --format `. The prefix matters because the taxonomy is +load-bearing — a consumer reading `rules-wix` would expect a compiler and a +translation unit, and there is neither. + `examples/09-heterogeneous/cuda` and `examples/09-heterogeneous/vulkan` consume `mcpp.rules.cuda` and `mcpp.rules.spirv` from `mcpp:plugins`, the way any project does. diff --git a/docs/31-authoring-a-rule-package.md b/docs/31-authoring-a-rule-package.md index dd770f425..92ffd6c9b 100644 --- a/docs/31-authoring-a-rule-package.md +++ b/docs/31-authoring-a-rule-package.md @@ -301,6 +301,61 @@ that declares the edge under another key can supply it, and refuses with a message naming what it looked for rather than running a command with an empty path. +## Authoring a distribution member (mcpp 2026.9.11.1+) + +A `dist-*` member is neither a rule nor a tool. It does not compile a +translation unit and it does not do its work while the build program runs: it +consumes **link outputs** and produces something a user installs — an `.msi`, a +`.deb`, an AppImage, a signed `.app`. The mechanism is an `artifact` action and +`mcpp pack --format `, documented in +[30 — Producing a distributable](30-build-mcpp.md#producing-a-distributable-pack_format--stage_dir-20269111). + +Everything in this chapter applies unchanged. Six things bind harder here, and +each is a mistake this category makes and a rule does not. + +**Declare unconditionally, submit conditionally.** `provides_pack_format` is +what the engine reads to answer "which formats does this graph provide", on a +build that asked for none. A member that declares only when asked works for its +author, who always passes their own format, and makes the set unknowable for +everyone else. + +**Expose a plan and a submit.** A distributable is the last thing before a +user's hands, so it is the most likely part of a build to need a +project-specific edit: a different compression level, one extra file, a second +signature. `generate_all(opt)` being exactly `submit(plan_all(opt))` is what +keeps such an edit from becoming a reimplementation of the member. + +**Name the input; do not harvest a directory.** A path that resolves to nothing +is silent, and a named input that is missing is an error. Measured: a WiX action +that bound a directory and harvested it produced a **valid, empty, 52 KB +installer with no diagnostic** when the path resolved to nothing on Windows. +`${mcpp.target_file:}` is the mechanism — a build program is told neither +the triple nor the fingerprint, and an unknown target name is refused rather +than expanded to an empty path. + +**Assert a floor on the member's own output, on the success path.** Where a +member can tell that its result is empty or implausible, it must say so through +`mcpp::warning`, because stderr on a successful build is discarded. This is the +same failure the paragraph above measured, caught one layer later. + +**Declare the tool where it will be looked up.** `xpkg_dir` answers from +`MCPP_XPKG_*_DIR`, which mcpp sets for the package *being built*. A +dependency's declaration provisions the payload without making it visible to a +consumer's build program, so a member that runs a payload tool declares it +itself — and says so when the lookup returns empty, rather than pointing at a +manifest the reader does not own. + +**A build must not reach the network, and a wrapped tool may.** Measured on +`appimagetool` 1.9.1: it downloads its type-2 runtime stub from a GitHub +release on every invocation unless `--runtime-file` names a local copy. A member +that wraps such a tool has to supply the file from its declared payload. +Install time is when a download is legitimate; build time is not, and a build +that fetches is neither reproducible nor usable offline. + +**One `(name, version)` names one payload.** A member that wraps a signing or +packaging tool inherits that tool's compatibility surface, so versioning in +lock-step with the wrapped tool is legitimate and says something true. + ## Current limitations - A rule feature that is in the package's own `[features] default` does not diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index 62e5d6c18..a787f1576 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -106,6 +106,7 @@ mcpp pack --mode self-contained # 别名:--mode bundle-all mcpp pack --target x86_64-linux-musl # 等价 --mode static mcpp pack --target aarch64-linux-musl # ARM64 等价写法 mcpp pack --format dir # 输出为目录,不打包 tarball +mcpp pack --format appimage # 由图里某个包提供的格式 mcpp pack -o myapp.tar.gz # 仅文件名:落到 target/dist/myapp.tar.gz mcpp pack -o /abs/path/myapp.tar.gz # 含目录:按字面路径输出 mcpp pack --profile dev # 换一个 profile 构建(默认 release) @@ -113,6 +114,42 @@ mcpp pack --no-strip # 按构建原样发货,不剥符号 mcpp pack --debug-symbols dbg/ # 把分离出的 *.debug 写到 dbg/ ``` +### `--format` 是一个轴,引擎只拥有其中两个取值 + +`tar` 与 `dir` 回答的问题,和 `msi` 与 `appimage` 回答的问题是同一个 —— 输出取什么 +形状 —— 所以它们是一个 flag 的取值,而不是第二个 flag 的开端。引擎持有什么、包持有 +什么,分界是: + +> **`mcpp pack` 拥有机制,以及那一种通用格式。其余每一种格式都住在包里,由 +> `mcpp pack` 分派过去。** + +那种通用格式就是它已经在产出的东西:一个解开就能跑的归档。它「通用」只在这里唯一 +要紧的那个意义上 —— 它不需要知道任何别人的发布。此外的一切都需要。dpkg 的 control +字段、AppImage 的 runtime、WiX 的 schema、Apple 的公证、Android 的签名方案:其中任何 +一个被绑进引擎,都会把一次 mcpp 的发布耦合到一次 mcpp 并不控制的发布上。这与本项目 +早已为语言做过的论证是同一个 —— Slang 被支持,而引擎里没有它的名字。 + +所以取值集合是开放的(mcpp 2026.9.11.1+)。`--format ` 在解析后的图里找到声明 +了 `` 的那个包,并把暂存树交给它;一个未知的取值会点名**当下确实可用**的那些, +而不是一份固定清单: + +``` +error: unknown --format 'bogus'. + available in this build: tar, dir, appimage + A format past `tar` and `dir` comes from a package in the resolved graph, which declares + it with `mcpp::provides_pack_format("")` in its build program. Add the package + that provides 'bogus' to [build-dependencies] and activate its feature. +``` + +这次拒绝发生在任何东西被编译之前。怎么写这样一个包,见 +[产出可分发物](30-build-mcpp.md#产出可分发物pack_format-与-stage_dir20269111); +引擎加的三样东西是:一棵 `artifact` action 可以消费的暂存树、`[package]` 的其余字段 +进入构建程序、以及这次分派本身。每一样都与格式无关 —— 而「与格式无关」正是判断某样 +东西该不该进引擎的判据。 + +被分派的格式作用于一个**程序** target。库包发的是一份接口加上每个三元组的预构建产物, +没有单独一棵暂存树,所以 `mcpp pack <库> --format ` 会被拒绝,而不是被忽略。 + `-o` 接受裸文件名时自动归到 `target/dist/`;含目录(相对或绝对) 时按字面路径输出。 @@ -355,6 +392,11 @@ force_bundle = ["libfoo.so"] # 即使命中 PEP 600 名单也强制打包 macOS **程序** bundling(Mach-O 依赖闭包,走 `otool -L` / `LC_LOAD_DYLIB`, 重定位走 `install_name_tool`)仍在规划中;在它落地之前,`mcpp pack <程序>` 会在该格式上拒绝,而不是产出一个只是看起来像 bundle 的东西。当前 `.zip` -之外的 Windows DLL 分发,以及 `.deb` / `.rpm` / AppImage 等格式,同样在规划中。本文档随 `mcpp pack` 实现演进,最新选项以 -`mcpp pack --help` 为准。 +之外的 Windows DLL 分发,同样在规划中。 + +`.deb`、`.rpm`、AppImage、`.msi` 这些分发格式**不在**这份清单上,而这是一个决定而不是 +一处遗漏:它们住在包里,经 `--format ` 到达用户,理由见上一节。`[pack]` 的内建 +模式不需要再添任何新成员。 + +本文档随 `mcpp pack` 实现演进,最新选项以 `mcpp pack --help` 为准。 diff --git a/docs/zh/21-the-target-triple.md b/docs/zh/21-the-target-triple.md index 9707ca39a..ef7dde21e 100644 --- a/docs/zh/21-the-target-triple.md +++ b/docs/zh/21-the-target-triple.md @@ -32,18 +32,24 @@ C 库。选中 `x86_64-linux-musl` 就是选中 musl-gcc 载荷,选中 | 段 | 内容 | 例 | |---|---|---| -| `arch` | 指令集 | `x86_64`、`aarch64`、`riscv64` | -| `os` | 操作系统,或 `none` | `linux`、`windows`、`macos`、`none` | -| `env` | 见下 —— 它在每个平台上是不同的轴 | `gnu`、`musl`、`msvc`、`elf` | +| `arch` | 指令集 | `x86_64`、`aarch64`、`riscv64`、`wasm32` | +| `os` | 操作系统,或 `none` | `linux`、`windows`、`macos`、`ios`、`emscripten`、`none` | +| `env` | 见下 —— 它在每个平台上是不同的轴 | `gnu`、`musl`、`msvc`、`android`、`elf` | 第三段值得留意,因为它在各处命名的并不是同一类东西: | 平台 | `env` 命名 | 取值 | |---|---|---| -| `linux` | **C 库** | `gnu`(glibc)、`musl` | +| `linux` | **C 库** | `gnu`(glibc)、`musl`、`android`(bionic) | | `windows` | **对象 ABI** | `gnu`(Itanium C++ ABI)、`msvc`(微软的) | | `none` | **对象格式** | `elf` | -| `macos` | 无;该平台不带这一段 | — | +| `macos`、`ios`、`emscripten` | 无;该平台不带这一段 | — | + +`android` 是一个 **C 库**,所以它落在 `musl` 落的那个位置上,OS 段仍是 `linux`。 +这个位置就是这处建模决定的全部:内核**就是** Linux,所以 ELF、`unix` family、 +`nasm -f elf64` 全都已经是对的;而一个 `os = "android"` 会让这三样默认全错,并 +且要求在每一处站点给出一个新答案。它与 `gnu` 的差别是 bionic、加载器路径和 SDK +—— 而这恰好就是 `env` 这一段存在的意义。 在 Windows 上这一段经常被读错,因为 `gnu` 这个词暗示了一个并不在场的 C 库。 对一份按构建期体系为 `x86_64-windows-gnu` 构建的产物实测: @@ -59,6 +65,27 @@ compiler-rt,C 库是 musl,C++ 运行时是 libc++,平台是 openkal。`gnu` 是 LLVM 词表里「非 MSVC 的那套 ABI」的标签,继承自 MinGW,而 clang 需要这个 拼写来选中正确的内部工具链。mcpp 改不了它。 +### 对象格式是一个轴,不是一处推导 + +一个三元组的二进制格式过去根本不是任何东西:它在每一处需要它的地方从 `os` 重新 +推导一遍。`is_pe()` 问 `os == "windows"`,产物命名再问一遍,打包器问第三遍。答案 +只有两个取值时,这是负担得起的。 + +`wasm32` 是 mcpp 词表里第一个格式不属于那两个的目标,而第三个取值会把那些推导变成 +**在每一处这样的站点上的一次添加** —— 而漏掉的那一处不会报错。它会静默地答 ELF, +因为 ELF 正是这棵树里每一个 `else` 分支所假设的东西。于是格式现在是一个答案: + +| 目标 | 格式 | +|---|---| +| `x86_64-linux-gnu`、`aarch64-linux-android`、`riscv64-none-elf` | ELF | +| `aarch64-macos`、`aarch64-ios` | Mach-O | +| `x86_64-windows-gnu`、`x86_64-windows-msvc` | PE | +| `wasm32-emscripten` | wasm | + +它与「有没有一个操作系统可供链接」**不是**同一个问题。一个裸机 RISC-V 映像是 ELF +且没有 OS;一个 wasm 模块有一层类 OS 的东西(Emscripten 的 POSIX 模拟)而不是 +ELF。把这两个轴并成一个,正是这处改动要消除的那个错误。 + ## 省略第三段 `-` 在每个平台上都是一个完整的目标: @@ -398,6 +425,10 @@ CRT;图供给时是 `musl`。一个目标字符串,两个不同的 C 库 —— | `thumbv8m.base-none-eabi` | preview | `llvm@22.1.8` | 载荷 | 载荷 | 载荷 | 载荷 | | `thumbv8m.main-none-eabi` | verified | `llvm@22.1.8` | 载荷 | 载荷 | 载荷 | 载荷 | | `thumbv8m.main-none-eabihf` | preview | `llvm@22.1.8` | 载荷 | 载荷 | 载荷 | 载荷 | +| `aarch64-linux-android` | planned | — | planned | planned | planned | planned | +| `x86_64-linux-android` | planned | — | planned | planned | planned | planned | +| `aarch64-ios` | planned | — | planned | planned | planned | planned | +| `wasm32-emscripten` | planned | — | planned | planned | planned | planned | `载荷` 这里有工具链载荷产出它 · `图` 没有载荷,但依赖可以供给系统 · `系统` 在机器上被找到,不是 mcpp 装的 · `SDK` 平台自己的 · diff --git a/docs/zh/30-build-mcpp.md b/docs/zh/30-build-mcpp.md index e4587085b..2552d352f 100644 --- a/docs/zh/30-build-mcpp.md +++ b/docs/zh/30-build-mcpp.md @@ -399,6 +399,11 @@ int main() { | `object` | 进**链接**集 | 链接边消费它们 | 资源编译器、`objcopy` 嵌 blob、生成的 `.def`、预编译 `.o` | | `artifact` | 一个新文件 | 它的**输入**是链接产物,所以在链接之后跑 | 签名、打包、size budget | +`artifact` 也是唯一允许写 `${mcpp.stage_dir}` 的 role —— 见下文 +[产出可分发物](#产出可分发物pack_format-与-stage_dir20269111)。另外三个跑在链接之前 +或与链接并行,没有任何已暂存的东西可读,所以 mcpp 会拒绝这个占位符,而不是把它展开成 +一个恰好存在的路径。 + 全程不涉及任何 phase 机制。`object` 与 `artifact` 由 ninja 自己的文件依赖定序 —— 这也是为什么 `artifact` 不会像朴素的「post 构建钩子」那样把自己重复施加一遍。 `source` 与 blocking 的 `check` 则由一条 order-only 边定序:从声明它的那个包的 @@ -470,6 +475,78 @@ mcpp 会播下一个带着该声明的占位文件,使 prepare 期的扫描与 内容一致 —— 与 `[modules].scan_overrides` 同一条「声明 + 验证」的取舍,build 期由 编译器自己的 P1689 输出复核。 +### 产出可分发物:`pack_format` 与 `stage_dir`(2026.9.11.1+) + +一个 `.msi`、一个 `.deb`、一个 AppImage、一个签过名的 `.app`,都不是那四个 role 的 +惯常活计,而它们全都是 `artifact`:每一个都消费**链接产物**,产出用户去安装的东西。 +引擎为它们加的是一套机制,而不是任何一种格式。 + +`mcpp pack --format ` 把 `` 交给解析后的图去解决,方式与 `--target` +够到一个引擎不必逐个认识的三元组相同。`tar` 与 `dir` 仍然是 `mcpp pack` 自己拥有的 +归档形状;此外的一切都来自某个包。 + +一个提供方有两半,而这两半不许被并成一半: + +```cpp +import mcpp; +#include +#include + +int main() { + // 第一半,无条件。 + mcpp::provides_pack_format("appimage"); + + // 第二半,有条件。 + if (std::string_view(mcpp::pack_format()) != "appimage") return 0; + + const std::string out = std::string(mcpp::out_dir()) + "/app.AppImage"; + mcpp::action a; + a.id = "appimage"; + a.role = "artifact"; + a.arg(tool).arg("${mcpp.stage_dir}").arg(out.c_str()) + .input("${mcpp.target_file:app}") + .output(out.c_str()) + .submit(); + return 0; +} +``` + +**无条件声明,有条件提交。** 声明是让引擎能回答一个发起请求的那次构建自己回答不了的 +问题:`--format bogus` 要点名**当下确实可用**的那些格式,`--help` 要说「解析后的图 +提供的任何格式」。两者读的都是一次「什么格式都没要」的 pass 收集到的集合。一个只在被 +问到时才声明的成员,对它的作者仍然照常工作 —— 作者永远传的是自己那个格式 —— 而对其他 +所有人,这个集合变成不可知的。对一个谁都没为之提交的格式,mcpp 会拒绝,而不是报告一次 +「什么包都没产出」的成功打包。 + +**`mcpp pack --format ` 会 prepare 两次。** 一条 `artifact` action 是一条 +ninja 边,而那棵暂存树是 mcpp 在链接**之后**产出的,所以这棵树不可能成为构建出它自己 +那一次 pass 的输入。第一次 pass 收集声明,并在任何东西被编译之前拒绝未知的格式;随后 +才是构建与暂存;第二次 pass 设上 `pack_format` 与 `pack_stage_dir`,并构建提供方提交 +的那条边。第二次 pass 里没有任何值是重新推导出来的 —— 三元组与暂存路径都是第一次 +pass 和那次暂存已经回答过的。 + +**暂存树是一个 bundle,不是一个根文件系统。** 它就是 `--mode vendored` 的含义: +`bin/`、`lib/`,可重定位、根在哪儿都行,并且已经过了 strip 策略、调试信息拆分与 +`include`/`exclude`。一个 AppImage、一个 `.app`、一个 `.msi` 要的就是它现在这个样子。 +而要一棵 FHS 树的格式(`.deb`、`.rpm`)自己负责重排布局,因为一个文件该落在哪个目录是 +那个格式的知识,不是引擎的。 + +`mcpp pack --format dir` 把同一棵树写到一个路径上就停下,这是人去查看一个成员将会拿到 +什么的方式。 + +**`mcpp pack` 报告的是这次请求**引入**的那些 artifact action。** 一条无论有没有人 +要格式都在场的 action —— 一个签名 stamp、一次 size budget —— 不是可分发物,点名它会 +是一个看起来像对的错答案。这个判据里没有任何一项是成员的性质:一个只打包一个具名程序、 +从不读暂存树的格式,与一个消费整个闭包的格式被同等识别。对一个谁都没为之提交的格式, +会被点名拒绝。 + +**写了 `${mcpp.stage_dir}` 的 action 会自动获得一条对这棵树的 manifest 的依赖。** +mcpp 会写出 `<暂存树>.stage-manifest` —— 一个兄弟文件,永不是成员,所以它不会跑进任何 +人的安装包里 —— 逐条列出每个已暂存文件的大小与相对路径。这条依赖由引擎添加,因为「用 +了」本身就意味着「依赖」:没有它,这条边只在链接产物变化时才变脏,而一个闭包多出了某个 +依赖的共享库、同时程序自己的字节并没有变的情况,会把上一次的可分发物原地留下,并报告为 +已是最新。 + 命令是 **argv 而不是 shell 字符串**(不假设存在 shell —— Windows 没有能依赖的那个), 插值只有封闭的一组: @@ -479,6 +556,7 @@ mcpp 会播下一个带着该声明的占位文件,使 prepare 期的扫描与 | `${mcpp.bin_dir}` | 产出的二进制所在目录 | | `${mcpp.compile_db}` | `compile_commands.json` 的路径(clang-tidy 的 `-p` 要的就是它) | | `${mcpp.target_file:}` | target `` 构建出的文件 | +| `${mcpp.stage_dir}` *(2026.9.11.1+)* | `mcpp pack` 暂存出的那棵树,绝对路径。仅 `artifact` role 可用,且仅在 `mcpp pack --format ` 下可用 | 上面的裸 stdout 协议仍是底层基底;`import mcpp;` 是其上的类型化层。 @@ -575,6 +653,13 @@ mcpp 会把它自己构建时用的**同一份** std 模块暂存过来,缓存 | `MCPP_LANGUAGE_MODULES` *(2026.9.7.1+)* | -- | 声明它的那个包设了 `[language] modules` 时为 `1`,否则 `0`。**生成**面向消费者声明的规则读它来在模块接口与头文件之间选择,项目因此只需说一次。旧引擎不设这个变量,规则把缺席读作 `0` —— 也就是这个变量存在之前每个消费者的行为 | | `MCPP_PKG_NAME` *(2026.9.7.1+)* | -- | 这个程序所构建的包的 `[package] name`。规则生成的每个名字都由它推导:消费者导入的模块、访问器所在的命名空间、生成头里的符号。在它存在之前,可用的最接近的答案是 `MCPP_MANIFEST_DIR` 的末段,那是目录名 —— 于是一个叫 `vulkan-saxpy` 的包放在名为 `app` 的目录下会生成 `app.shaders`,而工作区里每一个 `/app/` 都声称拥有同一个模块。旧引擎下缺席,规则把缺席读作「沿用先前的推导」 | | `MCPP_PKG_NAMESPACE` *(2026.9.7.1+)* | -- | `[package] namespace`。包未声明命名空间时为空。需要产出在索引范围内唯一的名字的规则用这一对而不是单用名字,因为包身份是 `(namespace, name)` | +| `MCPP_PKG_VERSION` *(2026.9.11.1+)* | `mcpp::package_version()` | `[package] version`。每一种安装包格式都要写版本号;在这个变量之前,项目只能把版本号在成员自己的 options 里再写一遍,而那份副本会与 `[package]` 漂移,且没有任何东西能发现 | +| `MCPP_PKG_DESCRIPTION` *(2026.9.11.1+)* | `mcpp::package_description()` | `[package] description`。包未声明时为空 | +| `MCPP_PKG_LICENSE` *(2026.9.11.1+)* | `mcpp::package_license()` | `[package] license` | +| `MCPP_PKG_AUTHORS` *(2026.9.11.1+)* | `mcpp::package_authors()` | `[package] authors`,以 `;` 连接。不用 `,`:一条 author 的惯例写法是 `Name `,名字里可能带逗号,以逗号连接的列表无法再切回原来的条目 | +| `MCPP_PKG_REPO` *(2026.9.11.1+)* | `mcpp::package_repo()` | `[package] repo` | +| `MCPP_PACK_FORMAT` *(2026.9.11.1+)* | `mcpp::pack_format()` | 本程序所处的这次 `mcpp pack` 的 `--format` 取值;任何普通构建下都为空。承载含义的正是这个空值 —— 成员据此为自己的提交加闸,于是 `mcpp build` 拿到的还是它一直以来的那张图 | +| `MCPP_PACK_STAGE_DIR` *(2026.9.11.1+)* | `mcpp::pack_stage_dir()` | `mcpp pack` 已经把闭包暂存到的位置,绝对路径;本次构建不在打包时为空。读它来判断这次要干的活是什么形状,而把 `${mcpp.stage_dir}` 写进 action —— 这样图里的路径与程序读到的路径不可能不一致 | | `MCPP_DEVICE_SOURCES` *(2026.9.5.2+)* | `mcpp::device_sources()` | 本包有效 `sources` 匹配到的设备类源文件(`.cu`、`.hip`…),相对包根,一行一个;没有时为空串。引擎一个都不编译 —— 由本程序引入的规则包把每一个变成一条 `mcpp::action`。已经过收窄:构建未覆盖的 `{ glob, accel }` 条目贡献为空,因此 `--no-accel` 得到空列表 | | `MCPP_OUT_DIR` | `mcpp::out_dir()` | mcpp 提供的可写输出/暂存目录 | | `MCPP_MANIFEST_DIR` | `mcpp::manifest_dir()` | 包根(= CWD) | @@ -668,7 +753,16 @@ shim,而可用的那份就在项目自己的环境里,根本不在 `PATH` 上。 什么都不会坏;只是这个名字声称了一个该包并不具有的来源。项目之外的规则自选前缀。 -**工具不是规则。** 规则说明一个编译单元如何被 mcpp 并不驱动的编译器编译:它提交一条 +**工具不是规则,而可分发物两者都不是。** 三类活计,成员的前缀说明它回答三个问题中的 +哪一个: + +| 前缀 | 回答的问题 | 编译编译单元 | 在构建程序里执行 | +|---|---|---|---| +| `rules-*` | 这个编译单元如何被编译 | 是 | 否 | +| `tools-*` | 构建程序自己需要做什么 | 否 | 是 | +| `dist-*` *(2026.9.11.1+)* | 链接之后出来的是什么,以及用户以什么形态安装它 | 否 | 否 | + +规则说明一个编译单元如何被 mcpp 并不驱动的编译器编译:它提交一条 action,由引擎调度。工具说明的是构建程序需要、而没有任何编译器执行的事,并在构建程序 运行时当场做掉。`mcpp.tools.embed`(feature `tools-embed`,mcpp 2026.9.5.4+)是第一个: 它把数据文件写成程序编译进去的头文件(字节数组或 32 位字数组),内容未变时不重写文件, diff --git a/docs/zh/31-authoring-a-rule-package.md b/docs/zh/31-authoring-a-rule-package.md index fabeaa331..5b0060b73 100644 --- a/docs/zh/31-authoring-a-rule-package.md +++ b/docs/zh/31-authoring-a-rule-package.md @@ -269,6 +269,50 @@ mcpp::floor("cuda.driver", runtime_needs); 一个 option 暴露出来,好让用别的键声明这条边的消费者能提供它;并且在找不到时以 一条点名「找的是什么」的消息拒绝,而不是拿一个空路径去执行命令。 +## 编写一个分发成员(mcpp 2026.9.11.1+) + +一个 `dist-*` 成员既不是规则也不是工具。它不编译编译单元,也不在构建程序运行期间把活 +干完:它消费**链接产物**,产出用户去安装的东西 —— 一个 `.msi`、一个 `.deb`、一个 +AppImage、一个签过名的 `.app`。机制是一条 `artifact` action 加上 +`mcpp pack --format `,见 +[30 —— 产出可分发物](30-build-mcpp.md#产出可分发物pack_format-与-stage_dir20269111)。 + +本章其余内容原样适用。有六条在这里绑得更紧,每一条都是这个类别会犯、而规则不会犯的 +错。 + +**无条件声明,有条件提交。** `provides_pack_format` 是引擎用来回答「这张图提供哪些 +格式」的东西,而回答发生在一次「什么格式都没要」的构建上。一个只在被问到时才声明的 +成员,对它的作者照常工作 —— 作者永远传自己那个格式 —— 而对其他所有人,这个集合变成 +不可知的。 + +**同时提供 plan 与 submit。** 可分发物是交到用户手上之前的最后一环,因此它是整个构建 +里最可能需要项目自己改一笔的部分:换一个压缩等级、多带一个文件、加第二个签名。 +`generate_all(opt)` 恰好等于 `submit(plan_all(opt))`,正是让这样一笔改动不至于变成把 +成员重新实现一遍的东西。 + +**点名输入,不要去 harvest 一个目录。** 一个解析成空的路径是无声的,而一个缺失的具名 +输入是错误。实测:一条 WiX action 绑定一个目录并 harvest 它,当那个路径在 Windows 上 +解析成空时,产出了一个**有效的、空的、52 KB 的安装包,并且没有任何诊断**。 +`${mcpp.target_file:}` 就是这个机制 —— 构建程序既不知道三元组也不知道指纹,而一个 +不存在的 target 名会被拒绝,而不是展开成一个空路径。 + +**在成功路径上,为自己的输出断言一条下界。** 只要成员能判断自己的结果是空的或不合理 +的,它就必须经 `mcpp::warning` 说出来,因为成功构建的 stderr 会被丢弃。这与上一段实测到 +的是同一个失败,只是被拦在了后一层。 + +**在工具会被查找的那个位置声明它。** `xpkg_dir` 从 `MCPP_XPKG_*_DIR` 回答,而这些是 +mcpp 为**正在被构建的那个包**设置的。一个依赖的声明会把载荷装上,却不会让它对消费者的 +构建程序可见,所以运行载荷工具的成员要自己声明它 —— 并且在查找返回空时把这件事说出来, +而不是指向一份读者并不拥有的 manifest。 + +**构建不许碰网络,而被包起来的工具可能会碰。** 在 `appimagetool` 1.9.1 上实测:除非用 +`--runtime-file` 指定一份本地副本,它每次被调用都会从一个 GitHub release 下载它的 +type-2 runtime 存根。包装这类工具的成员必须从自己声明的载荷里把这个文件供上。安装期 +是下载合法的时候;构建期不是,而一个会去取东西的构建既不可复现也不能离线用。 + +**一个 `(name, version)` 只指一份载荷。** 包装签名或打包工具的成员继承了那个工具的兼容 +面,所以与被包装的工具同步版本是正当的,而且陈述了一件真事。 + ## 当前边界 - 规则包自己 `[features] default` 里的规则 feature 不隐含 `host-module`。 diff --git a/mcpp.toml b/mcpp.toml index 2dfc8efd0..4b3194b2e 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.9.10.2" +version = "2026.9.11.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/buildmcpp/src/directives.cppm b/modules/buildmcpp/src/directives.cppm index d0557d39b..5dc9dc742 100644 --- a/modules/buildmcpp/src/directives.cppm +++ b/modules/buildmcpp/src/directives.cppm @@ -135,6 +135,21 @@ enum class Slot : std::size_t { // vendor knowledge in the package that has it and out of the engine. Facts, Floors, + // A DISTRIBUTION FORMAT THIS PACKAGE PROVIDES (`mcpp:pack-format=`). + // + // `mcpp pack --format ` resolves `` through the graph the same + // way `--target` reaches a triple: the engine holds the DISPATCH and no + // format. The value is a bare name and means nothing to this file, which is + // what keeps dpkg's control fields, WiX's schema and Apple's notarisation + // out of an engine whose release would otherwise be coupled to theirs. + // + // COLLECTED FROM A BUILD THAT ASKED FOR NOTHING, which is why it is a slot + // and not a side effect of the request. `mcpp pack --format bogus` names + // what is available and `--help` says "plus any format the resolved graph + // provides"; both read this set on a pass where `MCPP_PACK_FORMAT` is + // empty. See `mcpp::provides_pack_format` for the author-facing half of the + // same rule -- declare unconditionally, submit conditionally. + PackFormats, Count }; inline constexpr std::size_t kSlotCount = static_cast(Slot::Count); @@ -217,7 +232,7 @@ struct Def { int sinceProtocol; }; -inline constexpr std::array kTable{{ +inline constexpr std::array kTable{{ // wire tag slot scope transform must missingPrefix missingSuffix since {"cxxflag", "cxxflag", Slot::CxxFlags, Scope::PackagePrivate, Transform::Verbatim, false, "", "", 1}, {"cflag", "cflag", Slot::CFlags, Scope::PackagePrivate, Transform::Verbatim, false, "", "", 1}, @@ -336,6 +351,20 @@ inline constexpr std::array kTable{{ // Slot::Facts for the shape of each value. {"fact", "fact", Slot::Facts, Scope::Claim, Transform::Verbatim, false, "", "", 7}, {"floor", "floor", Slot::Floors, Scope::Claim, Transform::Verbatim, false, "", "", 7}, + // `tag` IS NON-EMPTY FOR THE REASON `warning`'S IS, AND IT MATTERS MORE + // HERE. A build program's result is cached and a hit does not re-run it, so + // a declaration that lived only on the run path would be present on the + // first build of a project and absent on every later one -- and the pass + // that reads it is `mcpp pack`, which is never the first build. The set + // would then be empty exactly when a user asks for a format, and the + // refusal would name nothing. + // + // kCacheEpoch is NOT bumped. An entry written before this row carries no + // `d pack-format` line, and the program that wrote it could not emit one, + // so replaying it yields what that program said. An older engine reading a + // newer entry already discards the whole record through the unknown-tag + // path. + {"pack-format", "pack-format", Slot::PackFormats, Scope::Claim, Transform::Verbatim, false, "", "", 9}, }}; // ── Collected output of one run ──────────────────────────────────────────── @@ -813,6 +842,13 @@ void apply(mcpp::manifest::Manifest& m, const Directives& d) { m.runtimeConfig.requirements.push_back(std::move(req)); } + // A NAME, CARRIED AND NOT INTERPRETED. The engine compares it against + // `--format` and hands the request to whoever claimed it; nothing here + // parses it, because a distribution format has less claim to a name in the + // engine than a language does, and Slang is already supported without one. + for (auto const& f : d.at(Slot::PackFormats)) + bc.packFormats.push_back(f); + // Build-graph nodes. Decoded here rather than at parse time so the cache // stores the payload verbatim and a replay is byte-identical to a run. for (auto const& payload : d.at(Slot::Actions)) { diff --git a/modules/buildmcpp/src/program_protocol.cppm b/modules/buildmcpp/src/program_protocol.cppm index 7301dcfa7..501374f12 100644 --- a/modules/buildmcpp/src/program_protocol.cppm +++ b/modules/buildmcpp/src/program_protocol.cppm @@ -66,7 +66,12 @@ export namespace mcpp::build::program_protocol { // `--exclude-libs`) had no way out. Same cost as v5's: a package calling // `mcpp::link_flag()` fails on an older engine at the build.mcpp COMPILE, // because that engine's bundled module has no such function. -inline constexpr int kProtocolVersion = 8; +// v9: adds `pack-format` -- the outlet by which a package says which +// distribution format it provides, so `mcpp pack --format ` can dispatch +// to it. Same cost as v5's: a package calling `mcpp::provides_pack_format()` +// fails on an older engine at the build.mcpp COMPILE, because that engine's +// bundled module has no such function. +inline constexpr int kProtocolVersion = 9; // ── Cache-format epoch ───────────────────────────────────────────────────── // diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index b8b3718fa..d39f7c792 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -385,6 +385,23 @@ struct BuildAction { // two are matched against each other. std::string packageName; Role role = Role::Source; + // Set by the engine, never by the build program: this action's command or + // inputs named `${mcpp.stage_dir}`. + // + // ITS ONE JOB IS THE IMPLICIT DEPENDENCY. An action that names the staged + // tree gains an edge to that tree's manifest, so it is dirty when the + // staged SET changes and not only when a link output does. The dependency + // is implied by the use, so a member author cannot forget it. + // + // IT IS NOT HOW `mcpp pack --format ` DECIDES WHICH ACTION IS THE + // DISTRIBUTABLE, and briefly was. Not every format consumes the closure: an + // `.msi` built from ONE NAMED PROGRAM takes `${mcpp.target_file:}` + // and never looks at the tree -- which is the shape the guidance + // recommends, after a bind path that resolved to nothing produced a valid, + // empty, 52 KB installer. So the member that followed the guidance was the + // member that check refused. The dispatch asks instead which artifact + // actions the REQUEST INTRODUCED; see mcpp.pack.pipeline. + bool consumesStageDir = false; std::vector inputs; // absolute or package-relative std::vector outputs; // ditto; declared, see INV-D // Object only: which link units receive the outputs. Empty = every LINKED @@ -693,6 +710,17 @@ struct BuildConfig : BuildInputs { // (`mcpp:action=`). Empty for every package that does not use one, so an // ordinary build is untouched. std::vector actions; + // Distribution formats this package's build program declared it provides + // (`mcpp:pack-format=`). Empty for every package that ships no such member, + // so an ordinary build is untouched. + // + // THE ENGINE HOLDS THE DISPATCH AND NOT THE FORMAT. `mcpp pack --format + // ` looks the name up in the union of these lists, exactly as + // `--target` reaches a triple the engine did not have to know + // individually. `.deb`'s control fields, WiX's schema and Apple's + // notarisation each couple a release to a release mcpp does not control, + // and a name here is the whole of what the engine learns. + std::vector packFormats; bool staticStdlib = true; // #336 — the C++ runtime DISTRIBUTION contract: what the artifact promises // about the machine that runs it ("self-contained" | "toolchain-coupled" | diff --git a/modules/toolchain-model/src/model.cppm b/modules/toolchain-model/src/model.cppm index 8229fa7db..448357758 100644 --- a/modules/toolchain-model/src/model.cppm +++ b/modules/toolchain-model/src/model.cppm @@ -497,10 +497,15 @@ std::vector graph_runtime_compile_flags(const Toolchain& tc) { // defect was invisible until a second architecture was built. if (t->arch == "aarch64") out.emplace_back("--rtlib=compiler-rt"); if (t->is_pe()) out.emplace_back("-fdwarf-exceptions"); - if (t->is_pe() || t->os == "macos") out.emplace_back("-femulated-tls"); + // OBJECT FORMAT, NOT OS: `is_mach_o()` covers iOS along with macOS, which + // `os == "macos"` used to miss. Both need the emulated-TLS model for the + // same reason PE does -- `_tlv_bootstrap` is loader-bootstrapped there + // exactly as `_tls_index` is on PE. + if (t->is_pe() || t->is_mach_o()) out.emplace_back("-femulated-tls"); // MACH-O ONLY, AND THE REASON IS THAT WEAK-DEF IS A RUN-TIME MECHANISM - // THERE. See the note on this function for the measurement. - if (t->os == "macos") { + // THERE. See the note on this function for the measurement. `is_mach_o()` + // rather than `os == "macos"`: the mechanism is ld64's, which iOS shares. + if (t->is_mach_o()) { out.emplace_back("-fvisibility=hidden"); out.emplace_back("-fvisibility-inlines-hidden"); } @@ -524,8 +529,11 @@ bool target_supports_full_static(std::string_view targetTriple, bool hostCapabil // false is what keeps the two mechanisms from both emitting the flag. if (t->is_pe()) return false; - // macOS cannot fully static-link: libSystem must stay dynamic. - if (t->os == "macos") return false; + // Mach-O cannot fully static-link: libSystem (macOS) / the Apple + // equivalent (iOS) must stay dynamic. `is_mach_o()`, paired with `is_pe()` + // above, so this reads as "for each object format" rather than leaving + // iOS to fall through to the `linux` line below by accident. + if (t->is_mach_o()) return false; // Linux ELF — glibc or musl, native or cross. This is the line that was // previously gated on the HOST being Linux. diff --git a/modules/toolchain-model/src/triple.cppm b/modules/toolchain-model/src/triple.cppm index 2a605a65b..795c629ce 100644 --- a/modules/toolchain-model/src/triple.cppm +++ b/modules/toolchain-model/src/triple.cppm @@ -29,6 +29,37 @@ import mcpp.platform; export namespace mcpp::toolchain::triple { +// WHAT A TARGET PRODUCES, AS ONE ANSWER RATHER THAN A DERIVATION AT EACH SITE. +// +// The binary format used not to be anything: it was re-derived from `os` +// wherever it was needed -- `is_pe()` asked `os == "windows"`, artifact naming +// asked again, the packer asked a third time -- and that is affordable only +// while the answer has two values. A THIRD produces an addition at every such +// site, and a site that was missed does not fail: it silently answers "ELF", +// because ELF is what every `else` branch in the tree assumes. +// +// That is why this exists before wasm needs it rather than after. `wasm32` is +// the first target in mcpp's vocabulary whose object format is neither of the +// two the tree was written around, and #597 is a target-model change for +// exactly this reason -- not because a table row is hard. +// +// IT IS NOT THE SAME QUESTION AS `is_freestanding()`, and merging them would be +// the mistake this replaces. "Which container do objects come in" and "is there +// an operating system to link against" are different axes: a bare-metal +// RISC-V image is ELF with no OS, and a wasm module has an OS-like layer +// (Emscripten's POSIX emulation) and is not ELF. +enum class ObjectFormat { Elf, MachO, Pe, Wasm }; + +std::string_view to_string(ObjectFormat f) { + switch (f) { + case ObjectFormat::Elf: return "ELF"; + case ObjectFormat::MachO: return "Mach-O"; + case ObjectFormat::Pe: return "PE"; + case ObjectFormat::Wasm: return "wasm"; + } + return "ELF"; +} + struct Triple { std::string arch; // "x86_64" | "aarch64" | "riscv64" | ... (GNU spelling) std::string os; // "linux" | "macos" | "windows" @@ -116,7 +147,29 @@ struct Triple { if (is_msvc_env()) return arch + "-pc-windows-msvc"; return arch + "-w64-windows-gnu"; } + // APPLE'S OTHER OS. Same `arm64` spelling and the same vendor segment; + // what differs is the SDK and the deployment-target flag. + // + // NO VERSION IS BAKED IN, unlike the macOS branch above, and that is a + // decision rather than an omission. `-miphoneos-version-min` belongs to + // the layer that also owns the SDK path and the `.app` bundle -- a + // distribution plugin -- and a default written here would be a second + // place that answers it. clang picks its own when nothing says. + if (os == "ios") { + const std::string a = (arch == "aarch64") ? "arm64" : arch; + return a + "-apple-ios"; + } + // ANDROID IS LINUX, AND THE ENV SEGMENT IS WHERE IT SAYS SO. clang also + // accepts an API level fused onto the OS segment + // (`aarch64-linux-android24`), which selects which bionic symbols are + // visible; it is omitted here for the reason the iOS version is -- + // the minimum platform version is the project's statement, and clang + // has a default. if (os == "linux") return arch + "-unknown-linux-" + (env.empty() ? "gnu" : env); + // Emscripten's own effective triple. The vendor segment is `unknown` + // and the OS segment is the platform layer rather than a kernel, which + // is why `object_format()` reads the ARCH for this row. + if (os == "emscripten") return arch + "-unknown-emscripten"; if (os == "none") return str(); // freestanding: already LLVM's form return str(); } @@ -124,7 +177,39 @@ struct Triple { bool is_musl() const { return env == "musl"; } bool is_msvc_env() const { return env == "msvc"; } bool is_windows_gnu() const { return os == "windows" && env == "gnu"; } - bool is_pe() const { return os == "windows"; } + + // THE SINGLE DERIVATION. Every question about the container objects come in + // is answered here and nowhere else -- see `ObjectFormat` for why a third + // value makes that a requirement rather than a preference. + // + // The arch test precedes the ELF fallback because a wasm target's OS + // segment names a platform layer (`emscripten`), not a format, and the + // fallback would otherwise claim ELF for it -- the silent wrong answer this + // whole axis exists to remove. + ObjectFormat object_format() const { + if (os == "windows") return ObjectFormat::Pe; + if (os == "macos" || os == "ios") return ObjectFormat::MachO; + if (arch.starts_with("wasm")) return ObjectFormat::Wasm; + return ObjectFormat::Elf; + } + + // Kept as its own name because it is what 30-odd sites already ask, and now + // reads the single answer rather than re-deriving one. + bool is_pe() const { return object_format() == ObjectFormat::Pe; } + bool is_mach_o() const { return object_format() == ObjectFormat::MachO; } + bool is_wasm() const { return object_format() == ObjectFormat::Wasm; } + + // APPLE, AS ONE QUESTION. `os == "macos"` was the whole of it while macOS + // was the only Apple row; iOS shares the object format, the linker, the + // `arm64` spelling and `codesign`, and differs in the SDK and the + // deployment-target flag. A site that means "Apple" and asks "macOS" gets + // iOS wrong in the direction that still links. + bool is_apple() const { return os == "macos" || os == "ios"; } + // Android is Linux with a different C library and a different loader path. + // `os` stays `linux` for that reason -- it is the kernel, and every + // Linux-shaped decision in the tree is right about it -- and the env + // segment carries what differs. + bool is_android() const { return env == "android"; } // Bare metal: there is no OS to link against. THE predicate every // freestanding decision keys off, spelled once here so no consumer @@ -154,9 +239,18 @@ struct Triple { bool pin_is_capability() const { return is_freestanding() || (is_pe() && is_musl()); } // cfg() `family` dimension: unix | windows. + // + // iOS and Android are unix for the reason macOS and Linux are: the + // predicate answers about the API surface a source can assume, and both are + // POSIX. Emscripten is unix on the same test rather than on a claim about + // wasm -- it supplies a POSIX emulation, and a source guarded by + // `cfg(unix)` compiles against it. A target with no OS still answers + // nothing, unchanged: `cfg(unix)` on bare metal would be false in a way no + // source could act on. std::string family() const { if (os == "windows") return "windows"; - if (os == "linux" || os == "macos") return "unix"; + if (os == "linux" || os == "macos" || os == "ios" + || os == "emscripten") return "unix"; return {}; } @@ -413,6 +507,86 @@ inline constexpr TargetInfo kKnownTargets[] = { // library for these targets arrives from the dependency graph. { "armv7a-none-eabi", "verified", "bare","llvm@22.1.8","", true }, { "armv7a-none-eabihf", "verified", "bare","llvm@22.1.8","", true }, + + // ── The three platforms a package cannot add ──────────────────────────── + // + // A package can add a language, a tool, an action, a payload and a + // generated module. IT CANNOT ADD A TRIPLE: identity is these three + // strings and this table is compiled into the binary, so every layer above + // -- the `.apk` step, the `.app` step, the `.html`+`.wasm` step, the + // runner, the signing -- waits on a row here and on nothing else in the + // engine. Registering the rows is what turns each of those into a plugin + // that can be written rather than a plugin that has nowhere to attach. + // + // ALL FOUR ARE `planned`, WHICH IS A REFUSAL AND NOT A GAP. The tier gate + // refuses a planned row with `tier-planned` naming the row, so + // `mcpp build --target aarch64-linux-android` says the vocabulary has this + // target and nothing is wired yet -- rather than `unknown target`, which + // was false, or a build that resolves and produces nothing, which would be + // worse than either. What each row still needs is recorded in + // .agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md + // section 3, and it is a payload in every case, never engine work. + // + // THE PREREQUISITE NOBODY LISTS IS ANSWERED FOR TWO OF THE THREE. mcpp is + // module-first, so a row whose toolchain cannot compile a module interface + // unit would be worse than its absence. Measured 2026-09-11: `import std` + // works on both the NDK's clang 18 and Emscripten's, and neither needs a + // fork or a compiler upgrade -- what both need is the generated module + // surface their vendor chose not to install (133 files, 620 KB, taken from + // the libc++ revision matching `_LIBCPP_VERSION`, which for Emscripten is + // NOT the version its clang reports). Apple's half is not measurable on a + // Linux host and is the one genuinely open question of the three. + + // ANDROID IS THE SMALLEST OF THE THREE, and the ranking is the opposite of + // the demand ranking. `aarch64` is already an arch, ELF is already the + // object format, and Linux is already the OS: what was missing is an `env` + // value and a sysroot that points at an NDK. No `pin`, because no cross + // payload exists yet -- `xim:android-ndk` is the row's whole remaining + // cost, and until it lands `[target.].sysroot` is the escape hatch + // for a machine that has an NDK already. + { "aarch64-linux-android", "planned", "", "", "", false }, + // The emulator's row. Not a convenience: x86_64 is what an Android + // emulator image runs, so a row for the device without one for the + // emulator describes a target nothing in CI can execute. + { "x86_64-linux-android", "planned", "", "", "", false }, + + // iOS IS NEXT. `aarch64-macos` is `verified`, so Mach-O, `arm64`, the + // linker and the Apple half of the toolchain model all exist; what is + // missing is an `os` value and the iPhoneOS SDK. + // + // THE SDK IS A LICENCE QUESTION AND NOT A PACKAGING ONE, which is why this + // row carries no `sysroot`. The NDK is Apache-2.0 and Emscripten is MIT, + // both redistributable; the iPhoneOS SDK is neither. The recipe should + // reach for the lowest of three tiers its licence allows -- redistribute, + // fetch from upstream without a mirror, or locate what the machine already + // has -- and say which tier it took, because a consumer reading "locator" + // needs to know that is a licence conclusion rather than an unfinished + // recipe. `msvc@system` is the shape of the third tier and mcpp already + // has it. + // + // The simulator is deliberately not a row. It has its own SDK and produces + // its own object, so folding it in would make two targets share an + // identity -- the mistake `x86_64-windows-musl` was added to undo. + { "aarch64-ios", "planned", "", "", "", false }, + + // WEB IS THE OUTLIER, AND IT IS THE ONLY ONE OF THE THREE THAT CHANGES THE + // MODEL RATHER THAN EXTENDING A TABLE. A new arch (`wasm32`), a new os + // (`emscripten`), and -- the sharp part -- a new OBJECT FORMAT, which + // before `ObjectFormat` existed was not a field at all but a derivation + // repeated at every site that needed it. That is why this is + // https://github.com/mcpp-community/mcpp/issues/597 and not a table row. + // + // IT IS NOW ONLY THAT. The standard-library half is answered: measured + // 2026-09-11, `em++` compiles and links `import std` with NO additional + // flags once the module surface from llvm 20.1.7 is present -- the release + // matching Emscripten's `_LIBCPP_VERSION` of 200100, not the 22.0.0git its + // clang reports -- and `node app.js` printed the expected output. So #597 + // is one problem rather than two. + // + // `defaultStatic` is true because wasm has no dynamic loader in the sense + // the other rows mean: an Emscripten link produces one module plus its + // JavaScript, and there is no shared object for a search path to find. + { "wasm32-emscripten", "planned", "wasm","", "", true }, }; inline std::span known_targets() { return kKnownTargets; } @@ -762,6 +936,19 @@ std::optional parse(std::string_view s) { // "mingw32" is the GNU os segment for ALL MinGW targets (64-bit // included — historical residue); it means windows + gnu env. if (starts_with(k, "mingw")) { t.os = "windows"; sawOs = true; t.env = "gnu"; t.envExplicit = true; continue; } + // APPLE'S SECOND OS. `starts_with` for the same reason the macOS + // branch above uses it: an effective triple carries the deployment + // target on this segment (`arm64-apple-ios17.0`). The simulator is a + // different row and is deliberately not spelled here -- it has a + // different SDK and a different object, so folding it into this one + // would make two targets share an identity. + if (starts_with(k, "iphoneos") || starts_with(k, "ios")) + { t.os = "ios"; sawOs = true; t.env.clear(); continue; } + // EMSCRIPTEN IS AN OS SEGMENT, NOT AN ENV. It names the platform layer + // a wasm module is compiled against -- its POSIX emulation, its + // filesystem shim, its `main` loop -- which is the same kind of thing + // `linux` names and not the same kind of thing `musl` names. + if (starts_with(k, "emscripten")) { t.os = "emscripten"; sawOs = true; t.env.clear(); continue; } // Bare-metal object-format / ABI segments. Only meaningful with // os=none: `riscv64-none-elf`, `arm-none-eabi`, `arm-none-eabihf`. @@ -773,20 +960,38 @@ std::optional parse(std::string_view s) { } if (t.os != "macos") { + // ANDROID IS AN ENV SEGMENT ON A LINUX OS, and that placement is + // the whole of the modelling decision. The kernel IS Linux, so + // every Linux-shaped answer in the tree -- ELF, the `unix` family, + // `nasm -f elf64` -- is already right; what differs is the C + // library (bionic), the loader path and the SDK. An `os = "android"` + // would have made all three of those wrong by default and required + // a new answer at each site. + // + // `androideabi` is the 32-bit ARM spelling and resolves to the same + // env: the EABI half is the ARM calling convention, which `armv7a` + // already carries in the arch segment. + if (k == "android" || starts_with(k, "androideabi")) { + t.env = "android"; t.envExplicit = true; continue; + } if (k == "musl" || starts_with(k, "musleabi")) { t.env = "musl"; t.envExplicit = true; continue; } if (k == "gnu" || starts_with(k, "gnueabi")) { t.env = "gnu"; t.envExplicit = true; continue; } // starts_with: clang effective triples can carry a version suffix // on the env segment ("…-windows-msvc19.44.35211"). if (starts_with(k, "msvc")) { t.env = "msvc"; t.envExplicit = true; continue; } } - // Unrecognized segment (androideabi, wasi, …): not in mcpp's target - // language — treat as unparseable rather than guessing. + // Unrecognized segment (wasi, …): not in mcpp's target language — + // treat as unparseable rather than guessing. return std::nullopt; } if (!sawOs) return std::nullopt; - // macOS carries no env segment at all, so nothing was declined there. - if (t.os == "macos") { t.env.clear(); t.envExplicit = false; } + // macOS carries no env segment at all, so nothing was declined there. iOS + // and Emscripten are the same shape: the platform layer is the whole of the + // identity past the arch, and there is no C-library axis to decline. + if (t.os == "macos" || t.os == "ios" || t.os == "emscripten") { + t.env.clear(); t.envExplicit = false; + } // THE FILL STAYS, AND THE FACT THAT IT WAS A FILL IS NOW RECORDED. // `x86_64-linux` is the canonical identity `x86_64-linux-gnu` — every // directory name and cache key downstream depends on that — but it is NOT diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index 6a73bf867..677aa8234 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.9.10.2"; +inline constexpr std::string_view MCPP_VERSION = "2026.9.11.1"; } // namespace mcpp diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index bd30891a7..b2f89602f 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -130,6 +130,47 @@ struct BuildProgramEnv { // way to ask. See hostprogram::package_name for what it replaced. std::string packageName; std::string packageNamespace; + // THE REST OF `[package]`, FOR THE MEMBER OF THE COLLECTION THAT NEEDS IT. + // + // A rule generates a declaration and needs the package's NAME. A member + // that produces a DISTRIBUTABLE needs more: every installer format carries + // a version, and most carry a description, a licence and a maintainer. + // Without these a project has to restate them in the member's options, + // where they can drift from `[package]` with nothing able to detect it -- + // the second copy of a value whose first copy mcpp has already parsed. + // + // `packageAuthors` is joined with ';' rather than ',' because an author + // entry is conventionally `Name ` and a name may carry a comma. + // Empty under an engine that predates these, which a member reads as "fall + // back to whatever you did before". + std::string packageVersion; + std::string packageDescription; + std::string packageLicense; + std::string packageAuthors; + std::string packageRepo; + // ── The packaging pass this build is part of (mcpp 2026.9.11.1+) ──────── + // + // Empty for every ordinary build, and that is the value that carries the + // meaning: a member which produces a distributable SUBMITS NOTHING unless + // the format it provides was asked for. `mcpp build` therefore has the + // graph it always had, and the dist edge exists only in the pass that + // wants it. + // + // The value is the `--format` argument verbatim -- `tar`, `dir`, or a name + // a package provides. It rides the same env vector as everything else here, + // so `contract_hash` folds it into the build program's re-run key: the + // second pass re-runs exactly the programs whose answer this changes. + std::string packFormat; + // Where `mcpp pack` has ALREADY STAGED the closure, absolute. Non-empty + // only in the second pass, and only then because a staged tree is produced + // by mcpp after the link -- so a graph generated before the link cannot + // name a directory that does not exist yet. + // + // This is the value `${mcpp.stage_dir}` expands to. A member reads it to + // decide the shape of the work (which of `bin/`, `lib/`, `share/` the tree + // actually has) and writes the placeholder into the action, so the two + // never disagree. + std::filesystem::path packStageDir; // Whether this package builds C++ modules (`[language] modules`). // // Reported because a rule package that GENERATES a consumer-facing @@ -530,6 +571,13 @@ contract_env(const fs::path& root, const fs::path& outDir, const BuildProgramEnv e.emplace_back("MCPP_MANIFEST_DIR", root.string()); e.emplace_back("MCPP_PKG_NAME", env.packageName); e.emplace_back("MCPP_PKG_NAMESPACE", env.packageNamespace); + e.emplace_back("MCPP_PKG_VERSION", env.packageVersion); + e.emplace_back("MCPP_PKG_DESCRIPTION", env.packageDescription); + e.emplace_back("MCPP_PKG_LICENSE", env.packageLicense); + e.emplace_back("MCPP_PKG_AUTHORS", env.packageAuthors); + e.emplace_back("MCPP_PKG_REPO", env.packageRepo); + e.emplace_back("MCPP_PACK_FORMAT", env.packFormat); + e.emplace_back("MCPP_PACK_STAGE_DIR", env.packStageDir.string()); std::string csv; for (auto const& f : env.features) { if (!csv.empty()) csv += ','; diff --git a/src/build/distribution.cppm b/src/build/distribution.cppm index 52ca0d694..aef02adcf 100644 --- a/src/build/distribution.cppm +++ b/src/build/distribution.cppm @@ -129,9 +129,22 @@ enum class Format { Elf, MachO, Pe }; Format format_for(std::string_view targetTriple, Format hostFallback) { if (auto parsed = mcpp::toolchain::triple::parse(targetTriple)) { if (parsed->is_pe()) return Format::Pe; - if (parsed->os == "macos") return Format::MachO; + // `is_mach_o()`, not `os == "macos"`: the latter answered the + // opposite-hosts defect above for macOS and would still get iOS + // wrong the same way, since iOS's `os` is `ios`. + if (parsed->is_mach_o()) return Format::MachO; if (parsed->os == "linux" || parsed->os == "none") return Format::Elf; + // THIS `Format` HAS NO FOURTH MEMBER YET. A wasm32-emscripten + // triple parses here and falls out of every branch above (`is_pe()` + // and `is_mach_o()` are both false, and its `os` is `emscripten`, + // neither `linux` nor `none`) to the substring fallback below, which + // also does not name it, and then to `hostFallback` -- so today this + // function still answers the machine's own format for wasm rather + // than the target's, the same defect class its own header measured + // for macOS. Adding `Format::Wasm` is deferred to whoever gives this + // module a Mach-O-shaped mechanism for it (see `resolve`'s `switch`), + // not attempted here. } if (targetTriple.find("windows") != std::string_view::npos || targetTriple.find("mingw") != std::string_view::npos) diff --git a/src/build/flags.cppm b/src/build/flags.cppm index 7c2a64c18..c2adf9181 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -691,6 +691,40 @@ CompileFlags compute_flags(const BuildPlan& plan) { const auto linkIntentFlavor = [&] { if (isMingwTc) return LinkIntentFlavor::PeGnu; if (isMsvcDialect) return LinkIntentFlavor::PeMsvc; + // THE OBJECT FORMAT IS ASKED OF THE PARSED TRIPLE, AND THE SUBSTRING + // TEST BELOW IS NOW ONLY THE ESCAPE HATCH. + // + // `plan.toolchain.targetTriple` is mcpp's CANONICAL spelling, and + // `aarch64-macos` contains neither "apple" nor "darwin" -- so an + // explicit `--target aarch64-macos`, a verified row, fell through to + // `Elf`. Only a NATIVE macOS build was right, and by a different + // branch: an empty triple reaching the `needs_explicit_libcxx` rescue + // below. That is why nothing caught it -- the two paths through this + // function disagreed and only one of them was exercised. + // + // The substring test is kept for a triple `parse` REJECTS, which is + // the `[target.]` escape hatch: an author may name a spelling + // outside the canonical vocabulary, and it is then an LLVM-shaped + // string where "apple" and "windows" do appear. + if (auto t = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple)) { + switch (t->object_format()) { + case mcpp::toolchain::triple::ObjectFormat::MachO: + return LinkIntentFlavor::MachO; + case mcpp::toolchain::triple::ObjectFormat::Pe: + return LinkIntentFlavor::PeGnu; + case mcpp::toolchain::triple::ObjectFormat::Wasm: + // No `LinkIntentFlavor::Wasm` exists, and inventing one + // here would be a link-contract decision rather than a + // format question -- what `link_lib` and a search path + // even mean for an Emscripten link is the open half of + // #597. `Elf` is the wrong answer and is the one this + // returns; it is named here so the gap is visible rather + // than reached by falling off the end of a switch. + return LinkIntentFlavor::Elf; + case mcpp::toolchain::triple::ObjectFormat::Elf: + return LinkIntentFlavor::Elf; + } + } auto triple = plan.toolchain.targetTriple; std::ranges::transform(triple, triple.begin(), [](unsigned char c) { return std::tolower(c); }); diff --git a/src/build/graph_shape.cppm b/src/build/graph_shape.cppm index f8581c8e7..679d2d043 100644 --- a/src/build/graph_shape.cppm +++ b/src/build/graph_shape.cppm @@ -62,11 +62,22 @@ std::string_view to_string(GraphShape shape) { // `mcpp build --no-accel`, `mcpp build` -- the third reported "Finished in // 0.00s" and `mcpp run` executed the CPU variant. A graph an override chose // says so, and the fast paths, which run only without overrides, decline it. +// `packFormat` records the DISTRIBUTION FORMAT this graph was generated for, +// and empty means "none" -- an ordinary build. It rides this line for exactly +// the reason the other two fields do, and it is the third instance of one +// failure: `mcpp pack --format appimage` makes a build program submit an +// artifact action a plain build must not have, and the two graphs land in the +// same directory because the format is deliberately NOT in the fingerprint +// (putting it there would cost a full recompile to package an already-built +// tree). So `pack --format X` then `build` would replay a graph carrying a dist +// edge, which is the `A then B then A` shape both other fields exist to stop. std::string header_line(GraphShape shape, std::string_view scheduleTag, - bool accelOverridden = false) { - return std::format("# mcpp:graph={};schedule={};accel={}", + bool accelOverridden = false, + std::string_view packFormat = {}) { + return std::format("# mcpp:graph={};schedule={};accel={};dist={}", to_string(shape), scheduleTag, - accelOverridden ? "override" : "default"); + accelOverridden ? "override" : "default", + packFormat.empty() ? std::string_view("none") : packFormat); } // Read the shape back. `nullopt` means "this file does not say" — a build.ninja @@ -162,13 +173,39 @@ std::string read_accel_selection(const std::filesystem::path& ninjaPath) { return {}; } -// A graph the fast paths may replay: the package's own targets, and the -// device variant the manifest names rather than one a flag chose. Both fast -// paths run only when no override is present, so a graph an override wrote is -// never the graph a plain build would produce. +// The distribution format this graph was generated for, or "none". Empty when +// the file predates the field, which callers treat as a miss for the reason +// read_shape gives. +std::string read_pack_format(const std::filesystem::path& ninjaPath) { + std::ifstream input(ninjaPath); + if (!input) return {}; + std::string line; + for (int i = 0; i < 8 && std::getline(input, line); ++i) { + constexpr std::string_view prefix = "# mcpp:graph="; + if (!line.starts_with(prefix)) continue; + auto value = std::string_view(line).substr(prefix.size()); + while (!value.empty() && (value.back() == '\r' || value.back() == ' ')) + value.remove_suffix(1); + constexpr std::string_view key = ";dist="; + const auto at = value.find(key); + if (at == std::string_view::npos) return {}; + auto rest = value.substr(at + key.size()); + if (const auto semi = rest.find(';'); semi != std::string_view::npos) + rest = rest.substr(0, semi); + return std::string(rest); + } + return {}; +} + +// A graph the fast paths may replay: the package's own targets, the device +// variant the manifest names rather than one a flag chose, and no distribution +// edge. All three fast-path callers run only for a plain build, so a graph any +// of the three axes was pointed at is never the graph a plain build would +// produce. bool is_plain_build_graph(const std::filesystem::path& ninjaPath) { return read_shape(ninjaPath) == GraphShape::Normal - && read_accel_selection(ninjaPath) == "default"; + && read_accel_selection(ninjaPath) == "default" + && read_pack_format(ninjaPath) == "none"; } } // namespace mcpp::build diff --git a/src/build/hostprogram.cppm b/src/build/hostprogram.cppm index 88e21b8d6..fed7c862b 100644 --- a/src/build/hostprogram.cppm +++ b/src/build/hostprogram.cppm @@ -124,6 +124,33 @@ inline void fact(const char* name, const char* version) { } inline void floor(const char* spec) { std::printf("mcpp:floor=%s\n", spec); } +// ── The distributable channel (mcpp 2026.9.11.1+) ─────────────────────────── +// +// `mcpp pack --format ` dispatches to whichever package provides +// ``, exactly as `--target` reaches a triple the engine did not have to +// know individually. This is how a package says which name it answers for: +// +// mcpp::provides_pack_format("appimage"); +// +// DECLARE UNCONDITIONALLY, SUBMIT CONDITIONALLY. The declaration must not be +// gated on `pack_format()`, and the reason is that the engine has to be able to +// answer a question the requesting build cannot: `mcpp pack --format bogus` +// names what IS available, and `--help` says "plus any format the resolved +// graph provides". Both read the set collected from this outlet, on a build +// that asked for nothing. A member that declared only when asked would still +// work for its author -- they always pass their own format -- and would make +// the set unknowable for everyone else. +// +// The work itself is the other half: +// +// if (std::string_view(mcpp::pack_format()) == "appimage") { ... submit ... } +// +// Two names are reserved for the engine's own archive shapes and are refused +// here: `tar` and `dir`. +inline void provides_pack_format(const char* name) { + std::printf("mcpp:pack-format=%s\n", name); +} + // The memory layout for a freestanding link. Reaches the CONSUMER's link line // (like link_lib/link_search, unlike include_dir), because the package that // knows a board's layout is not the package being built. @@ -390,6 +417,53 @@ inline const char* manifest_dir() { return env_or("MCPP_MANIFEST // rule package working unchanged. inline const char* package_name() { return env_or("MCPP_PKG_NAME"); } inline const char* package_namespace() { return env_or("MCPP_PKG_NAMESPACE"); } + +// THE REST OF `[package]`, BECAUSE A DISTRIBUTABLE CARRIES IT. +// +// `package_name()` above exists so a generated declaration can be named. These +// exist for the other member of the collection: every installer format states a +// version, and most state a description, a licence and a maintainer. A member +// without them has to ask the PROJECT to restate values mcpp has already +// parsed, in the member's own options, where the copy drifts from `[package]` +// and nothing can detect that it has. +// +// `package_authors()` is a ';'-separated list -- not ',', because an author is +// conventionally `Name ` and a name may carry a comma. +// +// Empty under an engine older than 2026.9.11.1. A member that needs one must +// say so itself when it is empty, naming the value it wanted: only the member +// knows whether the absence is fatal. +inline const char* package_version() { return env_or("MCPP_PKG_VERSION"); } +inline const char* package_description() { return env_or("MCPP_PKG_DESCRIPTION"); } +inline const char* package_license() { return env_or("MCPP_PKG_LICENSE"); } +inline const char* package_authors() { return env_or("MCPP_PKG_AUTHORS"); } +inline const char* package_repo() { return env_or("MCPP_PKG_REPO"); } + +// WHICH DISTRIBUTABLE THIS PASS WAS ASKED FOR, or "" for every ordinary build. +// +// The empty value is the one that carries the meaning: a member gates its +// submission on this, so `mcpp build` has the graph it always had and a dist +// edge exists only in the pass that wants one. See `provides_pack_format` for +// the half that must NOT be gated. +inline const char* pack_format() { return env_or("MCPP_PACK_FORMAT"); } + +// WHERE `mcpp pack` HAS ALREADY STAGED THE CLOSURE, absolute; "" when this +// build is not packing. +// +// The tree is what `mcpp pack` computes and then, until this existed, threw +// away: the dependency closure after the strip policy, the debug-symbol split +// and `include`/`exclude`. It is a BUNDLE tree -- `bin/`, `lib/`, relocatable, +// rooted anywhere -- which is what an AppImage, a `.app` and an `.msi` want as +// it stands. A format that wants a root filesystem instead (`.deb`, `.rpm`) +// owns the re-layout, because which directory a file belongs in is that +// format's knowledge and not the engine's. +// +// READ IT HERE TO DECIDE, WRITE `${mcpp.stage_dir}` INTO THE ACTION. The +// directory exists while this program runs, so a member enumerates it to learn +// which of `bin/`, `lib/`, `share/` the tree actually has; the action's command +// then names it through the placeholder, so the path in the graph and the path +// this program read cannot disagree. +inline const char* pack_stage_dir() { return env_or("MCPP_PACK_STAGE_DIR"); } inline bool has_feature(const char* name) { char buf[256] = "MCPP_FEATURE_"; unsigned long o = 13; diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 4ff138ce9..cdebeeef0 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -260,15 +260,18 @@ std::string pe_link_flag(const BuildPlan& plan, bool sep, std::string shared_soname_flag(const LinkUnit& lu, const BuildPlan& plan) { if (lu.kind != LinkUnit::SharedLibrary) return ""; const auto t = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple); - const std::string os = t ? t->os - : (mcpp::platform::is_macos ? "macos" - : mcpp::platform::is_windows ? "windows" : "linux"); + // WHICH FLAG SPELLING, ASKED OF THE OBJECT FORMAT DIRECTLY rather than of + // an `os` string: the prior `os == "macos"` fell through to the ELF + // branch (`-Wl,-soname`, a GNU ld/BFD flag ld64 does not accept) for + // iOS, whose `os` is `ios`. + const bool pe = t ? t->is_pe() : bool(mcpp::platform::is_windows); + const bool macho = t ? t->is_mach_o() : bool(mcpp::platform::is_macos); // PE records no such name: a DLL is found by the filename in the importing // module's import table, and there is nothing to override. - if (os == "windows") return ""; + if (pe) return ""; const std::string name = lu.soname.empty() ? lu.output.filename().string() : lu.soname; - if (os == "macos") return "-Wl,-install_name,@rpath/" + name; + if (macho) return "-Wl,-install_name,@rpath/" + name; return lu.soname.empty() ? "" : "-Wl,-soname," + lu.soname; } @@ -291,7 +294,11 @@ std::string shared_soname_flag(const LinkUnit& lu, const BuildPlan& plan) { // script's syntax is not what the author wrote. std::string exports_file_contents(const LinkUnit& lu, std::string_view os) { std::string out; - if (os == "macos") { + // Which SYMBOL-TABLE SYNTAX, which is a property of the object format: + // iOS links with the same ld64 and the same leading-underscore Mach-O + // symbol table as macOS, so it takes this branch too rather than the + // GNU version-script one below, which ld64 does not parse. + if (os == "macos" || os == "ios") { // One symbol per line. Mach-O symbols carry a leading underscore that // the C++ source never writes, so it is added here -- the author names // the symbol, not the object format's spelling of it. @@ -316,7 +323,10 @@ std::string exports_flag(const LinkUnit& lu, std::string_view os, const std::filesystem::path& file) { if (lu.kind != LinkUnit::SharedLibrary || lu.exportPatterns.empty()) return ""; if (os == "windows") return ""; - if (os == "macos") + // iOS alongside macOS, matching `exports_file_contents`: same linker, + // same flag. Leaving it out sent an iOS shared-library link a GNU + // `--version-script` for a Mach-O symbol list, which ld64 rejects. + if (os == "macos" || os == "ios") return "-Wl,-exported_symbols_list," + file.generic_string(); return "-Wl,--version-script=" + file.generic_string(); } @@ -626,7 +636,8 @@ std::string emit_ninja_string(const BuildPlan& plan) { // write this one file and the fast path has to know what it is about to // replay. Must stay within the first few lines — see read_shape. append(mcpp::build::header_line(plan.graphShape, plan.scheduleTag, - plan.accelOverridden) + "\n"); + plan.accelOverridden, + plan.packFormat) + "\n"); append("ninja_required_version = 1.11\n\n"); // All compile/link flags are computed once via flags.cppm. diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 34433a1f8..0fe1db43d 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -270,6 +270,20 @@ struct BuildPlan { // absolute and engine variables already substituted by the time they get // here, so the backend only has to spell edges. std::vector actions; + // The distribution formats the RESOLVED GRAPH provides, sorted and unique + // (`mcpp:pack-format=`). This is what `mcpp pack --format ` resolves + // against and what an unknown value's refusal names. + // + // Collected on EVERY prepare, including the pass that asked for no format + // at all -- which is the pass that has to answer "what is available". See + // `mcpp::provides_pack_format` for the author-facing rule that makes this + // possible: declare unconditionally, submit conditionally. + std::vector providedPackFormats; + // Non-empty when this prepare is the second pass of `mcpp pack --format + // `: the staged closure `${mcpp.stage_dir}` expanded to. Recorded on + // the plan so the graph header line can say which format wrote this graph, + // and the fast paths can decline to replay it for a plain build. + std::string packFormat; std::vector runtimeLibraryDirs; // ONLY the dependency packages' [runtime] library_dirs (not toolchain/ // payload dirs). These are the dirs that must be baked into the produced @@ -541,8 +555,12 @@ std::vector shared_library_link_flags( const mcpp::toolchain::triple::Triple& target) { std::vector flags; const bool pe = n.sharedNeedsImportLib; + // WHICH RPATH SYNTAX, ASKED OF THE OBJECT FORMAT. `target.os == "macos"` + // used to answer this and missed iOS, which links with the same ld64 and + // wants the same `@loader_path` -- `os == "ios"` would otherwise take the + // ELF branch below and hand `$ORIGIN` to a linker that has no such token. const bool macho = target.empty() ? bool(mcpp::platform::is_macos) - : target.os == "macos"; + : target.is_mach_o(); if (pe) { flags.push_back(import_library_for(t, n).generic_string()); } else { @@ -796,9 +814,16 @@ std::vector runtime_search_closure( auto t = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple); return t ? *t : mcpp::toolchain::triple::Triple{}; }(); + // `!= "macos" && != "windows"` READ AS "ELF" BY EXCLUSION, WHICH IS THE + // ONE ANSWER THAT MUST NEVER BE REACHED BY EXCLUDING EVERYTHING ELSE: + // iOS and wasm32-emscripten both satisfy that double negative (their `os` + // is `ios` / `emscripten`, neither string), so this line called an iOS + // Mach-O and a wasm module ELF and would have handed both a DT_RPATH + // mechanism neither format has. Asked of `object_format()` directly, the + // single derivation, instead. const bool elfTarget = triple.empty() ? bool(mcpp::platform::is_linux) - : (triple.os != "macos" && triple.os != "windows"); + : (triple.object_format() == mcpp::toolchain::triple::ObjectFormat::Elf); // THE ARTIFACT'S OWN DIRECTORY — `$ORIGIN` (#415). // @@ -1045,10 +1070,13 @@ make_plan(const mcpp::manifest::Manifest& manifest, // The loader-tag contract exists only where DT_RPATH/DT_RUNPATH do. // Mach-O and PE have neither, so they get no flag rather than a branch in - // every consumer. + // every consumer. Asked of `object_format()`: the exclusion form this + // used to be (`!= "macos" && != "windows"`) answers "ELF" for any `os` it + // does not name, which is wrong the same way for iOS and for + // wasm32-emscripten — see the sibling derivation earlier in this file. const bool elfTarget = targetTriple.empty() ? bool(mcpp::platform::is_linux) - : (targetTriple.os != "macos" && targetTriple.os != "windows"); + : (targetTriple.object_format() == mcpp::toolchain::triple::ObjectFormat::Elf); auto loader_tag_flag = [&](LinkUnit::Kind kind) -> std::string { if (!elfTarget) return {}; using mcpp::build::loader::Form; diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 69b9b47fb..8f8f9cf13 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -61,6 +61,7 @@ import mcpp.build.runtime_validation; // declared artifact -> identity verdict import mcpp.build.cache_key; import mcpp.pack.abi_tag; // the tag a prebuilt dependency is checked against import mcpp.pack.prebuilt; // …and the check itself +import mcpp.pack.stage_tree; // where `${mcpp.stage_dir}` points, and its manifest import mcpp.build.build_program; import mcpp.build.directives; // directive table: mark / fold_private_tail import mcpp.build.tool_store; // #355 host tools: store layout + key + overrides @@ -998,6 +999,24 @@ export struct BuildOverrides { // One says which PACKAGES enter the graph, the other which TOOLS are // installed, and `mcpp run` needs the second without the first. bool will_run = false; + // ── The packaging pass, when this prepare is one (mcpp 2026.9.11.1+) ──── + // + // `mcpp pack --format ` prepares TWICE, and these two fields are the + // whole difference between the passes. The first sets neither: build + // programs run, declare the formats they provide, and submit no dist + // action because none was asked for. The second sets both, after the link + // and after staging, so the claiming member submits an action whose input + // is a directory that by then exists. + // + // NEITHER VALUE IS DERIVED HERE, AND THAT IS THE POINT. `pack_stage_dir` is + // a function of the package name, the version, the resolved triple and the + // mode, and the resolved triple is not known until a prepare has run. + // Computing it a second time before prepare -- from the host triple, say -- + // is the shape where two derivations of one value agree on every machine + // the author has. Both are read out of what the first pass and `make_plan` + // already answered. + std::string pack_format; + std::filesystem::path pack_stage_dir; }; // ── git dependency helpers ────────────────────────────────────────────────── @@ -1134,6 +1153,34 @@ mcpp::platform::process::RunResult run_with_network_retry( return r; } +// `[package]`, for the build program of the package that declares it. +// +// ONE CALL RATHER THAN A FIELD PER SITE. Two places build a +// `BuildProgramEnv` -- the dependency loop and the root -- and the values a +// build program is told about its own package are the same question in both. +// Setting them field by field at each site is how the two answers drift: the +// root gained `packageName` and the dependency loop gained it separately, and +// a value added to only one of them is a rule package that works for a root +// project and not for a dependency, with nothing failing to say so. +void fill_package_build_env(mcpp::build::BuildProgramEnv& e, + const mcpp::manifest::Manifest& m) +{ + e.packageName = m.package.name; + e.packageNamespace = m.package.namespace_; + e.packageVersion = m.package.version; + e.packageDescription = m.package.description; + e.packageLicense = m.package.license; + e.packageRepo = m.package.repo; + // ';' rather than ',': an author entry is conventionally `Name ` + // and a name may carry a comma, so a comma-joined list cannot be split back + // into the entries it was made from. + e.packageAuthors.clear(); + for (auto const& a : m.package.authors) { + if (!e.packageAuthors.empty()) e.packageAuthors += ';'; + e.packageAuthors += a; + } +} + void fill_target_build_env(mcpp::build::BuildProgramEnv& e, const mcpp::toolchain::Toolchain* tc) { @@ -8198,8 +8245,9 @@ prepare_build(bool print_fingerprint, // The DECLARING package's setting, not the root project's: a rule // generating a declaration for this package must match how this // package is compiled. - bpEnv.packageName = pkg.manifest.package.name; - bpEnv.packageNamespace = pkg.manifest.package.namespace_; + fill_package_build_env(bpEnv, pkg.manifest); + bpEnv.packFormat = overrides.pack_format; + bpEnv.packStageDir = overrides.pack_stage_dir; bpEnv.languageModules = pkg.manifest.language.modules; bpEnv.ruleModules = pkg.manifest.buildConfig.ruleModules; if (auto dit = deviceSourcesByPackage.find(pkg.root.string()); dit != deviceSourcesByPackage.end()) @@ -9126,8 +9174,9 @@ prepare_build(bool print_fingerprint, bpEnv.toolsBin = projectSubosBin; bpEnv.profile = effectiveProfile; bpEnv.accel = resolvedAccel(); - bpEnv.packageName = m->package.name; - bpEnv.packageNamespace = m->package.namespace_; + fill_package_build_env(bpEnv, *m); + bpEnv.packFormat = overrides.pack_format; + bpEnv.packStageDir = overrides.pack_stage_dir; bpEnv.languageModules = m->language.modules; bpEnv.ruleModules = m->buildConfig.ruleModules; if (auto dit = deviceSourcesByPackage.find(root->string()); dit != deviceSourcesByPackage.end()) @@ -10242,7 +10291,29 @@ prepare_build(bool print_fingerprint, // become an edge with a blank path, and ninja reports that far away // from the typo that caused it. std::set unresolvedTargets; - auto substitute = [&](std::string s) { + // `${mcpp.stage_dir}` used where there is no staged tree, and used by an + // action whose role runs before the link. Both are refusals rather than + // empty expansions: an empty path is a token the command still accepts, + // and the tool then reads the build directory root -- which exists, so + // the mistake produces a plausible artifact instead of a diagnostic. + // Section 2 of the design record measured that shape: a valid, empty, + // 52 KB installer with nothing said about it. + std::set stageDirNoPass, stageDirWrongRole; + // WHETHER *THIS* ACTION REFERENCED THE STAGED TREE, and deliberately a + // flag rather than a set keyed on the action's id: an id is unique + // within the package that declared it and nothing more, so two packages + // may each submit a `dist` action called `package`. A set would then + // hand one package's implicit dependency to the other's edge -- the + // shape where a predicate is right and the object is wrong, which does + // not fail, it answers about something else. + // + // The diagnostic sets below stay keyed by id because a diagnostic + // NAMES ids and a collision there costs a duplicate line, not a wrong + // edge. + bool thisActionUsesStageDir = false; + const bool stagePass = !overrides.pack_stage_dir.empty(); + auto substitute = [&](std::string s, const char* actionId, + mcpp::manifest::BuildAction::Role role) { auto rep = [&](std::string_view what, const std::string& with) { for (std::size_t p; (p = s.find(what)) != std::string::npos; ) s.replace(p, what.size(), with); @@ -10250,6 +10321,21 @@ prepare_build(bool print_fingerprint, rep("${mcpp.out_dir}", ctx.plan.outputDir.string()); rep("${mcpp.bin_dir}", (ctx.plan.outputDir / "bin").string()); rep("${mcpp.compile_db}", ctx.plan.compileDbPath.string()); + // ABSOLUTE, unlike `${mcpp.target_file:}` and for the same reason + // stated the other way round: the staged tree lives outside the + // build directory and no ninja edge produces it, so there is no + // edge-declared spelling to agree with. `${mcpp.out_dir}` above is + // absolute on the same grounds. + if (s.find("${mcpp.stage_dir}") != std::string::npos) { + if (!stagePass) { + stageDirNoPass.insert(actionId); + } else if (role != mcpp::manifest::BuildAction::Role::Artifact) { + stageDirWrongRole.insert(actionId); + } else { + thisActionUsesStageDir = true; + } + rep("${mcpp.stage_dir}", overrides.pack_stage_dir.string()); + } constexpr std::string_view kTf = "${mcpp.target_file:"; for (std::size_t p; (p = s.find(kTf)) != std::string::npos; ) { auto close = s.find('}', p); @@ -10278,22 +10364,74 @@ prepare_build(bool print_fingerprint, // mcpp#534's ordering edge is scoped to this name. auto owner = mcpp::build::qualified_package_name(mm); for (auto a : mm.buildConfig.actions) { - for (auto& x : a.inputs) x = substitute(x); - for (auto& x : a.outputs) x = substitute(x); - for (auto& x : a.command) x = substitute(x); + thisActionUsesStageDir = false; + const auto sub = [&](std::string v) { + return substitute(std::move(v), a.id.c_str(), a.role); + }; + for (auto& x : a.inputs) x = sub(x); + for (auto& x : a.outputs) x = sub(x); + for (auto& x : a.command) x = sub(x); // Same closed vocabulary as outputs — a depfile commonly // wants to live at `${mcpp.out_dir}/.d`, beside the // output it describes, and `prepare_actions` above // deliberately left a `${mcpp.` depfile untouched for // exactly this phase to resolve. - if (!a.depfile.empty()) a.depfile = substitute(a.depfile); + if (!a.depfile.empty()) a.depfile = sub(a.depfile); + // THE DEPENDENCY IS IMPLIED BY THE USE, so a member author + // cannot forget it. Without this the edge is dirty only when a + // link output changes, and a staged set that grew a dependency's + // shared library while the program's own bytes did not would + // leave the previous distributable in place, reported as + // up to date. + if (thisActionUsesStageDir) { + a.consumesStageDir = true; + a.inputs.push_back( + mcpp::pack::stage_manifest_path(overrides.pack_stage_dir).string()); + } a.packageName = owner; ctx.plan.actions.push_back(std::move(a)); } + // Every package's declaration, on every pass. Sorted and de-duplicated + // below so the refusal's list reads the same whatever order resolution + // walked the graph in. + for (auto const& f : mm.buildConfig.packFormats) + ctx.plan.providedPackFormats.push_back(f); }; collect(*m); for (std::size_t i = 1; i < packages.size(); ++i) collect(packages[i].manifest); + std::ranges::sort(ctx.plan.providedPackFormats); + ctx.plan.providedPackFormats.erase( + std::ranges::unique(ctx.plan.providedPackFormats).begin(), + ctx.plan.providedPackFormats.end()); + ctx.plan.packFormat = overrides.pack_format; + if (!stageDirNoPass.empty()) { + std::string ids; + for (auto const& n : stageDirNoPass) ids += (ids.empty() ? "" : ", ") + n; + return std::unexpected(std::format( + "build.mcpp action(s) [{}] reference ${{mcpp.stage_dir}}, and this " + "build is not packaging.\n" + " The staged tree is produced by `mcpp pack` after the link, so " + "it does not exist during\n" + " a plain build and there is nothing for the placeholder to name.\n" + " Gate the submission on the format you provide:\n" + " mcpp::provides_pack_format(\"\"); // always\n" + " if (std::string_view(mcpp::pack_format()) == \"\") " + "// then submit\n" + " and reach the tree with `mcpp pack --format `.", ids)); + } + if (!stageDirWrongRole.empty()) { + std::string ids; + for (auto const& n : stageDirWrongRole) ids += (ids.empty() ? "" : ", ") + n; + return std::unexpected(std::format( + "build.mcpp action(s) [{}] reference ${{mcpp.stage_dir}} with a role " + "other than \"artifact\".\n" + " Only an artifact action runs after the link, and the staged tree " + "is a link output's\n" + " successor: a source, object or check action is scheduled before " + "there is anything to stage.\n" + " use: role = \"artifact\"", ids)); + } if (!unresolvedTargets.empty()) { std::string bad, known; for (auto const& n : unresolvedTargets) bad += (bad.empty() ? "" : ", ") + n; @@ -11384,14 +11522,29 @@ prepare_build(bool print_fingerprint, ctx.plan.runtimeBinding), nullptr, false); if (binding.is_discarded()) binding = nlohmann::json::object(); - auto triple = ctx.tc.targetTriple; - std::ranges::transform(triple, triple.begin(), - [](unsigned char c) { return std::tolower(c); }); - const bool pe = triple.find("windows") != std::string::npos - || triple.find("mingw") != std::string::npos; - const bool macho = triple.find("darwin") != std::string::npos - || triple.find("apple") != std::string::npos; - std::string format = pe ? "pe" : macho ? "macho" : "elf"; + // ASKED OF THE PARSED TRIPLE, with the substring test kept only for a + // spelling `parse` rejects. This field is the SECOND copy of a + // derivation `mcpp.build.dist::format_for` already owns, and it had + // the same defect: mcpp's canonical `aarch64-macos` contains neither + // "apple" nor "darwin", so an explicit `--target aarch64-macos` + // recorded `"elf"` while the native build on the same machine recorded + // `"macho"` -- one report contradicting the other about one machine. + std::string format = "elf"; + if (auto t = mcpp::toolchain::triple::parse(ctx.tc.targetTriple)) { + format = std::string(mcpp::toolchain::triple::to_string(t->object_format())); + std::ranges::transform(format, format.begin(), + [](unsigned char c) { return std::tolower(c); }); + if (format == "mach-o") format = "macho"; + } else { + auto triple = ctx.tc.targetTriple; + std::ranges::transform(triple, triple.begin(), + [](unsigned char c) { return std::tolower(c); }); + const bool pe = triple.find("windows") != std::string::npos + || triple.find("mingw") != std::string::npos; + const bool macho = triple.find("darwin") != std::string::npos + || triple.find("apple") != std::string::npos; + format = pe ? "pe" : macho ? "macho" : "elf"; + } // The ORDERED run-time search closure with provenance. Order is // semantics here, not presentation: it is what the loader will walk, // and the mutable SubOS farm sitting last is the invariant that keeps @@ -11410,7 +11563,7 @@ prepare_build(bool print_fingerprint, } nlohmann::json search = { {"format", format}, - {"link_library", pe ? "libpath" : "library_path"}, + {"link_library", format == "pe" ? "libpath" : "library_path"}, {"transitive_needed", format == "elf" ? "rpath_link" : "none"}, {"runtime", format == "pe" ? "deploy" : format == "macho" ? "loader_rpath" : "runpath"}, diff --git a/src/cli.cppm b/src/cli.cppm index a3e4db36a..c45023f2f 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -581,8 +581,14 @@ int run(int argc, char** argv) { .help("system | vendored (default) | self-contained | static")) .option(cl::Option("target").takes_value().multiple() .help("Triple, e.g. x86_64-linux-musl (repeatable: one leg per triple)")) + // "plus any format the resolved graph provides", because the + // values are no longer a fixed list. A package declares one with + // `mcpp::provides_pack_format` and `--format ` dispatches to + // it; an unknown value names what IS available rather than a + // constant, so the help text does not have to enumerate them. .option(cl::Option("format").takes_value() - .help("tar (default; .zip for a Windows target) | dir")) + .help("tar (default; .zip for a Windows target) | dir | any " + "format the resolved graph provides (e.g. appimage, msi)")) .option(cl::Option("output").short_name('o').takes_value() .help("Override output path")) // Packaging builds RELEASE by default — the artifact leaves this diff --git a/src/cli/cmd_publish.cppm b/src/cli/cmd_publish.cppm index d6da52da7..216232338 100644 --- a/src/cli/cmd_publish.cppm +++ b/src/cli/cmd_publish.cppm @@ -47,13 +47,25 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { opts.mode = *m; modeFromUser = true; } + // THE VALUE IS NOT VALIDATED HERE, AND THAT IS THE CHANGE. + // + // `tar` and `dir` are the archive shapes the engine owns. Everything else + // is a name a package provides, and which names those are is a property of + // the RESOLVED GRAPH -- so a refusal written here could only compare + // against a constant, which is exactly the coupling this whole mechanism + // exists to remove. The refusal moves to `build_and_pack`, after build + // programs have declared what they provide and before anything is + // compiled, where it can name what IS available instead of a fixed list. if (auto v = parsed.value("format")) { if (*v == "tar") opts.format = mcpp::pack::Format::Tar; else if (*v == "dir") opts.format = mcpp::pack::Format::Dir; - else { - mcpp::ui::error(std::format( - "invalid --format '{}'; expected tar | dir", *v)); + else if (v->empty()) { + mcpp::ui::error("--format needs a value: tar | dir | a format the " + "resolved graph provides"); return 2; + } else { + opts.format = mcpp::pack::Format::Dispatched; + opts.formatName = *v; } } if (auto v = parsed.value("output")) opts.output = *v; @@ -85,6 +97,23 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { auto route = mcpp::pack::route_pack_target(parsed.positional(0)); if (!route) { mcpp::ui::error(route.error()); return 2; } if (route->library) { + // A DISPATCHED FORMAT IS AN APPLICATION-BUNDLE OUTPUT, and a library + // package has no bundle: `mcpp pack ` produces an interface plus + // prebuilt binaries for one or more triples, and there is no single + // staged tree for a member to turn into an installer. Refused rather + // than ignored, because ignoring it would report `Packed` and hand back + // a library package while the user asked for an installer. + if (opts.format == mcpp::pack::Format::Dispatched) { + mcpp::ui::error(std::format( + "--format {} is a distributable produced from a program's staged " + "bundle, and '{}' is a library target.\n" + " A library package ships an interface plus prebuilt binaries " + "per triple; there is no\n" + " single staged tree to hand a distribution format.\n" + " use: --format tar | dir, or name a program target", + opts.formatName, route->targetName)); + return 2; + } if (modeFromUser) { mcpp::ui::warning(std::format( "--mode is an application-bundle depth and does not apply to the " diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index 413800c53..b2437b1f8 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -50,11 +50,33 @@ export namespace mcpp::pack { enum class Mode { None, Static, BundleProject, BundleAll }; -enum class Format { Tar, Dir }; +// WHAT SHAPE THE OUTPUT TAKES, and the third value is the one that is not a +// shape the engine knows. +// +// `tar` and `dir` answer the same question `msi` and `appimage` answer, so they +// belong on one axis -- which is why this is a wider set of values for one flag +// rather than a second flag. `Dispatched` carries a name the engine has never +// heard: `mcpp pack --format appimage` finds the provider among the resolved +// dependencies and hands it the staged tree, exactly as `--target` reaches a +// triple the engine did not have to know individually. +// +// The engine keeps `Tar` and `Dir` because an archive that extracts and runs is +// universal in the only sense that matters here: it needs no knowledge of +// anyone else's release. dpkg's control fields, AppImage's runtime, WiX's +// schema and Apple's notarisation each couple an mcpp release to a release mcpp +// does not control. +enum class Format { Tar, Dir, Dispatched }; struct Options { Mode mode = Mode::BundleProject; Format format = Format::Tar; + // The `--format` value when `format == Dispatched`. Empty otherwise. + // + // Not validated by the CLI, and deliberately: the set of valid values is a + // property of the RESOLVED GRAPH, so the refusal has to wait until build + // programs have declared what they provide. It arrives before anything is + // compiled, which is the earliest point at which it can be exact. + std::string formatName; std::filesystem::path output; // empty = derive from manifest std::string targetTriple; // empty = host // Where a dependency NAME may be resolved to a file. @@ -1149,6 +1171,29 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) // it and nothing has modified either one at this point (patchelf runs // further down). What changes is only which directory `$ORIGIN` // expands to while the loader is looking. + // + // A NON-ELF ARTIFACT REACHING THIS POINT ASSUMED ELF BY EXCLUSION. + // PE and Mach-O are refused by name above `run()`'s `#else`; nothing + // between there and here asks what is LEFT actually is ELF, because + // ELF used to be the only format left once those two were excluded. + // wasm32-emscripten is the first target where that assumption is + // false: `binfmt::identify` reports `Format::Unknown` for a `.wasm` + // module (it carries none of the three magics), and `ldd_parse` + // below runs the file through the same LD_TRACE_LOADED_OBJECTS + // mechanism the Mach-O branch above refuses by name rather than + // risk — on a host where `.wasm` is registered in `binfmt_misc`, + // that does not fail, it RUNS the module. + if (auto fmt = mcpp::pack::binfmt::identify(plan.builtBinary).format; + fmt != mcpp::pack::binfmt::Format::Elf) { + return std::unexpected(Error{std::format( + "cannot package the {} artifact '{}' yet.\n" + " Its dependency closure is resolved by running the " + "artifact under its own\n" + " dynamic linker, and this file is neither ELF, PE nor " + "Mach-O -- there is no\n" + " such linker to ask.", + mcpp::pack::binfmt::format_name(fmt), plan.binaryName)}); + } auto deps = ldd_parse(plan.builtBinary); if (!deps) return std::unexpected(Error{std::format( "ldd failed on {}: {}", plan.builtBinary.string(), deps.error())}); diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index bebbe7629..8739e805a 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -17,7 +17,9 @@ import mcpp.build.ninja; import mcpp.build.plan; import mcpp.config; import mcpp.fetcher.progress; +import mcpp.manifest; import mcpp.pack; +import mcpp.pack.stage_tree; import mcpp.pack.strip; import mcpp.toolchain.model; import mcpp.toolchain.registry; @@ -56,8 +58,24 @@ export int build_and_pack(Options opts, bool modeFromUser, ov.profile = opts.profile; ov.profile_fallback = "release"; + // QUIET FOR A DISPATCHED FORMAT, AND ONLY UNTIL THE VALUE IS VALIDATED. + // + // `mcpp pack --format bogus` must write NOTHING to stdout and exit 2 -- + // the machine-output contract, asserted by + // tests/e2e/202_machine_output_contract.sh, because this is the path a + // client hits when it probes an mcpp for a capability. The set of valid + // values is a property of the resolved graph, so the refusal cannot be + // decided until prepare has run, and prepare narrates what it resolves. + // + // Nothing is lost when the value IS valid: the dispatch pass prepares a + // second time and prints the same lines, so a successful + // `pack --format ` narrates once rather than twice. + const bool quietUntilValidated = + opts.format == mcpp::pack::Format::Dispatched && !mcpp::ui::is_quiet(); + if (quietUntilValidated) mcpp::ui::set_quiet(true); auto ctx = mcpp::build::prepare_build(/*print_fp=*/false, /*includeDevDeps=*/false, /*extraTargets=*/{}, ov); + if (quietUntilValidated) mcpp::ui::set_quiet(false); if (!ctx) { mcpp::ui::error(ctx.error()); return 2; @@ -83,15 +101,70 @@ export int build_and_pack(Options opts, bool modeFromUser, && opts.targetTriple.empty() && ctx->tc.targetTriple.find("-musl") == std::string::npos) { // Need to re-prepare the build with the musl target. - mcpp::build::BuildOverrides ov2; - ov2.target_triple = "x86_64-linux-musl"; - ov2.profile = opts.profile; - ov2.profile_fallback = "release"; - auto ctx2 = mcpp::build::prepare_build(false, false, {}, ov2); + // + // `ov` IS MUTATED RATHER THAN SHADOWED. It has to stay the record of + // what produced `ctx`, because the dispatch pass below re-enters + // prepare with the same overrides plus two fields -- and a second + // overrides object left behind here would make that pass differ from + // this build in a way nothing states. + ov.target_triple = "x86_64-linux-musl"; + // Quiet on the same grounds as the first prepare: this one also runs + // before `--format` has been validated. + if (quietUntilValidated) mcpp::ui::set_quiet(true); + auto ctx2 = mcpp::build::prepare_build(false, false, {}, ov); + if (quietUntilValidated) mcpp::ui::set_quiet(false); if (!ctx2) { mcpp::ui::error(ctx2.error()); return 2; } ctx = std::move(ctx2); } + // ─── Is the requested format one anything provides? ────────────── + // + // BEFORE THE BUILD, because a refusal that arrives after a full compile is + // a worse refusal, and because this is the earliest point at which it can + // be exact: build programs have now run and declared what they provide. + // + // The set is read from a pass that asked for NOTHING. That is what the + // "declare unconditionally, submit conditionally" rule buys -- a member + // that declared only when asked would leave this list empty exactly when a + // user names a format, and the refusal would name nothing. + if (opts.format == mcpp::pack::Format::Dispatched) { + auto const& provided = ctx->plan.providedPackFormats; + if (std::ranges::find(provided, opts.formatName) == provided.end()) { + std::string avail; + for (auto b : mcpp::pack::kBuiltinPackFormats) + avail += (avail.empty() ? "" : ", ") + std::string(b); + for (auto const& f : provided) { + if (mcpp::pack::is_builtin_pack_format(f)) continue; + avail += ", " + f; + } + mcpp::ui::error(std::format( + "unknown --format '{}'.\n" + " available in this build: {}\n" + " A format past `tar` and `dir` comes from a package in the " + "resolved graph, which declares\n" + " it with `mcpp::provides_pack_format(\"\")` in its build " + "program. Add the package\n" + " that provides '{}' to [build-dependencies] and activate its " + "feature.", + opts.formatName, avail, opts.formatName)); + return 2; + } + } + + // A package claiming a built-in name is silently unreachable, since the + // parser resolves `tar` and `dir` before consulting the graph at all. + // + // OUTSIDE THE DISPATCH BRANCH ABOVE, because the mistake is in the PACKAGE + // and does not depend on what this invocation asked for. Reported on every + // pack, so the author hears it on the plain `mcpp pack` they are most + // likely to run. + for (auto const& f : ctx->plan.providedPackFormats) + if (mcpp::pack::is_builtin_pack_format(f)) + mcpp::ui::warning(std::format( + "a package in this graph declares `mcpp:pack-format={}`, which " + "is one of the archive shapes `mcpp pack` owns; `--format {}` " + "will always select the built-in and never that package", f, f)); + auto be = mcpp::build::make_ninja_backend(); mcpp::build::BuildOptions bo; auto br = be->build(ctx->plan, bo); @@ -219,7 +292,133 @@ export int build_and_pack(Options opts, bool modeFromUser, return 1; } + // The staged tree is now on disk and final -- past the closure, the + // `$ORIGIN` rewriting, the strip and the debug split. Describe it, so an + // action that consumes it has something whose CONTENT changes when the + // staged set does. Best-effort: see write_stage_manifest. + mcpp::pack::write_stage_manifest(plan->stagingRoot); + auto pathCtx = mcpp::fetcher::make_path_ctx(&*cfg, ctx->projectRoot); + + // ─── The dispatch pass ─────────────────────────────────────────── + // + // A `role = "artifact"` action is a ninja edge, and the staged tree is + // produced here, in C++, AFTER ninja has finished. So an artifact action + // cannot depend on the staged tree in the pass that built it, and a + // single-pass `--format ` is not expressible. Two passes are, and + // every value this one needs was answered by the first: + // + // `ov` the overrides that produced the build above + // `plan->stagingRoot` from make_plan, which resolved the triple + // `opts.formatName` the request, already checked against the graph + // + // NOTHING IS RE-DERIVED, and that is the whole discipline of this block. + // `stagingRoot` is a function of the package name, the version, the + // resolved triple and the mode; the resolved triple is not known until a + // prepare has run, so computing it a second time before prepare -- from the + // host triple, say -- is the shape where two derivations of one value agree + // on every machine the author has and disagree on one they do not. + if (opts.format == mcpp::pack::Format::Dispatched) { + // WHICH ARTIFACT ACTIONS THIS BUILD ALREADY HAD, before a format was + // requested. The dispatch below reports what the REQUEST introduced, + // and this is the other half of that subtraction. + std::set> preexistingArtifacts; + for (auto const& a : ctx->plan.actions) + if (a.role == mcpp::manifest::BuildAction::Role::Artifact) + preexistingArtifacts.emplace(a.packageName, a.id); + + ov.pack_format = opts.formatName; + ov.pack_stage_dir = plan->stagingRoot; + auto distCtx = mcpp::build::prepare_build(false, false, {}, ov); + if (!distCtx) { mcpp::ui::error(distCtx.error()); return 2; } + + // WHICH ACTIONS ARE THE DISTRIBUTABLE: the artifact actions the REQUEST + // INTRODUCED. An action present in both passes existed before anyone + // asked for a format -- a codesign stamp, a size budget -- and + // reporting one as the package would be a wrong answer that looks like + // a right one. + // + // THE FIRST VERSION ASKED A NARROWER QUESTION AND GOT IT WRONG. It + // collected only actions naming `${mcpp.stage_dir}`, on the assumption + // that a distributable consumes the staged closure. Not every format + // does: an `.msi` built from ONE named program takes + // `${mcpp.target_file:}` and never looks at the tree, which is + // the shape section 6 of the design record recommends -- "name the + // input, do not harvest a directory", after a bind path that resolved + // to nothing produced a valid, empty, 52 KB installer. So the member + // that followed the guidance was the member the check refused, and the + // workaround was to name the placeholder as an unused input purely to + // satisfy it. Presence-in-this-pass is the property actually wanted, + // and it needs nothing of the member. + // + // Identity is (package, id): an id is unique within the package that + // declared it and nothing more. + std::vector distOutputs; + for (auto const& a : distCtx->plan.actions) { + if (a.role != mcpp::manifest::BuildAction::Role::Artifact) continue; + if (preexistingArtifacts.contains({a.packageName, a.id})) continue; + for (auto const& o : a.outputs) distOutputs.push_back(o); + } + // DECLARED AND THEN SUBMITTED NOTHING. The half of the contract a + // member is most likely to get wrong is the gate, and a member whose + // gate never opens leaves a pass that succeeds and produces no + // package. Refused by name rather than reported as success. + if (distOutputs.empty()) { + mcpp::ui::error(std::format( + "no action claimed --format '{}'.\n" + " A package declared it provides this format, and no build " + "program submitted a new\n" + " `role = \"artifact\"` action when it was asked for.\n" + " The provider must gate on the request and not on anything " + "else:\n" + " mcpp::provides_pack_format(\"{}\"); " + "// always\n" + " if (std::string_view(mcpp::pack_format()) == \"{}\") ..." + " // then submit", + opts.formatName, opts.formatName, opts.formatName)); + return 1; + } + + mcpp::ui::info("Distributing", std::format("{} v{} (--format {})", + plan->packageName, plan->packageVersion, opts.formatName)); + + // NO EXPLICIT GOALS. Everything but the dist edges is already up to + // date from the build above, so a full drive costs a graph scan and + // nothing else -- and an explicit goal set is how the 0.0.104 soname + // aliases went missing, because an edge reachable only through + // `default` is skipped under one. + mcpp::build::BuildOptions dbo; + auto dr = be->build(distCtx->plan, dbo); + if (!dr) { + if (!dr.error().diagnosticOutput.empty()) { + std::fputs(dr.error().diagnosticOutput.c_str(), stderr); + if (dr.error().diagnosticOutput.back() != '\n') std::fputs("\n", stderr); + } + mcpp::ui::error(dr.error().message); + return 1; + } + + // THE CRITERION IS THE FILE, NOT THE EXIT CODE. A cached build program + // replaying the first pass's answer, or a tool that writes nothing and + // exits 0, both leave ninja reporting success -- and section 2's + // measured failure was a packaging step that succeeded while carrying + // nothing. + std::error_code ec; + for (auto const& o : distOutputs) { + auto abs = std::filesystem::path(o).is_absolute() + ? std::filesystem::path(o) : distCtx->plan.outputDir / o; + if (!std::filesystem::is_regular_file(abs, ec) + && !std::filesystem::is_directory(abs, ec)) { + mcpp::ui::error(std::format( + "--format {} reported success and produced nothing at {}", + opts.formatName, abs.string())); + return 1; + } + mcpp::ui::status("Packed", mcpp::ui::shorten_path(abs, pathCtx)); + } + return 0; + } + auto outPath = (opts.format == mcpp::pack::Format::Tar) ? plan->archivePath : plan->stagingRoot; mcpp::ui::status("Packed", mcpp::ui::shorten_path(outPath, pathCtx)); diff --git a/src/pack/stage_tree.cppm b/src/pack/stage_tree.cppm new file mode 100644 index 000000000..b24f25241 --- /dev/null +++ b/src/pack/stage_tree.cppm @@ -0,0 +1,125 @@ +// mcpp.pack.stage_tree — what a staged tree promises an artifact action, and +// the one file that says so. +// +// `mcpp pack` has always computed a staged tree: the dependency closure after +// the strip policy, the debug-symbol split and `include`/`exclude`. Until +// `${mcpp.stage_dir}` it then compressed the tree and the directory was gone, +// so a `.deb`, an AppImage, a `.app` and an `.msi` each had to rebuild the same +// closure. Exposing it is one addition that serves every format and encodes no +// format's knowledge, which is the test for whether something belongs in the +// engine at all. +// +// WHY THERE IS A MANIFEST FILE AND NOT JUST A DIRECTORY. ninja identifies an +// input by a path and compares an mtime. A directory's mtime moves when its +// immediate entries change and not when a file two levels down is replaced, so +// naming the directory as an input would make the dist edge dirty for the wrong +// reasons and clean for the wrong reasons. The manifest is CONTENT-BEARING — +// one ` ` line per staged file, sorted — so it changes +// exactly when the staged set or any staged file's length changes, and it is a +// single ordinary file that ninja can compare. +// +// IT IS A SIBLING OF THE TREE, NOT A MEMBER OF IT. A file inside the staged +// directory would be collected by every format that packages the directory +// wholesale, and would then ship inside the user's installer. The sibling +// spelling is what keeps a mechanism the engine added from appearing in a +// product it does not own. +// +// The manifest deliberately records SIZES rather than content hashes. The tree +// can be hundreds of megabytes and is rebuilt on every pack; hashing it would +// make the common case pay for a distinction the uncommon case does not need, +// because a staged file whose length is unchanged and whose bytes differ can +// only have come from a rebuild, and a rebuild moved the link output that the +// dist edge also depends on. + +module; +#include + +export module mcpp.pack.stage_tree; + +import std; + +export namespace mcpp::pack { + +// The manifest that describes the tree staged at `stagingRoot`. +// +// A SIBLING, DERIVED FROM THE NAME RATHER THAN PLACED INSIDE. Spelled in one +// function because two readers need the same answer for different reasons: +// `prepare` names it as an implicit input while the file does not yet exist, +// and `pack::run` writes it. A second derivation of a path is a path that +// disagrees on the platform whose separator the author did not test. +std::filesystem::path stage_manifest_path(const std::filesystem::path& stagingRoot) { + auto p = stagingRoot; + p += ".stage-manifest"; + return p; +} + +// Write the manifest for the tree now on disk at `stagingRoot`. +// +// Best-effort by construction and deliberately so: the manifest is a +// dependency-tracking convenience, and a pack that produced a correct tree must +// not fail because a sibling bookkeeping file could not be written. A missing +// manifest makes the dist edge fail with ninja's own "missing and no known rule +// to make it", which names the file — a legible failure rather than a silent +// staleness. +bool write_stage_manifest(const std::filesystem::path& stagingRoot) { + std::error_code ec; + if (!std::filesystem::is_directory(stagingRoot, ec)) return false; + + std::vector lines; + for (auto const& entry : + std::filesystem::recursive_directory_iterator( + stagingRoot, std::filesystem::directory_options::skip_permission_denied, ec)) + { + if (ec) break; + // Symlinks are recorded by NAME AND NOT FOLLOWED. `bundle-all` + // dereferences a soname link while staging, so what remains is a real + // file; a link that survives points outside the tree, and following it + // would make the manifest describe a file the package does not carry. + if (!entry.is_regular_file(ec) || entry.is_symlink()) { + if (entry.is_symlink()) + lines.push_back(std::format("link {}", + std::filesystem::relative(entry.path(), stagingRoot, ec).generic_string())); + continue; + } + auto rel = std::filesystem::relative(entry.path(), stagingRoot, ec).generic_string(); + if (ec || rel.empty()) continue; + lines.push_back(std::format("{} {}", + std::filesystem::file_size(entry.path(), ec), rel)); + } + // Sorted, because a directory iteration order is not a promise. Two packs + // of one tree must produce identical bytes or the dist edge is dirty on + // every run for no reason. + std::ranges::sort(lines); + + std::string text; + for (auto const& l : lines) { text += l; text.push_back('\n'); } + + auto out = stage_manifest_path(stagingRoot); + // Compared before writing, for the reason `mcpp.build.stage` gives at + // length: rewriting identical bytes moves the mtime, and a moved mtime on + // an input is indistinguishable from a changed input. A pack that staged + // the same tree twice would rebuild the distributable both times. + if (std::ifstream in(out, std::ios::binary); in) { + std::string old((std::istreambuf_iterator(in)), + std::istreambuf_iterator()); + if (old == text) return true; + } + std::ofstream os(out, std::ios::binary | std::ios::trunc); + if (!os) return false; + os.write(text.data(), static_cast(text.size())); + return static_cast(os); +} + +// The two names `--format` answers for without consulting the graph. +// +// Held here rather than in the CLI because two layers need the same list: the +// parser decides whether a value is a built-in or a dispatch, and the +// declaration check refuses a package that claims one of them. A package +// claiming `tar` would be silently unreachable, since the built-in wins. +constexpr std::array kBuiltinPackFormats{"tar", "dir"}; + +bool is_builtin_pack_format(std::string_view name) { + return std::ranges::find(kBuiltinPackFormats, name) != kBuiltinPackFormats.end(); +} + +} // namespace mcpp::pack diff --git a/src/pack/strip.cppm b/src/pack/strip.cppm index ab2143732..6707ebb51 100644 --- a/src/pack/strip.cppm +++ b/src/pack/strip.cppm @@ -166,7 +166,12 @@ bool debug_info_is_in_band(std::string_view canonicalTriple) { seg.push_back(canonicalTriple.substr(i, j - i)); i = j + 1; } - if (seg.size() >= 2 && seg[1] == "macos") return false; // debug map + .dSYM + // Mach-O, not "macOS": iOS carries the same debug map + out-of-band + // .dSYM as macOS (same ld64, same object format). This module takes a + // string rather than a `Triple` on purpose (see the note on + // `debug_info_is_in_band` above), so the grouping `is_mach_o()` states is + // spelled out here instead of asked of it. + if (seg.size() >= 2 && (seg[1] == "macos" || seg[1] == "ios")) return false; if (seg.size() >= 3 && seg[2] == "msvc") return false; // separate .pdb return true; } diff --git a/src/toolchain/lifecycle.cppm b/src/toolchain/lifecycle.cppm index dc7fa6277..638014f6e 100644 --- a/src/toolchain/lifecycle.cppm +++ b/src/toolchain/lifecycle.cppm @@ -623,7 +623,18 @@ export int toolchain_list(const mcpp::config::GlobalConfig& cfg, if (!info->note.empty()) tags.emplace_back(info->note); if (info->defaultStatic) tags.push_back("static"); } - if (t != hostT && (t.os != hostT.os || t.arch != hostT.arch)) + // A TARGET THIS HOST CANNOT EXECUTE. Arch and OS are the usual answer, + // and deliberately not env: an `x86_64-linux-musl` artifact is static + // and runs here, so calling it cross would be false. + // + // The env axis matters for exactly one row today. An Android artifact + // needs bionic's loader at `/system/bin/linker64`, which no ordinary + // Linux host has -- so `x86_64-linux-android` agrees with this host on + // both segments the test looked at and cannot run on it. Spelled as a + // property rather than by adding `env != env`, which would have taken + // musl with it. + if (t != hostT && (t.os != hostT.os || t.arch != hostT.arch + || t.is_android())) tags.push_back("cross"); std::string out; for (auto& tag : tags) { if (!out.empty()) out += ", "; out += tag; } diff --git a/tests/e2e/638_pack_format_dispatch.sh b/tests/e2e/638_pack_format_dispatch.sh new file mode 100755 index 000000000..b452ea581 --- /dev/null +++ b/tests/e2e/638_pack_format_dispatch.sh @@ -0,0 +1,312 @@ +#!/usr/bin/env bash +# requires: gcc +# 638_pack_format_dispatch.sh — `mcpp pack --format ` dispatches to a +# package, and no distribution format lives in the engine. +# +# `tar` and `dir` answer the same question `msi` and `appimage` answer -- what +# shape does the output take -- so they are values of ONE flag, and everything +# past the two the engine owns comes from the resolved graph. What the engine +# adds is the mechanism: a staged tree an artifact action can consume, the rest +# of `[package]` in the build program, and the dispatch itself. +# +# The four properties this holds, each with the wrong answer it excludes: +# +# 1. DECLARE UNCONDITIONALLY, SUBMIT CONDITIONALLY. A build that asks for no +# format still declares one, so `--format bogus` can name what IS +# available. A member that declared only when asked works for its author +# and makes the set unknowable for everyone else. +# 2. THE STAGED TREE IS REAL WHEN THE ACTION RUNS. Asserted from INSIDE the +# action, by listing the tree into the output -- not by the action's exit +# code, because a command that writes nothing and exits 0 is the measured +# failure this whole mechanism exists to prevent. +# 3. A PLAIN BUILD HAS NO DISTRIBUTION EDGE, before or after a pack. +# 4. THE PLACEHOLDER REFUSES OUTSIDE A PACKAGING PASS, rather than expanding +# to an empty string that the tool would accept. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +# ── The provider: one build program, both halves of the contract ─────────── +mkdir -p app/src +cd app + +cat > mcpp.toml <<'EOF' +[package] +name = "app" +version = "2.5.0" +description = "a program that ships" +license = "Apache-2.0" +authors = ["Ada "] +repo = "https://example.org/app" + +[targets.app] +kind = "bin" +main = "src/main.cpp" +EOF + +cat > src/main.cpp <<'EOF' +#include +int main() { std::puts("app"); return 0; } +EOF + +# The dist step. A separate script because an action's command is an argv with +# no shell assumed, which is the same reason 188_build_actions.sh writes one. +# +# It writes the package VERSION and the staged tree's top level into its +# output, so the assertions below read what the action actually saw rather than +# whether it exited 0. +cat > dist.sh <<'EOF' +#!/usr/bin/env bash +set -e +version="$1"; stage="$2"; out="$3" +{ + echo "version=$version" + echo "staged:" + ls -1 "$stage" | sort +} > "$out" +EOF +chmod +x dist.sh + +cat > build.mcpp <<'EOF' +import mcpp; +#include +#include +#include +int main() { + // Half one: unconditional. This is what makes the set knowable on a build + // that asked for nothing. + mcpp::provides_pack_format("zap"); + + // Half two: conditional. A plain build must have no such edge. + if (std::string_view(mcpp::pack_format()) != "zap") return 0; + + const std::string root = mcpp::manifest_dir(); + const std::string out = std::string(mcpp::out_dir()) + "/app.zap"; + mcpp::action a; + a.id = "zap"; + a.role = "artifact"; + a.description = "zap"; + a.arg((root + "/dist.sh").c_str()) + // The VERSION comes from `[package]`, not from an option the project + // restates: a second copy drifts with nothing able to detect it. + .arg(mcpp::package_version()) + .arg("${mcpp.stage_dir}") + .arg(out.c_str()) + .input("${mcpp.target_file:app}") + .output(out.c_str()) + .submit(); + return 0; +} +EOF + +MCPP="${MCPP:-mcpp}" + +graph_dist() { sed -n '2p' "$1" | sed 's/.*;dist=//;s/;.*//'; } +find_graph() { find target -name build.ninja | head -1; } + +# ── 1. a plain build declares, and submits nothing ───────────────────────── +"$MCPP" build --release > b1.log 2>&1 || { cat b1.log; echo "FAIL: plain build failed"; exit 1; } +G=$(find_graph) +[ -n "$G" ] || { echo "FAIL: no build.ninja"; exit 1; } +[ "$(graph_dist "$G")" = "none" ] \ + || { sed -n '2p' "$G"; echo "FAIL: a plain build's graph is not dist=none"; exit 1; } +[ -z "$(find target -name 'app.zap' 2>/dev/null)" ] \ + || { echo "FAIL: a plain build produced a distributable"; exit 1; } + +# ── 2. an unknown format is refused, BEFORE the build, naming what exists ── +set +e +"$MCPP" pack --format bogus > b2.log 2>&1 +rc=$? +set -e +[ "$rc" -ne 0 ] || { cat b2.log; echo "FAIL: --format bogus was accepted"; exit 1; } +grep -q "unknown --format 'bogus'" b2.log \ + || { cat b2.log; echo "FAIL: the refusal does not name the value"; exit 1; } +# THE LIST IS THE POINT. A refusal that named a fixed list would be the +# coupling this mechanism removes; this one names the graph's own answer. +grep -q "available in this build: tar, dir, zap" b2.log \ + || { cat b2.log; echo "FAIL: the refusal does not name what is available"; exit 1; } +# Nothing was compiled to find that out. +grep -q "Compiling app" b2.log \ + && { cat b2.log; echo "FAIL: the refusal arrived after a compile"; exit 1; } +# AND NOTHING REACHED STDOUT. This is the path a client hits when it probes an +# mcpp for a capability, so the machine-output contract +# (202_machine_output_contract.sh) requires an empty stdout, a non-empty stderr +# and exit 2. It is asserted here as well because the tension is local to this +# feature: the valid set is a property of the resolved graph, so the refusal +# cannot be decided until prepare has run -- and prepare narrates what it +# resolves. Deciding it later is what broke the contract once. +set +e +so=$("$MCPP" pack --format bogus 2>/dev/null); rc=$? +se=$("$MCPP" pack --format bogus 2>&1 >/dev/null) +set -e +[ -z "$so" ] || { echo "FAIL: the refusal wrote to stdout: $(echo "$so" | head -1)"; exit 1; } +[ -n "$se" ] || { echo "FAIL: the refusal said nothing on stderr"; exit 1; } +[ "$rc" = 2 ] || { echo "FAIL: the refusal exited $rc, expected 2"; exit 1; } + +# ── 3. the dispatched format produces a file, and it saw the staged tree ─── +"$MCPP" pack --format zap > b3.log 2>&1 || { cat b3.log; echo "FAIL: pack --format zap failed"; exit 1; } +Z=$(find target -name 'app.zap' | head -1) +[ -n "$Z" ] || { cat b3.log; echo "FAIL: --format zap produced nothing"; exit 1; } +# The engine handed the build program the rest of `[package]`. +grep -qx "version=2.5.0" "$Z" \ + || { cat "$Z"; echo "FAIL: the action was not told the package version"; exit 1; } +# The action ran with a staged tree that already held the program. This is the +# assertion the ordering exists for: an artifact edge is scheduled by ninja and +# the tree is staged by mcpp after the link, so a single-pass design would run +# this against a directory that does not exist. +grep -qx "bin" "$Z" \ + || { cat "$Z"; echo "FAIL: the staged tree had no bin/ when the action ran"; exit 1; } +grep -q "Packed" b3.log \ + || { cat b3.log; echo "FAIL: the produced file was not reported"; exit 1; } +# The staged tree is described by a SIBLING of itself, never a member: a file +# inside it would ship inside every format that packages the directory. +[ -n "$(find target/dist -maxdepth 1 -name '*.stage-manifest' 2>/dev/null)" ] \ + || { echo "FAIL: no stage manifest beside the staged tree"; exit 1; } +[ -z "$(find target/dist -mindepth 2 -name '*.stage-manifest' 2>/dev/null)" ] \ + || { echo "FAIL: the stage manifest landed inside the staged tree"; exit 1; } + +# ── 4. a plain build AFTER the pack still has no distribution edge ───────── +rm -f "$Z" +"$MCPP" build --release > b4.log 2>&1 || { cat b4.log; echo "FAIL: build after pack failed"; exit 1; } +[ "$(graph_dist "$(find_graph)")" = "none" ] \ + || { echo "FAIL: the graph still says dist= after a plain build"; exit 1; } +[ ! -f "$Z" ] \ + || { echo "FAIL: a plain build regenerated the distributable"; exit 1; } + +# ── 5. the placeholder outside a packaging pass is refused ───────────────── +# The same action, ungated. An empty expansion would give the tool the build +# directory root, which exists -- so the mistake would produce a plausible +# artifact instead of a diagnostic. +cd "$TMP" +cp -r app ungated +cd ungated +cat > build.mcpp <<'EOF' +import mcpp; +#include +int main() { + const std::string root = mcpp::manifest_dir(); + const std::string out = std::string(mcpp::out_dir()) + "/app.zap"; + mcpp::action a; + a.id = "zap"; a.role = "artifact"; + a.arg((root + "/dist.sh").c_str()).arg("x").arg("${mcpp.stage_dir}").arg(out.c_str()) + .input("${mcpp.target_file:app}").output(out.c_str()).submit(); + return 0; +} +EOF +set +e +"$MCPP" build --release > b5.log 2>&1 +rc=$? +set -e +[ "$rc" -ne 0 ] || { cat b5.log; echo "FAIL: an ungated stage_dir built"; exit 1; } +grep -q "this build is not packaging" b5.log \ + || { cat b5.log; echo "FAIL: the refusal does not say why"; exit 1; } +grep -q "provides_pack_format" b5.log \ + || { cat b5.log; echo "FAIL: the refusal does not say what to do instead"; exit 1; } + +# ── 6. a role other than artifact cannot reach the staged tree ───────────── +# GATED, so this is a role refusal and not the one above. An ungated action is +# already refused in the FIRST pass of `mcpp pack`, before there is a staged +# tree, which is why the two cases need different fixtures rather than a +# one-word edit: only a gated action gets far enough for its role to matter. +cd "$TMP" +cp -r app wrongrole +cd wrongrole +cat > build.mcpp <<'EOF' +import mcpp; +#include +#include +int main() { + mcpp::provides_pack_format("zap"); + if (std::string_view(mcpp::pack_format()) != "zap") return 0; + const std::string root = mcpp::manifest_dir(); + const std::string out = std::string(mcpp::out_dir()) + "/app.zap"; + mcpp::action a; + a.id = "zap"; a.role = "source"; + a.arg((root + "/dist.sh").c_str()).arg("x").arg("${mcpp.stage_dir}").arg(out.c_str()) + .output(out.c_str()).submit(); + return 0; +} +EOF +set +e +"$MCPP" pack --format zap > b6.log 2>&1 +rc=$? +set -e +[ "$rc" -ne 0 ] || { cat b6.log; echo "FAIL: a source action reached the staged tree"; exit 1; } +grep -q 'other than "artifact"' b6.log \ + || { cat b6.log; echo "FAIL: the role refusal does not name the role"; exit 1; } + +# ── 7. declared and never submitted is a refusal, not a success ──────────── +# THE HALF A MEMBER AUTHOR IS MOST LIKELY TO GET WRONG. A gate that never +# opens leaves a pass that succeeds and produces no package, which reads as +# "packaging is not implemented yet" rather than as a defect in the member. +cd "$TMP" +cp -r app silent +cd silent +cat > build.mcpp <<'EOF' +import mcpp; +int main() { mcpp::provides_pack_format("zap"); return 0; } +EOF +set +e +"$MCPP" pack --format zap > b7.log 2>&1 +rc=$? +set -e +[ "$rc" -ne 0 ] || { cat b7.log; echo "FAIL: a format nothing claimed reported success"; exit 1; } +grep -q "no action claimed --format 'zap'" b7.log \ + || { cat b7.log; echo "FAIL: the refusal does not name the unclaimed format"; exit 1; } + +# ── 8. an artifact action that predates the request is not the package ───── +# THE CRITERION IS "WHAT THE REQUEST INTRODUCED", and this is what distinguishes +# it from "any artifact action". A codesign stamp or a size budget is also an +# artifact action and is present whether or not a format was asked for; naming +# one as the distributable would be a wrong answer that looks like a right one. +# +# It is also what an earlier revision got wrong from the other side: the check +# collected only actions naming ${mcpp.stage_dir}, which refused a member that +# packages ONE NAMED PROGRAM and never reads the tree -- the shape the design +# record recommends, after a bind path that resolved to nothing produced a +# valid, empty, 52 KB installer. So this fixture submits both shapes: an +# ungated stamp that must be ignored, and a gated action that names no staged +# tree at all and must still be reported. +cd "$TMP" +cp -r app twoshapes +cd twoshapes +cat > build.mcpp <<'EOF' +import mcpp; +#include +#include +int main() { + mcpp::provides_pack_format("zap"); + const std::string root = mcpp::manifest_dir(); + + // Ungated: present in both passes, so it is not the distributable. + const std::string stamp = std::string(mcpp::out_dir()) + "/size.stamp"; + mcpp::action s; + s.id = "size-budget"; s.role = "artifact"; + s.arg((root + "/dist.sh").c_str()).arg("stamp").arg(root.c_str()).arg(stamp.c_str()) + .input("${mcpp.target_file:app}").output(stamp.c_str()); + s.submit(); + + if (std::string_view(mcpp::pack_format()) != "zap") return 0; + // Gated, and it names NO staged tree: the program arrives through + // ${mcpp.target_file:...} exactly as an MSI's one File row does. + const std::string out = std::string(mcpp::out_dir()) + "/app.zap"; + mcpp::action a; + a.id = "zap"; a.role = "artifact"; + a.arg((root + "/dist.sh").c_str()).arg("named").arg(root.c_str()).arg(out.c_str()) + .input("${mcpp.target_file:app}").output(out.c_str()); + a.submit(); + return 0; +} +EOF +"$MCPP" pack --format zap > b8.log 2>&1 || { cat b8.log; echo "FAIL: a member that reads no staged tree was refused"; exit 1; } +grep -q "app.zap" b8.log \ + || { cat b8.log; echo "FAIL: the gated action was not reported as the package"; exit 1; } +grep -q "size.stamp" b8.log \ + && { cat b8.log; echo "FAIL: an action predating the request was reported as the package"; exit 1; } +# Both files exist -- the stamp was built, it was simply not the answer. +[ -n "$(find target -name 'size.stamp' 2>/dev/null)" ] \ + || { echo "FAIL: the ungated artifact action did not run at all"; exit 1; } + +echo "PASS: 638_pack_format_dispatch" diff --git a/tests/matrix/expected.tsv b/tests/matrix/expected.tsv index 8fb38b869..0badcac48 100644 --- a/tests/matrix/expected.tsv +++ b/tests/matrix/expected.tsv @@ -46,10 +46,14 @@ # 那台机器。 # # 各台格数不同,而这是事实不是遗漏: -# linux-x86_64 40 gcc + llvm × 12 目标(payload 24 / graph 16) -# linux-aarch64 16 只有 musl-gcc —— llvm 在非 x86_64 Linux 上被显式延缓 -# macos-arm64 20 只有 llvm × 12 目标(payload 12 / graph 8) -# windows-x86_64 40 llvm + msvc@system +# linux-x86_64 74 gcc + llvm(payload 50 / graph 24) +# linux-aarch64 33 只有 musl-gcc —— llvm 在非 x86_64 Linux 上被显式延缓 +# macos-arm64 37 只有 llvm(payload 25 / graph 12) +# windows-x86_64 74 llvm + msvc@system +# +# 这几个数字是**声明**,和表里的行一样参与比对(compare.sh 先比总数再比每一格), +# 所以加了目标行就必须同时改它们。2026-09-11 加入方案 §3 的四个平台行时,四台 +# 宿主各 +12 格:两种体系 × 该宿主声明的编译器 × 4 个目标。 graph linux-aarch64 aarch64-linux-gnu gcc@16.1.0 - - - - - unsupported tier-planned graph linux-aarch64 aarch64-linux-musl gcc@16.1.0 - - - - - unsupported layer-requirement graph linux-aarch64 riscv64-linux-musl gcc@16.1.0 - - - - - unsupported tier-planned @@ -220,3 +224,65 @@ payload windows-x86_64 x86_64-windows-msvc llvm@22.1.8 x86_64-pc-windows-msvc no payload windows-x86_64 x86_64-windows-msvc msvc@system x86_64-pc-windows-msvc none msvc(payload) (payload) - ok none payload windows-x86_64 x86_64-windows-musl llvm@22.1.8 - - - - - unsupported host-cannot-serve payload windows-x86_64 x86_64-windows-musl msvc@system - - - - - unsupported capability-pin + +# ── 方案 §3 的三个平台:词表里有,还没有任何东西接线 ────────────────────── +# +# 四行全部 `planned`,于是全部十二格(两种体系 × 四台宿主的编译器轴)都是 +# `unsupported / tier-planned`。这些格子不是占位:它们断言的是**拒绝的形状**—— +# 一个 planned 目标必须报「词表里有这一行,还没有接线」,而不是 `unknown target` +# (那是假的),也不是一次解析通过却什么都没建出来的构建(那更糟)。 +# +# 每一行接上线的时候,这里对应的那一格会从 tier-planned 变成别的东西,而这张表 +# 会因此变红 —— 那正是它该做的事。缺的东西每一处都是**载荷**,不是引擎: +# aarch64-linux-android / x86_64-linux-android xim:android-ndk +# aarch64-ios iPhoneOS SDK(先要一次许可判断) +# wasm32-emscripten xim:emsdk,以及 #597 的目标模型 +# +graph linux-aarch64 aarch64-linux-android gcc@16.1.0 - - - - - unsupported tier-planned +graph linux-x86_64 aarch64-linux-android gcc@16.1.0 - - - - - unsupported tier-planned +graph linux-x86_64 aarch64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +graph macos-arm64 aarch64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +graph windows-x86_64 aarch64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +graph windows-x86_64 aarch64-linux-android msvc@system - - - - - unsupported tier-planned +payload linux-aarch64 aarch64-linux-android gcc@16.1.0 - - - - - unsupported tier-planned +payload linux-x86_64 aarch64-linux-android gcc@16.1.0 - - - - - unsupported tier-planned +payload linux-x86_64 aarch64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +payload macos-arm64 aarch64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +payload windows-x86_64 aarch64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +payload windows-x86_64 aarch64-linux-android msvc@system - - - - - unsupported tier-planned +graph linux-aarch64 x86_64-linux-android gcc@16.1.0 - - - - - unsupported tier-planned +graph linux-x86_64 x86_64-linux-android gcc@16.1.0 - - - - - unsupported tier-planned +graph linux-x86_64 x86_64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +graph macos-arm64 x86_64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +graph windows-x86_64 x86_64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +graph windows-x86_64 x86_64-linux-android msvc@system - - - - - unsupported tier-planned +payload linux-aarch64 x86_64-linux-android gcc@16.1.0 - - - - - unsupported tier-planned +payload linux-x86_64 x86_64-linux-android gcc@16.1.0 - - - - - unsupported tier-planned +payload linux-x86_64 x86_64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +payload macos-arm64 x86_64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +payload windows-x86_64 x86_64-linux-android llvm@22.1.8 - - - - - unsupported tier-planned +payload windows-x86_64 x86_64-linux-android msvc@system - - - - - unsupported tier-planned +graph linux-aarch64 aarch64-ios gcc@16.1.0 - - - - - unsupported tier-planned +graph linux-x86_64 aarch64-ios gcc@16.1.0 - - - - - unsupported tier-planned +graph linux-x86_64 aarch64-ios llvm@22.1.8 - - - - - unsupported tier-planned +graph macos-arm64 aarch64-ios llvm@22.1.8 - - - - - unsupported tier-planned +graph windows-x86_64 aarch64-ios llvm@22.1.8 - - - - - unsupported tier-planned +graph windows-x86_64 aarch64-ios msvc@system - - - - - unsupported tier-planned +payload linux-aarch64 aarch64-ios gcc@16.1.0 - - - - - unsupported tier-planned +payload linux-x86_64 aarch64-ios gcc@16.1.0 - - - - - unsupported tier-planned +payload linux-x86_64 aarch64-ios llvm@22.1.8 - - - - - unsupported tier-planned +payload macos-arm64 aarch64-ios llvm@22.1.8 - - - - - unsupported tier-planned +payload windows-x86_64 aarch64-ios llvm@22.1.8 - - - - - unsupported tier-planned +payload windows-x86_64 aarch64-ios msvc@system - - - - - unsupported tier-planned +graph linux-aarch64 wasm32-emscripten gcc@16.1.0 - - - - - unsupported tier-planned +graph linux-x86_64 wasm32-emscripten gcc@16.1.0 - - - - - unsupported tier-planned +graph linux-x86_64 wasm32-emscripten llvm@22.1.8 - - - - - unsupported tier-planned +graph macos-arm64 wasm32-emscripten llvm@22.1.8 - - - - - unsupported tier-planned +graph windows-x86_64 wasm32-emscripten llvm@22.1.8 - - - - - unsupported tier-planned +graph windows-x86_64 wasm32-emscripten msvc@system - - - - - unsupported tier-planned +payload linux-aarch64 wasm32-emscripten gcc@16.1.0 - - - - - unsupported tier-planned +payload linux-x86_64 wasm32-emscripten gcc@16.1.0 - - - - - unsupported tier-planned +payload linux-x86_64 wasm32-emscripten llvm@22.1.8 - - - - - unsupported tier-planned +payload macos-arm64 wasm32-emscripten llvm@22.1.8 - - - - - unsupported tier-planned +payload windows-x86_64 wasm32-emscripten llvm@22.1.8 - - - - - unsupported tier-planned +payload windows-x86_64 wasm32-emscripten msvc@system - - - - - unsupported tier-planned diff --git a/tests/unit/test_build_directives.cpp b/tests/unit/test_build_directives.cpp index 3943d9488..90f22ea86 100644 --- a/tests/unit/test_build_directives.cpp +++ b/tests/unit/test_build_directives.cpp @@ -208,7 +208,8 @@ TEST(BuildDirectives, SerializeDeserializeRoundTrip) { "mcpp:include-dir=inc\n" "mcpp:include-dir-after=after\n" "mcpp:fact=widget.driver=1.2\n" - "mcpp:floor=widget.driver >= 1.0\n"); + "mcpp:floor=widget.driver >= 1.0\n" + "mcpp:pack-format=appimage\n"); std::ostringstream os; dirs::serialize(os, d); @@ -258,6 +259,44 @@ TEST(BuildDirectives, FactsAndFloorsAreClaimsThatFoldIntoRuntimeDeclarations) { EXPECT_EQ(m.runtimeConfig.requirements[0].phase, "build"); } +TEST(BuildDirectives, APackFormatIsANameCarriedAndNotInterpreted) { + auto d = parse("mcpp:pack-format=appimage\n" + "mcpp:pack-format=msi\n"); + EXPECT_EQ(d.at(dirs::Slot::PackFormats), + (std::vector{"appimage", "msi"})); + + mcpp::manifest::Manifest m; + dirs::apply(m, d); + EXPECT_EQ(m.buildConfig.packFormats, + (std::vector{"appimage", "msi"})); + // The engine holds the DISPATCH and no format, so a name reaches neither + // the compile line nor the link line. A row that leaked into either would + // put dpkg's or WiX's vocabulary on a command line. + EXPECT_TRUE(m.buildConfig.cflags.empty()); + EXPECT_TRUE(m.buildConfig.cxxflags.empty()); + EXPECT_TRUE(m.buildConfig.ldflags.empty()); + EXPECT_TRUE(m.buildConfig.sources.empty()); + EXPECT_TRUE(m.buildConfig.actions.empty()); +} + +TEST(BuildDirectives, APackFormatDeclarationIsPersisted) { + // THE HALF THAT WOULD OTHERWISE BE MISSED. A build program's result is + // cached and a hit does not re-run it, and the pass that READS this set is + // `mcpp pack`, which is never a project's first build. A declaration that + // was not persisted would therefore be present exactly once and absent + // every time it mattered, and `--format ` would refuse naming + // nothing. + // + // `SerializeDeserializeRoundTrip` above asserts the round trip for every + // tagged row, so this only has to hold the field that puts the row in that + // set -- which is the field a new row is most likely to be added without. + const auto* def = dirs::find_by_wire("pack-format"); + ASSERT_NE(def, nullptr); + EXPECT_FALSE(def->tag.empty()) << "a tag-less row is not replayed on a cache hit"; + EXPECT_EQ(def->slot, dirs::Slot::PackFormats); + EXPECT_EQ(def->scope, dirs::Scope::Claim); +} + TEST(BuildDirectives, AClaimReachesNeitherCompileNorLink) { auto d = parse("mcpp:fact=widget.driver=1.2\n" "mcpp:floor=widget.driver >= 2.0\n"); diff --git a/tests/unit/test_distribution.cpp b/tests/unit/test_distribution.cpp index 83ede0dc4..db33297ae 100644 --- a/tests/unit/test_distribution.cpp +++ b/tests/unit/test_distribution.cpp @@ -645,6 +645,13 @@ TEST(Distribution, FormatIsTakenFromTheTargetAndNotTheFallback) { for (auto fb : {dist::Format::Elf, dist::Format::MachO, dist::Format::Pe}) { EXPECT_EQ(dist::format_for("aarch64-macos", fb), dist::Format::MachO); EXPECT_EQ(dist::format_for("x86_64-macos", fb), dist::Format::MachO); + // iOS shares macOS's Mach-O format. Before `format_for` asked + // `is_mach_o()` instead of `os == "macos"`, this triple matched none + // of the branches inside the parsed-triple block and fell through to + // `fb` itself -- so the assertion below would have failed for two of + // the three fallbacks in this loop, the exact "decided by the + // machine doing the building" defect this test exists to catch. + EXPECT_EQ(dist::format_for("aarch64-ios", fb), dist::Format::MachO); EXPECT_EQ(dist::format_for("x86_64-windows-gnu", fb), dist::Format::Pe); EXPECT_EQ(dist::format_for("x86_64-linux-gnu", fb), dist::Format::Elf); EXPECT_EQ(dist::format_for("aarch64-linux-musl", fb), dist::Format::Elf); diff --git a/tests/unit/test_graph_shape.cpp b/tests/unit/test_graph_shape.cpp index 40a75ae9d..88c63f554 100644 --- a/tests/unit/test_graph_shape.cpp +++ b/tests/unit/test_graph_shape.cpp @@ -3,10 +3,21 @@ import std; import mcpp.build.graph_shape; // The header line build.ninja carries is what the fast paths read before any -// plan exists. Three facts ride it -- the graph's shape, the module-edge -// schedule, and (2026.9.5.3+) whether a `--accel` / `--no-accel` override -// chose the device variant -- and the fast paths decline a graph that says -// anything other than "plain build, manifest's own variant". +// plan exists. Four facts ride it -- the graph's shape, the module-edge +// schedule, (2026.9.5.3+) whether a `--accel` / `--no-accel` override chose +// the device variant, and (2026.9.11.1+) which distribution format the graph +// was generated for -- and the fast paths decline a graph that says anything +// other than "plain build, manifest's own variant, no distribution edge". +// +// THE FOURTH FIELD IS VERIFIED HERE AND NOT ONLY END TO END, deliberately. +// `mcpp pack --format ` prepares twice and the second pass writes its +// graph into the SAME fingerprint directory a plain build uses, because the +// format is not in the fingerprint. Measured on 2026-09-11: a plain build +// after that pass regenerates the graph even with this field ignored, so some +// earlier freshness condition already declines -- which means an end-to-end +// assertion would pass whether or not the field works, and would keep passing +// if the field were deleted. A read-side invariant only has to hold once +// (see the module header); this is where it is held. namespace { @@ -26,11 +37,16 @@ std::filesystem::path write_graph(const std::string& first) { } // namespace -TEST(GraphShape, TheHeaderNamesShapeScheduleAndSelection) { +TEST(GraphShape, TheHeaderNamesShapeScheduleSelectionAndFormat) { EXPECT_EQ(mcpp::build::header_line(mcpp::build::GraphShape::Normal, "none", false), - "# mcpp:graph=normal;schedule=none;accel=default"); + "# mcpp:graph=normal;schedule=none;accel=default;dist=none"); EXPECT_EQ(mcpp::build::header_line(mcpp::build::GraphShape::WithTests, "two-phase", true), - "# mcpp:graph=test;schedule=two-phase;accel=override"); + "# mcpp:graph=test;schedule=two-phase;accel=override;dist=none"); + // An empty format reads as "none" rather than as an empty field: the value + // has to be a word, because `read_pack_format` returning "" already means + // "this file predates the field", and the two must not collide. + EXPECT_EQ(mcpp::build::header_line(mcpp::build::GraphShape::Normal, "none", false, "appimage"), + "# mcpp:graph=normal;schedule=none;accel=default;dist=appimage"); } TEST(GraphShape, OnlyAPlainGraphWithTheManifestsVariantIsReplayed) { @@ -40,6 +56,19 @@ TEST(GraphShape, OnlyAPlainGraphWithTheManifestsVariantIsReplayed) { EXPECT_FALSE(is_plain_build_graph(write_graph(header_line(GraphShape::Normal, "none", true)))); // The test-shaped graph was already refused. EXPECT_FALSE(is_plain_build_graph(write_graph(header_line(GraphShape::WithTests, "none", false)))); + // A DISTRIBUTION EDGE IS NOT PART OF A PLAIN BUILD. `mcpp pack --format + // appimage` makes a build program submit an artifact action consuming the + // staged tree; replaying that graph for a plain build would produce a + // distributable as a side effect of `mcpp build`, from a staged tree that + // is no longer guaranteed to describe this build. + EXPECT_FALSE(is_plain_build_graph( + write_graph(header_line(GraphShape::Normal, "none", false, "appimage")))); + EXPECT_EQ(read_pack_format( + write_graph(header_line(GraphShape::Normal, "none", false, "appimage"))), + "appimage"); + EXPECT_EQ(read_pack_format( + write_graph(header_line(GraphShape::Normal, "none", false))), + "none"); } TEST(GraphShape, AGraphThatPredatesTheFieldIsAMiss) { @@ -50,5 +79,15 @@ TEST(GraphShape, AGraphThatPredatesTheFieldIsAMiss) { auto p = write_graph("# mcpp:graph=normal;schedule=none"); EXPECT_EQ(read_shape(p), GraphShape::Normal); EXPECT_EQ(read_accel_selection(p), ""); + EXPECT_EQ(read_pack_format(p), ""); EXPECT_FALSE(is_plain_build_graph(p)); + + // Written by a 2026.9.10.2 mcpp: shape, schedule and selection, no + // distribution field. Same rule one field later -- absent is a miss, and + // must not be read as "none", or the very first build after an upgrade + // would replay a graph this binary cannot describe. + auto q = write_graph("# mcpp:graph=normal;schedule=none;accel=default"); + EXPECT_EQ(read_accel_selection(q), "default"); + EXPECT_EQ(read_pack_format(q), ""); + EXPECT_FALSE(is_plain_build_graph(q)); } diff --git a/tests/unit/test_hostflags.cpp b/tests/unit/test_hostflags.cpp index 34a9d7188..07a214cf4 100644 --- a/tests/unit/test_hostflags.cpp +++ b/tests/unit/test_hostflags.cpp @@ -342,6 +342,18 @@ TEST(GraphRuntimeFlags, MachOTakesEmulatedTlsAndHiddenVisibilityButNotDwarf) { EXPECT_TRUE(has(f, "-fvisibility-inlines-hidden")); } +// iOS shares macOS's object format (Mach-O, ld64), so it must share this +// exact set of flags. Before `is_mach_o()` replaced `os == "macos"` here, an +// iOS triple matched neither the PE nor the macOS branch and this function +// silently returned no flags at all for it. +TEST(GraphRuntimeFlags, IosTakesTheSameFlagsAsMacOS) { + auto f = mcpp::toolchain::graph_runtime_compile_flags(graph_tc("aarch64-ios")); + EXPECT_FALSE(has(f, "-fdwarf-exceptions")); + EXPECT_TRUE(has(f, "-femulated-tls")); + EXPECT_TRUE(has(f, "-fvisibility=hidden")); + EXPECT_TRUE(has(f, "-fvisibility-inlines-hidden")); +} + // ELF takes NONE of them, and that is a decision rather than an omission. // There a `thread_local` is a fixed offset from the thread pointer, which the // C library establishes itself; adding the flag would work, cost an diff --git a/tests/unit/test_loader_contract.cpp b/tests/unit/test_loader_contract.cpp index 52643e75a..1862b219d 100644 --- a/tests/unit/test_loader_contract.cpp +++ b/tests/unit/test_loader_contract.cpp @@ -60,9 +60,18 @@ TEST(GraphShape, UnlabelledOrUnknownGraphIsNeverPlain) { }; EXPECT_TRUE(is_plain_build_graph( - write("normal.ninja", "# banner\n# mcpp:graph=normal;schedule=none;accel=default\nrule x\n"))); + write("normal.ninja", "# banner\n# mcpp:graph=normal;schedule=none;accel=default;dist=none\nrule x\n"))); EXPECT_FALSE(is_plain_build_graph( - write("test.ninja", "# banner\n# mcpp:graph=test;schedule=none;accel=default\nrule x\n"))); + write("test.ninja", "# banner\n# mcpp:graph=test;schedule=none;accel=default;dist=none\nrule x\n"))); + + // A plain-shaped graph that `mcpp pack --format ` wrote (2026.9.11.1+): + // it carries an artifact edge consuming a staged tree, which a plain build + // must not have. These lines are spelled by hand here rather than through + // `header_line`, which is the point of this copy -- a reader of build.ninja + // sees the text, and a field added to the producer without being added to + // the reader would still pass a test that only compared the two. + EXPECT_FALSE(is_plain_build_graph( + write("dist.ninja", "# mcpp:graph=normal;schedule=none;accel=default;dist=appimage\n"))); // A plain-shaped graph an `--accel` / `--no-accel` build wrote (2026.9.5.3+): // the variant a flag chose is not the variant a plain build produces. @@ -74,6 +83,14 @@ TEST(GraphShape, UnlabelledOrUnknownGraphIsNeverPlain) { EXPECT_FALSE(is_plain_build_graph( write("no-selection.ninja", "# banner\n# mcpp:graph=normal\nrule x\n"))); + // A graph from 2026.9.10.2: shape, schedule and selection, no distribution + // field. Same rule one field later. This assertion is what caught the + // second copy of these lines when the field was added -- the line above it + // was the CURRENT spelling in one file and became the LEGACY spelling in + // both, and only a test that spells it out could say so. + EXPECT_FALSE(is_plain_build_graph( + write("no-dist.ninja", "# banner\n# mcpp:graph=normal;schedule=none;accel=default\nrule x\n"))); + // A build.ninja from before the marker existed. It MUST read as a miss: // treating it as plain is precisely the replay #407 is about. EXPECT_FALSE(is_plain_build_graph( diff --git a/tests/unit/test_pack_relocate.cpp b/tests/unit/test_pack_relocate.cpp index 518819287..c1146ea70 100644 --- a/tests/unit/test_pack_relocate.cpp +++ b/tests/unit/test_pack_relocate.cpp @@ -325,6 +325,8 @@ TEST(PackStrip, WhetherStrippingAppliesIsAskedOfTheTargetNotTheCompiler) { EXPECT_FALSE(mcpp::pack::debug_info_is_in_band("x86_64-windows-msvc")); EXPECT_FALSE(mcpp::pack::debug_info_is_in_band("aarch64-macos")); EXPECT_FALSE(mcpp::pack::debug_info_is_in_band("x86_64-macos")); + // iOS carries the same Mach-O debug map + out-of-band .dSYM as macOS. + EXPECT_FALSE(mcpp::pack::debug_info_is_in_band("aarch64-ios")); // Segment-wise, not substring: mcpp has been bitten by a triple predicate // that answered on a substring before. EXPECT_TRUE(mcpp::pack::debug_info_is_in_band("macos64-linux-gnu")); diff --git a/tests/unit/test_pack_stage_tree.cpp b/tests/unit/test_pack_stage_tree.cpp new file mode 100644 index 000000000..5bf709694 --- /dev/null +++ b/tests/unit/test_pack_stage_tree.cpp @@ -0,0 +1,141 @@ +#include + +import std; +import mcpp.pack.stage_tree; + +// The staged tree is what `mcpp pack` computes and, until `${mcpp.stage_dir}`, +// then threw away. Two things about it are the engine's contract with a +// distribution member, and both are asserted here rather than end to end, +// because the end-to-end criterion for either is "the distributable is +// rebuilt", which a wrong answer also satisfies. + +namespace { + +struct Tmp { + std::filesystem::path path; + Tmp() { + path = std::filesystem::temp_directory_path() + / std::format("mcpp_stage_tree_{}", std::random_device{}()); + std::filesystem::create_directories(path); + } + ~Tmp() { + std::error_code ec; + std::filesystem::remove_all(path, ec); + } +}; + +void write_file(const std::filesystem::path& p, std::string_view body) { + std::filesystem::create_directories(p.parent_path()); + std::ofstream os(p, std::ios::binary); + os << body; +} + +std::string read_file(const std::filesystem::path& p) { + std::ifstream is(p, std::ios::binary); + return std::string{std::istreambuf_iterator(is), {}}; +} + +} // namespace + +TEST(PackStageTree, TheManifestIsASiblingAndNeverAMember) { + Tmp t; + auto stage = t.path / "app-1.0.0-x86_64-linux-gnu"; + std::filesystem::create_directories(stage); + + auto manifest = mcpp::pack::stage_manifest_path(stage); + EXPECT_EQ(manifest.parent_path(), stage.parent_path()); + // A file INSIDE the tree would be collected by every format that packages + // the directory wholesale, and would then ship inside the user's + // installer. Asserted as a path relationship because that is the property, + // and a spelling change that broke it would otherwise only show up as an + // extra file in a released package. + EXPECT_FALSE(manifest.string().starts_with(stage.string() + "/")); + EXPECT_FALSE(manifest.string().starts_with(stage.string() + "\\")); + + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage)); + std::error_code ec; + for (auto const& e : std::filesystem::recursive_directory_iterator(stage, ec)) + FAIL() << "the manifest landed inside the tree: " << e.path().string(); +} + +TEST(PackStageTree, TheManifestChangesWhenTheStagedSetDoes) { + Tmp t; + auto stage = t.path / "app"; + write_file(stage / "bin" / "app", "0123456789"); + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage)); + const auto first = read_file(mcpp::pack::stage_manifest_path(stage)); + EXPECT_NE(first.find("bin/app"), std::string::npos); + EXPECT_NE(first.find("10 "), std::string::npos); + + // A DEPENDENCY'S SHARED LIBRARY JOINING THE CLOSURE. This is the case the + // manifest exists for: the program's own bytes need not have changed, so an + // edge that depended only on the link output would report the previous + // distributable as up to date. + write_file(stage / "lib" / "libdep.so.1", "xx"); + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage)); + const auto second = read_file(mcpp::pack::stage_manifest_path(stage)); + EXPECT_NE(first, second); + EXPECT_NE(second.find("lib/libdep.so.1"), std::string::npos); + + // A staged file whose LENGTH changed, with no entry added or removed. + write_file(stage / "bin" / "app", "0123456789abcdef"); + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage)); + EXPECT_NE(read_file(mcpp::pack::stage_manifest_path(stage)), second); +} + +TEST(PackStageTree, StagingTheSameTreeTwiceLeavesTheManifestUntouched) { + Tmp t; + auto stage = t.path / "app"; + write_file(stage / "bin" / "app", "same"); + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage)); + auto manifest = mcpp::pack::stage_manifest_path(stage); + const auto before = std::filesystem::last_write_time(manifest); + + // Rewriting identical bytes would move the mtime, and a moved mtime on an + // input is indistinguishable from a changed input: a pack that staged the + // same tree twice would rebuild the distributable both times. Same rule + // `mcpp.build.stage` states at length for a staged BMI. + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage)); + EXPECT_EQ(std::filesystem::last_write_time(manifest), before); +} + +TEST(PackStageTree, TheOrderOfADirectoryWalkIsNotAPromise) { + // Two trees with the same contents produce the same bytes. Without the + // sort, an iteration order that differed between runs would make the dist + // edge dirty on every pack for no reason -- which reads as "packaging is + // slow" rather than as a defect. + Tmp a, b; + for (auto const& root : {a.path, b.path}) { + write_file(root / "s" / "bin" / "app", "aa"); + write_file(root / "s" / "lib" / "z.so", "bbb"); + write_file(root / "s" / "share" / "doc" / "readme", "c"); + ASSERT_TRUE(mcpp::pack::write_stage_manifest(root / "s")); + } + EXPECT_EQ(read_file(mcpp::pack::stage_manifest_path(a.path / "s")), + read_file(mcpp::pack::stage_manifest_path(b.path / "s"))); +} + +TEST(PackStageTree, TheEngineOwnsExactlyTwoFormatNames) { + // `tar` and `dir` are the archive shapes `mcpp pack` owns; every other + // value of `--format` is a name a package provides. The list lives beside + // the staged tree because two layers need the same answer -- the parser, + // which decides whether a value is a built-in or a dispatch, and the + // declaration check, which refuses a package that claims one of them and + // would therefore be silently unreachable. + EXPECT_TRUE(mcpp::pack::is_builtin_pack_format("tar")); + EXPECT_TRUE(mcpp::pack::is_builtin_pack_format("dir")); + EXPECT_FALSE(mcpp::pack::is_builtin_pack_format("appimage")); + EXPECT_FALSE(mcpp::pack::is_builtin_pack_format("msi")); + EXPECT_FALSE(mcpp::pack::is_builtin_pack_format("")); + EXPECT_EQ(mcpp::pack::kBuiltinPackFormats.size(), 2u); +} + +TEST(PackStageTree, AMissingTreeIsRefusedRatherThanDescribedAsEmpty) { + Tmp t; + // An empty manifest for a directory that does not exist would say "nothing + // is staged", which is what a correct pack of an empty bundle also says. + // The two must not be spelled alike. + EXPECT_FALSE(mcpp::pack::write_stage_manifest(t.path / "never-staged")); + EXPECT_FALSE(std::filesystem::exists( + mcpp::pack::stage_manifest_path(t.path / "never-staged"))); +} diff --git a/tests/unit/test_toolchain_triple.cpp b/tests/unit/test_toolchain_triple.cpp index 3e609f611..3302b3a89 100644 --- a/tests/unit/test_toolchain_triple.cpp +++ b/tests/unit/test_toolchain_triple.cpp @@ -515,3 +515,200 @@ TEST(Triple, EveryTableRowIsItsOwnCanonicalForm) { << info.canonical << " does not round-trip"; } } + +// ── The object-format axis, and the three platforms it exists for ─────────── +// +// The binary format used to be re-derived from `os` at every site that needed +// it, which is affordable only while the answer has two values. These hold the +// single derivation, because a site that misses a third value does not fail -- +// it silently answers ELF, which is what every `else` branch in the tree +// assumes. + +TEST(Triple, TheObjectFormatIsOneAnswerAndNotADerivation) { + EXPECT_EQ(parse("x86_64-linux-gnu")->object_format(), ObjectFormat::Elf); + EXPECT_EQ(parse("x86_64-linux-musl")->object_format(), ObjectFormat::Elf); + EXPECT_EQ(parse("aarch64-linux-android")->object_format(), ObjectFormat::Elf); + EXPECT_EQ(parse("riscv64-none-elf")->object_format(), ObjectFormat::Elf); + EXPECT_EQ(parse("x86_64-windows-gnu")->object_format(), ObjectFormat::Pe); + EXPECT_EQ(parse("x86_64-windows-msvc")->object_format(), ObjectFormat::Pe); + EXPECT_EQ(parse("aarch64-macos")->object_format(), ObjectFormat::MachO); + EXPECT_EQ(parse("aarch64-ios")->object_format(), ObjectFormat::MachO); + EXPECT_EQ(parse("wasm32-emscripten")->object_format(), ObjectFormat::Wasm); +} + +TEST(Triple, TheFormatQuestionIsNotTheOperatingSystemQuestion) { + // The two axes were conflated before `ObjectFormat` existed, and merging + // them is the mistake it replaces: a bare-metal image is ELF with no OS, + // and a wasm module has an OS-like layer and is not ELF. + auto bare = parse("riscv64-none-elf"); + EXPECT_TRUE(bare->is_freestanding()); + EXPECT_EQ(bare->object_format(), ObjectFormat::Elf); + + auto web = parse("wasm32-emscripten"); + EXPECT_FALSE(web->is_freestanding()); + EXPECT_TRUE(web->is_wasm()); +} + +TEST(Triple, IsPeAndIsMachOReadTheSingleAnswer) { + EXPECT_TRUE (parse("x86_64-windows-gnu")->is_pe()); + EXPECT_FALSE(parse("x86_64-windows-gnu")->is_mach_o()); + EXPECT_TRUE (parse("aarch64-ios")->is_mach_o()); + EXPECT_FALSE(parse("aarch64-ios")->is_pe()); + // The row that used to answer this by `os == "windows"` and would have + // answered ELF for wasm. + EXPECT_FALSE(parse("wasm32-emscripten")->is_pe()); + EXPECT_FALSE(parse("wasm32-emscripten")->is_mach_o()); +} + +TEST(Triple, AndroidIsAnEnvOnALinuxOs) { + auto t = parse("aarch64-linux-android"); + ASSERT_TRUE(t.has_value()); + EXPECT_EQ(t->arch, "aarch64"); + // THE PLACEMENT IS THE MODELLING DECISION. The kernel is Linux, so ELF, + // the `unix` family and `nasm -f elf64` are all already right; an + // `os = "android"` would have made every one of them wrong by default. + EXPECT_EQ(t->os, "linux"); + EXPECT_EQ(t->env, "android"); + EXPECT_TRUE(t->is_android()); + EXPECT_EQ(t->family(), "unix"); + EXPECT_EQ(t->str(), "aarch64-linux-android"); + EXPECT_EQ(t->llvm_triple(), "aarch64-unknown-linux-android"); + + // `androideabi` is the 32-bit ARM spelling of the same env: the EABI half + // is the calling convention, which the arch segment already carries. + auto eabi = parse("armv7a-linux-androideabi"); + ASSERT_TRUE(eabi.has_value()); + EXPECT_EQ(eabi->env, "android"); + + // The env fill must not reach an Android request. `x86_64-linux` is still + // `gnu`; `x86_64-linux-android` is not. + EXPECT_EQ(parse("x86_64-linux")->env, "gnu"); + EXPECT_EQ(parse("x86_64-linux-android")->env, "android"); +} + +TEST(Triple, IosIsAppleWithoutBeingMacos) { + auto t = parse("aarch64-ios"); + ASSERT_TRUE(t.has_value()); + EXPECT_EQ(t->os, "ios"); + EXPECT_TRUE(t->env.empty()); + EXPECT_EQ(t->family(), "unix"); + EXPECT_EQ(t->str(), "aarch64-ios"); + // Apple's own spelling of the architecture, as the macOS branch already + // produces. No deployment target is baked in: that flag belongs to the + // layer that owns the SDK, and a default here would be a second answer. + EXPECT_EQ(t->llvm_triple(), "arm64-apple-ios"); + + // A SITE THAT MEANS "APPLE" AND ASKS "macOS" GETS iOS WRONG IN THE + // DIRECTION THAT STILL LINKS, which is why the predicate exists. + EXPECT_TRUE(parse("aarch64-macos")->is_apple()); + EXPECT_TRUE(parse("aarch64-ios")->is_apple()); + EXPECT_FALSE(parse("aarch64-linux-musl")->is_apple()); + + // An effective triple carries the deployment target on this segment. + auto eff = parse("arm64-apple-ios17.0"); + ASSERT_TRUE(eff.has_value()); + EXPECT_EQ(eff->os, "ios"); +} + +TEST(Triple, EmscriptenIsAnOsSegmentAndNotAnEnv) { + auto t = parse("wasm32-emscripten"); + ASSERT_TRUE(t.has_value()); + EXPECT_EQ(t->arch, "wasm32"); + // It names the platform layer a module is compiled against -- the POSIX + // emulation, the filesystem shim, the main loop -- which is the kind of + // thing `linux` names and not the kind of thing `musl` names. + EXPECT_EQ(t->os, "emscripten"); + EXPECT_TRUE(t->env.empty()); + EXPECT_EQ(t->str(), "wasm32-emscripten"); + EXPECT_EQ(t->llvm_triple(), "wasm32-unknown-emscripten"); + // `unix` on the test the predicate actually applies -- what API surface a + // source may assume -- rather than on a claim about wasm. + EXPECT_EQ(t->family(), "unix"); + // NASM is x86-family by construction and must decline rather than choose. + EXPECT_FALSE(t->nasm_format().has_value()); +} + +TEST(Triple, TheThreePlatformsAreRegisteredAndPlanned) { + // A row here is what every layer above waits on: the `.apk` step, the + // `.app` step, the `.html`+`.wasm` step, the runner and the signing all + // attach to a triple, and a package cannot add one. + // + // `planned` is a REFUSAL and not a gap -- the tier gate answers + // `tier-planned` naming the row, rather than `unknown target`, which would + // be false, or a build that resolves and produces nothing, which is worse. + for (auto name : {"aarch64-linux-android", "x86_64-linux-android", + "aarch64-ios", "wasm32-emscripten"}) { + auto t = parse(name); + ASSERT_TRUE(t.has_value()) << name; + EXPECT_EQ(t->str(), name); + auto* info = find_known_target(*t); + ASSERT_NE(info, nullptr) << name; + EXPECT_EQ(info->tier, "planned") << name; + // No pin and no sysroot: what each row still needs is a PAYLOAD, and + // naming a compiler that cannot serve the target would be a claim the + // row cannot keep. + EXPECT_TRUE(info->pin.empty()) << name; + EXPECT_TRUE(info->sysroot.empty()) << name; + } +} + +TEST(Triple, TheCanonicalSpellingIsNotSEARCHABLEForAVENDORNAME) { + // WHY A SUBSTRING TEST ON THE CANONICAL TRIPLE IS WRONG, stated as a fact + // about the vocabulary rather than as a comment somewhere else. + // + // Two sites derived the object format by looking for "apple" / "darwin" / + // "windows" / "mingw" in `plan.toolchain.targetTriple`. That string is + // mcpp's CANONICAL spelling, and `aarch64-macos` contains none of those + // words -- so an explicit `--target aarch64-macos`, a `verified` row, was + // recorded and linked as ELF. A NATIVE macOS build was right by a + // different branch (an empty triple), which is why the two paths through + // one function disagreed and only the exercised one was correct. + // + // The words appear in the LLVM spelling, which is a different string and + // the reason the mistake is easy to make: + // + // aarch64-macos -> arm64-apple-macos14.0 + // ^ the identity ^ what clang is given + for (auto name : {"aarch64-macos", "x86_64-macos", "aarch64-ios"}) { + auto t = parse(name); + ASSERT_TRUE(t.has_value()) << name; + const std::string canonical = t->str(); + EXPECT_EQ(canonical.find("apple"), std::string::npos) << canonical; + EXPECT_EQ(canonical.find("darwin"), std::string::npos) << canonical; + // And the format is right anyway, because it is asked of the fields. + EXPECT_EQ(t->object_format(), ObjectFormat::MachO) << canonical; + // The LLVM spelling is where the vendor name lives. + EXPECT_NE(t->llvm_triple().find("apple"), std::string::npos) + << t->llvm_triple(); + } + // The one family the substring test got right, and only by luck: the + // canonical spelling happens to carry the OS name. + EXPECT_NE(std::string(parse("x86_64-windows-gnu")->str()).find("windows"), + std::string::npos); +} + +TEST(Triple, EveryKnownRowHasAnObjectFormatAndNoneFallsThrough) { + // THE DENOMINATOR IS THE TABLE. A row added without an answer here would + // otherwise be covered by a test whose name says every row is -- and the + // answer it would get is ELF, because ELF is what every `else` branch in + // the tree assumes. + std::size_t elf = 0, macho = 0, pe = 0, wasm = 0; + for (auto const& row : known_targets()) { + auto t = parse(row.canonical); + ASSERT_TRUE(t.has_value()) << row.canonical; + EXPECT_EQ(t->str(), row.canonical) << "a row that is not its own canonical form"; + switch (t->object_format()) { + case ObjectFormat::Elf: ++elf; break; + case ObjectFormat::MachO: ++macho; break; + case ObjectFormat::Pe: ++pe; break; + case ObjectFormat::Wasm: ++wasm; break; + } + } + // Each format has at least one row, which is what makes the axis worth + // having: a fourth value with no row would be an enum nothing produces. + EXPECT_GT(elf, 0u); + EXPECT_GT(macho, 0u); + EXPECT_GT(pe, 0u); + EXPECT_EQ(wasm, 1u) << "wasm32-emscripten is the only wasm row today"; + EXPECT_EQ(elf + macho + pe + wasm, known_targets().size()); +}