No distribution format lives in the engine, and the three platforms a package cannot add (2026.9.11.1) - #605
Merged
Merged
Conversation
…ution format lives in the engine (2026.9.11.1)
`tar` and `dir` answer the same question `msi` and `appimage` answer -- what
shape does the output take -- so they are values of one flag rather than the
beginning of a second one. The split that keeps every other format out of the
engine is: `mcpp pack` owns the mechanism and the one universal format, and
every other format lives in a package that `mcpp pack` dispatches to.
The universal format is what it already produces: an archive that extracts and
runs, universal in the only sense that matters here -- it needs no knowledge of
anyone else's release. Everything past it does. dpkg's control fields,
AppImage's runtime, WiX's schema, Apple's notarisation, Android's signing
scheme: each one bound into the engine couples an mcpp release to a release mcpp
does not control. The project already made this argument for languages, where
Slang is supported without being named in the engine, and a distribution format
has less claim to a name in the engine than a language does.
Three additions, each FORMAT-NEUTRAL, which is the test for whether something
belongs in the engine at all:
- A staged tree an artifact action can consume. `mcpp pack` already computes
one -- the dependency closure after the strip policy, the debug-symbol split
and `include`/`exclude` -- and then compressed it and the directory was
gone, so a `.deb`, an AppImage, a `.app` and an `.msi` each had to rebuild
it. `${mcpp.stage_dir}` exposes it.
- The rest of `[package]` in the build program: `MCPP_PKG_VERSION` /
`_DESCRIPTION` / `_LICENSE` / `_AUTHORS` / `_REPO` and the matching
`mcpp::package_*()`. Every installer states a version; without these a
project restates it in the member's own options, where the copy drifts from
`[package]` with nothing able to detect it.
- `--format` resolving its value through the graph. A package declares with
`mcpp::provides_pack_format("<name>")`; `--format <name>` finds the provider
among the resolved dependencies. The refusal for an unknown value names what
IS available rather than a constant, and arrives before anything is
compiled.
DECLARE UNCONDITIONALLY, SUBMIT CONDITIONALLY. This is the load-bearing rule of
the dispatch and the one a member author is most likely to get wrong, because a
member that gets it wrong still works for whoever wrote it -- they always pass
their own format. The declaration must not be gated, or the engine can never
answer "which formats does this graph provide"; the submission must be, or a
plain `mcpp build` grows an edge it must not have. A format nothing submitted
for is refused by name rather than reported as a pack that produced no package.
`mcpp pack --format <name>` PREPARES TWICE, AND NOTHING IS RE-DERIVED BETWEEN
THE PASSES. An artifact action is a ninja edge and the staged tree is produced
after the link, so the tree cannot be an input of the pass that built it. The
first pass collects declarations and refuses an unknown format; the build and
the staging follow; the second pass sets `pack_format` and `pack_stage_dir` and
builds the submitted edge. Its triple and staged path come from what the first
pass and `make_plan` already answered -- `stagingRoot` is a function of the
resolved triple, and a second derivation of it before prepare is the shape where
two answers agree on every machine the author has.
build.ninja's header line gains a fourth field, `dist=`, and the fast paths
require it to read `none`. The format is deliberately NOT in the fingerprint --
putting it there would cost a full recompile to package an already-built tree --
so the two graphs share a directory, and `target/<triple>/<fp>/build.ninja` is
shared mutable state two fast paths replay. That is the third instance of the
failure `graph=` and `accel=` each already record. The criterion is a unit test
rather than an end-to-end assertion: measured on 2026-09-11, a plain build after
the pack pass regenerates the graph even with the field ignored, so an
end-to-end check would pass whether or not the field works and would keep
passing if it were deleted.
`${mcpp.stage_dir}` REFUSES rather than expanding to nothing, in two places: a
build that is not packaging, and a role other than `artifact`. An empty path is
still a token the command accepts, and the tool then reads the build directory
root, which exists -- so the mistake produces a plausible artifact instead of a
diagnostic. The measured prototype is a valid, empty, 52 KB installer with
nothing said about it.
An action that names the placeholder automatically gains a dependency on
`<staged tree>.stage-manifest` -- a sibling, never a member, so it never travels
inside anyone's installer. The engine adds it because the use implies it:
without it the edge is dirty only when a link output changes, and a closure that
grew a dependency's shared library while the program's own bytes did not would
leave the previous distributable in place, reported as up to date.
Build-program protocol v9 (`mcpp:pack-format=`). The row carries a non-empty
`tag`, so the declaration is replayed from the build program's cache record --
the pass that reads the set is `mcpp pack`, which is never a project's first
build, and an unpersisted declaration would be absent exactly when a user names
a format.
Tests: 6 unit tests for the staged-tree contract, the graph-shape field in both
of its two readers, and the directive row's persistence; one e2e holding the
four properties of the dispatch, each with the wrong answer it excludes.
Docs: `docs/10` for the `--format` axis, `docs/30` for the three-category
taxonomy and the new placeholders and accessors, `docs/31` for the six
constraints a distribution member owes its consumer. Both languages.
`mcpp pack --format bogus` must write NOTHING to stdout and exit 2. That is the
machine-output contract, and 202_machine_output_contract.sh asserts it for
exactly this command, because it is the path a client hits when it probes an
mcpp for a capability -- the most common machine-facing failure, and the one
that used to print to stdout.
Moving the refusal from the CLI parser to after `prepare_build` broke it. It
had to move: the set of valid values is a property of the RESOLVED GRAPH, so a
refusal written in the parser could only compare against a constant, which is
the coupling this whole mechanism exists to remove. But prepare narrates what
it resolves, so the refusal now arrived after three lines on stdout.
FAIL: unsupported value (pack) wrote to stdout: Resolving toolchain
Measured on macos-arm64, and it would have failed on every platform -- the
macOS shard is simply the one that reached it first.
The fix is to be quiet until the value is validated, and only then. Nothing is
lost when the value IS valid: the dispatch pass prepares a second time and
prints the same lines, so a successful `pack --format <name>` narrates once
rather than twice. `--format tar` and `--format dir` are untouched, because
their values were never in question.
The musl re-prepare is quieted on the same grounds: it also runs before the
format has been validated.
The assertion is added to 638_pack_format_dispatch.sh as well as living in 202.
That is deliberate duplication: the tension is local to this feature -- the
valid set needs the graph, and the graph narrates -- so the test for the
feature should fail when the contract does, rather than only the general
contract test noticing.
…s an axis
Section 3 of the design record draws the boundary exactly: a package can add a
language, a tool, an action, a payload and a generated module, and IT CANNOT ADD
A TRIPLE. Identity is three strings and `kKnownTargets` is compiled into the
binary, so every layer below the first -- the `.apk` step, the `.app` step, the
`.html`+`.wasm` step, the runner, the signing, the non-C++ glue -- waits on a
row here and on nothing else in the engine. Registering the rows is what turns
each of those from a plugin with nowhere to attach into a plugin that can be
written.
aarch64-linux-android x86_64-linux-android aarch64-ios wasm32-emscripten
ALL FOUR ARE `planned`, WHICH IS A REFUSAL AND NOT A GAP. The tier gate answers
`tier-planned` naming the row:
error: target 'aarch64-linux-android' is registered but not yet supported
(planned) -- no toolchain is published for it yet
error: target 'aarch64-linux-androideabi' (which resolves to
'aarch64-linux-android') is registered but not yet supported
rather than `unknown target`, which was false, or a build that resolves and
produces nothing, which section 3.1 argues would be worse than the row's
absence. What each row still needs is a PAYLOAD in every case and never engine
work: `xim:android-ndk`, `xim:emsdk`, and for iOS a licence reading before a
packaging decision.
THE OBJECT FORMAT IS NOW ONE ANSWER, AND THAT IS WHAT #597 ACTUALLY NEEDED.
The binary format was never a field. It was re-derived from `os` wherever it was
needed -- `is_pe()` asked `os == "windows"`, artifact naming asked again, the
packer asked a third time -- which is affordable only while the answer has two
values. `wasm32` is the first target in mcpp's vocabulary whose format is
neither, and a THIRD value turns those derivations into an addition at every
such site. A site that is missed does not fail: it silently answers ELF,
because ELF is what every `else` branch in the tree assumes. `ObjectFormat` is
that addition made once, with `is_pe()` / `is_mach_o()` / `is_wasm()` reading it.
It is deliberately NOT the same question as `is_freestanding()`. A bare-metal
RISC-V image is ELF with no OS; a wasm module has an OS-like layer
(Emscripten's POSIX emulation) and is not ELF. Merging the two axes is the
mistake this replaces.
ANDROID'S PLACEMENT IS THE MODELLING DECISION: `env = "android"` on a `linux`
OS, not `os = "android"`. The kernel IS Linux, so ELF, the `unix` family and
`nasm -f elf64` are already right; an OS value would have made every one of
them wrong by default and required a new answer at each site. What differs from
`gnu` is bionic, the loader path and the SDK -- which is what an `env` value is
for. `androideabi` resolves to the same env: the EABI half is the ARM calling
convention, which the arch segment already carries.
`is_apple()` exists because a site that means "Apple" and asks "macOS" gets iOS
wrong in the direction that still links. iOS shares the object format, the
linker, the `arm64` spelling and `codesign` with macOS, and differs in the SDK
and the deployment-target flag. No deployment target is baked into
`llvm_triple()` for it, unlike the macOS branch: `-miphoneos-version-min`
belongs to the layer that owns the SDK and the bundle, and a default here would
be a second place that answers it.
A DISPLAY DEFECT THE ROWS EXPOSED. `x86_64-linux-android` showed no `cross` tag,
because that test compared arch and OS only -- and this target agrees with an
x86_64 Linux host on both. An Android artifact needs bionic's loader at
`/system/bin/linker64`, which no ordinary Linux host has, so it cannot run
there. Spelled as a property rather than by adding `env != env`, which would
have taken `x86_64-linux-musl` with it -- that one is static and does run here.
48 new cells in tests/matrix/expected.tsv, one per (mode, host, compiler) the
table declares, all `unsupported / tier-planned`; the declared per-host counts
move with them, because compare.sh checks the total before it checks a cell.
The day a row is wired, its cells go red and say so. Verified locally:
50/50 payload and 24/24 graph on linux-x86_64.
Docs: `docs/21`'s segment tables gain the new values, a section states the
object-format axis and why it is not the freestanding question, and the
host/target matrix gains four rows. Both READMEs' platform tables record what
each row waits on. Both languages.
Sunrisepeak
changed the title
mcpp pack --format <name> dispatches to a package, and no distribution format lives in the engine (2026.9.11.1)… names the staged tree
`mcpp pack --format <name>` reported as the distributable only those artifact
actions that named `${mcpp.stage_dir}`, on the assumption that a distributable
consumes the staged closure. NOT EVERY FORMAT DOES, and the one that does not is
the one the guidance recommends.
An `.msi` built from ONE NAMED PROGRAM takes `${mcpp.target_file:<name>}` and
never looks at the tree. That is `docs/31`'s "name the input, do not harvest a
directory", written after a bind path that resolved to nothing produced a
valid, empty, 52 KB installer with no diagnostic. So a member following that
guidance failed the check, and `mcpp pack --format msi` reported
no action claimed --format 'msi'
after a `wix build` that had succeeded. The workaround available to the member
was to declare the placeholder as an extra, otherwise-unused input purely to
satisfy the engine -- which also gave it a dependency on a tree its one `File`
row never reads.
The property actually wanted is presence in the dispatch pass. An artifact
action present whether or not a format was asked for -- a codesign stamp, a
size budget -- existed before anyone asked, and reporting one as the package
would be a wrong answer that looks like a right one. So the first pass's
artifact actions are collected, and the dispatch reports the difference.
Identity is (package, id): an id is unique within the package that declared it
and nothing more.
`consumesStageDir` keeps its one real job -- the implicit dependency on the
staged tree's manifest, so an action that reads the tree is dirty when the
staged SET changes and not only when a link output does. Its comment now says
what it is not.
`638_pack_format_dispatch.sh` gains the case that pins this from both sides: a
fixture submitting an UNGATED artifact stamp and a GATED action that names no
staged tree at all. The gated one must be reported and the stamp must not, and
the stamp must still have been built -- it was simply not the answer. Verified
load-bearing by removing the guard, which reports `size.stamp` as the package
and fails the test.
Found while porting `mcpp.dist.wix`, which is to say by writing the second
member. The first one consumed the staged tree and agreed with the check by
accident.
…d two copies were searching for a vendor name
An audit of the 35 sites that derived the binary format from `os`. Twelve of
them asked "which object format" and were converted to read `object_format()`;
the rest ask a different question -- which payload to install, which loader
variable a platform reads, which flag spelling a compiler wants -- and are
unchanged. The classification mattered more than the count: a site that means
"which OS" and is converted becomes wrong in a new way.
WHAT EACH CONVERTED SITE ANSWERED FOR `aarch64-ios` BEFORE:
-femulated-tls, -fvisibility-hidden not passed (Mach-O needs both)
shared_library_link_flags $ORIGIN, which ld64 rejects
shared_soname_flag -Wl,-soname, a BFD-only flag
exports_file_contents / exports_flag a GNU version script
debug_info_is_in_band true; Mach-O splits to dSYM
dist::format_for the BUILD HOST's format
Every one of those is the ELF branch reached by falling off the end of a
two-valued test, which is the failure `ObjectFormat` was introduced to make
impossible. Verified for every row in `kKnownTargets` that the answer changes
only for `aarch64-ios` and `wasm32-emscripten`, and only toward correctness.
TWO COPIES WERE SEARCHING FOR A VENDOR NAME IN THE WRONG STRING, AND ONE OF
THEM AFFECTS AN ALREADY-VERIFIED ROW.
`compute_flags`'s `linkIntentFlavor` and `resolution.json`'s `format` both
derived the object format by looking for "apple" / "darwin" / "windows" /
"mingw" in `plan.toolchain.targetTriple`. That string is mcpp's CANONICAL
spelling, and `aarch64-macos` contains none of those words. The words live in
the LLVM spelling, which is a different string -- the build report prints both,
either side of an arrow:
Target aarch64-macos → arm64-apple-macos14.0
^ the identity ^ what clang is given
So an explicit `--target aarch64-macos` linked and recorded as ELF. A NATIVE
macOS build was right by a different branch -- an empty triple reaching the
`needs_explicit_libcxx` rescue -- which is why nothing caught it: two paths
through one function disagreed and only the exercised one was correct. Both now
ask the parsed triple, and the substring test survives only for a spelling
`parse` REJECTS, which is the `[target.<triple>]` escape hatch where an
LLVM-shaped string is what an author actually wrote.
`test_toolchain_triple.cpp` states this as a fact about the vocabulary rather
than as a comment elsewhere: for `aarch64-macos`, `x86_64-macos` and
`aarch64-ios` it asserts the canonical spelling contains neither "apple" nor
"darwin", that the format is Mach-O anyway because it is asked of the fields,
and that the LLVM spelling is where the vendor name lives. A second test takes
its denominator from the table, so a row added without an answer cannot be
covered by a test whose name says every row is.
TWO GAPS ARE NAMED RATHER THAN GUESSED. `dist::Format` and `LinkIntentFlavor`
have no `Wasm` member, so `wasm32-emscripten` still resolves to `Elf` in both.
What "self-contained" and "link_lib" mean for an Emscripten link is a
distribution-contract decision and the open half of #597, not a rename; the
switch names the case so the gap is visible instead of reached by falling
through. `mcpp::pack::run` now refuses a file that is neither ELF, PE nor
Mach-O by name, where it previously handed anything not-PE-not-Mach-O to
`LD_TRACE_LOADED_OBJECTS`.
Docs: the design record's section 3.1 is corrected twice, because two of its
guesses were measured wrong in the same direction -- a vendor had already done
the work and nobody looked. NDK r30 ships the 133-file module surface itself
(r27 shipped none), so for Android there is nothing to derive; and Apple's
libc++ IS a build of a public revision (210106 -> llvmorg-21.1.6), the SDK
ships no surface, and an `import std` Mach-O arm64 binary was linked from a
Linux host. Section 8 names the pattern rather than only listing the facts.
109 unit tests pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The mcpp side of
.agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md, in full. Two halves, one release, and a commit per checkpoint.The design record's boundary is what makes them one PR: a package can add a language, a tool, an action, a payload and a generated module, and it cannot add a triple. So the dispatch mechanism and the target rows are the same kind of thing — the engine's contribution to a layering where everything else lives in packages.
Part 1 —
mcpp pack --format <name>dispatches to a packagetaranddiranswer the same questionmsiandappimageanswer — what shape does the output take — so they are values of one flag rather than the beginning of a second one.The universal format is what it already produces: an archive that extracts and runs, universal in the only sense that matters here — it needs no knowledge of anyone else's release. Everything past it does. dpkg's control fields, AppImage's runtime, WiX's schema, Apple's notarisation: each one bound into the engine couples an mcpp release to a release mcpp does not control. This project already made the argument for languages, where Slang is supported without being named in the engine.
Three additions, each format-neutral, which is the test for whether something belongs in the engine at all.
${mcpp.stage_dir}artifactaction can consume.mcpp packalready computes one — the dependency closure after the strip policy, the debug-symbol split andinclude/exclude— then compressed it and the directory was gone, so a.deb, an AppImage, a.appand an.msieach had to rebuild it.mcpp::package_version()+ four more[package]in the build program. Every installer states a version; without this a project restates it in the member's own options, where the copy drifts from[package]with nothing able to detect it.mcpp::provides_pack_format()--formatresolves through the graph. An unknown value names what is available rather than a constant, before anything is compiled.[pack]'s built-in modes become one provider among several rather than the model.docs/10's Planned Support loses.deb/.rpm/ AppImage, which is now a decision rather than an omission.Declare unconditionally, submit conditionally
The load-bearing rule, and the one a member author is most likely to get wrong — because a member that gets it wrong still works for whoever wrote it: they always pass their own format. The declaration must not be gated, or the engine can never answer "which formats does this graph provide". The submission must be, or a plain
mcpp buildgrows an edge it must not have.Two passes, and nothing re-derived between them
An
artifactaction is a ninja edge and the staged tree is produced after the link, so a single-pass--formatis not expressible. The first pass collects declarations and refuses an unknown format; the build and staging follow; the second setspack_formatandpack_stage_dirfrom what the first pass andmake_planalready answered.stagingRootdepends on the resolved triple, and a second derivation of it before prepare is the shape where two answers agree on every machine the author has.The graph is state, so the graph says what it is
# mcpp:graph=normal;schedule=none;accel=default;dist=none— a fourth field, and the fast paths require it to readnone. The format is deliberately not in the fingerprint (that would cost a full recompile to package an already-built tree), so the two graphs share a directory. Third instance of the failuregraph=andaccel=each already record.The criterion is a unit test, not an end-to-end assertion, and that is measured. A plain build after the pack pass regenerates the graph even with this field ignored, so an end-to-end check would pass whether or not the field works — and would keep passing if it were deleted.
Part 2 — Android, iOS and wasm are rows, and the object format is an axis
Every layer below the first — the
.apkstep, the.appstep, the.html+.wasmstep, the runner, the signing, the non-C++ glue — waits on a row inkKnownTargetsand on nothing else in the engine.All four
planned, which is a refusal and not a gap:rather than
unknown target, which was false, or a build that resolves and produces nothing, which §3.1 argues would be worse than the row's absence. What each row still needs is a payload in every case and never engine work.The object format is now one answer
This is what #597 actually needed. The binary format was never a field: it was re-derived from
oswherever it was needed —is_pe()askedos == "windows", artifact naming asked again, the packer a third time — which is affordable only while the answer has two values.wasm32is the first target whose format is neither, and a third value turns those into an addition at every such site, where a missed site does not fail: it silently answers ELF, because ELF is what everyelsebranch assumes.ObjectFormatis that addition made once. It is deliberately notis_freestanding(): a bare-metal RISC-V image is ELF with no OS; a wasm module has an OS-like layer and is not ELF.Android's placement is the modelling decision
env = "android"on alinuxOS, notos = "android". The kernel is Linux, so ELF, theunixfamily andnasm -f elf64are already right; an OS value would have made every one of them wrong by default. What differs fromgnuis bionic, the loader path and the SDK — which is what anenvvalue is for.is_apple()exists because a site that means "Apple" and asks "macOS" gets iOS wrong in the direction that still links.A display defect the rows exposed:
x86_64-linux-androidshowed nocrosstag, because that test compared arch and OS only — and this target agrees with an x86_64 Linux host on both. An Android artifact needs bionic's loader and cannot run there. Spelled as a property rather thanenv != env, which would have takenx86_64-linux-muslwith it — that one is static and does run here.Two defects of my own, found by measurement
An unsupported
--formatwrote to stdout. Moving the refusal pastprepare_buildbroke the machine-output contract —FAIL: unsupported value (pack) wrote to stdout: Resolving toolchainon macos-arm64, and it would have failed everywhere. Fixed by staying quiet only until the value is validated; nothing is lost, because the dispatch pass narrates the same lines.The "which action is the distributable" criterion refused the member that followed the guidance. It collected only actions naming
${mcpp.stage_dir}, assuming a distributable consumes the closure. An.msibuilt from one named program takes${mcpp.target_file:<name>}and never looks at the tree — which is exactly what §6 recommends, after a bind path that resolved to nothing produced a valid, empty, 52 KB installer. Sodist-wixhad to name the placeholder as an unused input purely to satisfy the check. The dispatch now asks which artifact actions the request introduced, which needs nothing of the member.Verification
test_toolchain_triplecovering the object format, the three vocabularies and the planned rows.tests/e2e/638_pack_format_dispatch.shholds eight properties, each paired with the wrong answer it excludes — including an artifact action present in both passes, which must not be reported as the package. Verified load-bearing by removing the guard: it reportssize.stampand the test fails.255,256,266,627); all four A/B'd against released 2026.9.10.2 produce byte-identical logs, and627passes standalone.Docs:
docs/10,docs/21,docs/30,docs/31, both READMEs' platform tables. Both languages. Build-program protocol goes to v9.The first members of the new category (
dist-appimage,dist-wix,dist-apple) andtools-embed'stable()are a companion PR inmcpp-plugins; they need this release.xim:appimagetoolis already merged inxim-pkgindex, and the whole Linux chain is verified end to end — the produced AppImage runs and prints its program's output.