Skip to content

feat(compat): the eight general-purpose libraries a modern Vulkan app needs - #370

Merged
Sunrisepeak merged 4 commits into
mainfrom
feat/vulkan-rt-general-libs
Sep 8, 2026
Merged

feat(compat): the eight general-purpose libraries a modern Vulkan app needs#370
Sunrisepeak merged 4 commits into
mainfrom
feat/vulkan-rt-general-libs

Conversation

@Sunrisepeak

Copy link
Copy Markdown
Member

Audited Stehsaer/vulkan-rt's dependencies against this index: of its thirteen, four were already here (vulkan-headers, vulkan-hpp, vulkan-memory-allocator, nlohmann_json). This adds the rest except SDL3, which is a package-sized job of its own — see below.

Nothing here is specific to that project. glm, doctest, argparse and libcoro are the general answers to math, testing, argument parsing and coroutines; the cpptrace/libassert pair is the general answer to "an assertion that tells you what the values were".

package version shape
compat.glm 1.0.2 header-only, include root is the tarball root
compat.doctest 2.4.12 header-only; main comes from a macro, not a feature
compat.argparse 3.2 header-only
compat.mio 2023.3.3 header-only, commit-pinned (upstream has never cut a tag)
compat.gzip-hpp 0.1.0 header-only over compat.zlib, + two #include_next shims
compat.cpptrace 1.0.4 whole-source glob + defines
compat.libassert 2.2.1 8 TUs, depends on cpptrace
compat.libcoro 0.16.0 core half only — the tarball cannot build the rest

CN mirrors, all eight

gitcode.com/mcpp-res/<slug>, upstream re-hosted byte-identically and verified as such: every mirror was downloaded back and its sha256 compared against the declared one — 8/8 identical. That equality is what lets a single sha256 serve both arms of the url table.

Four of the eight are not "expose the headers and move on"

compat.gzip-hpp is frozen at a point where it no longer compiles. utils.hpp names uint8_t six times and decompress.hpp once, and neither includes <cstdint>:

include/gzip/utils.hpp:14:32: error: 'uint8_t' does not name a type

It is a consumer-side failure — the package has no C++ TU of its own, so cxxflags = {"-include", "cstdint"} (the repair compat.redis-plus-plus uses) cannot reach the TU that breaks. The one lever a descriptor has is a same-named shim first on the include path reaching upstream's with #include_next, which is exactly what compat.catch2 does for its <new> defect. Two shims, because decompress.hpp and utils.hpp are independent entry points, and include_dirs must list mcpp_generated first.

compat.cpptrace is shape E without the config header. Six symbol backends, six unwinders, three demanglers — and every backend file opens with its own #ifdef, so the source list is one glob and the whole configuration lives in defines, the way curl's unselected TLS backends compile to nothing. The backends chosen are the dependency-free ones on every platform: libgcc's _Unwind_Backtrace + dladdr + __cxa_demangle on unix, DbgHelp on windows. ⚠️ The cost is documented rather than hidden: dladdr does not read DWARF, so frames carry function names and no file:line. A libdwarf feature can add that later without changing what a consumer writes.

compat.libassert takes cpptrace as a real dependency where upstream FetchContents a private copy, so a consumer linking both gets one. magic_enum turns out not to be needed at all — LIBASSERT_USE_MAGIC_ENUM is switched on only under LIBASSERT_BUILD_TESTING, and the library sources never mention it.

compat.libcoro ships the core only, and the tarball decides that. Upstream's networking half resolves DNS through c-ares, which lives in vendor/c-ares — a git submodule, so an empty directory in any GitHub archive (measured: 0 files). The source list is upstream's LIBCORO_SOURCE_FILES before the networking block appends to it, transcribed rather than globbed, because scheduler.cpp, poll.cpp, the epoll/kqueue notifiers and all of net/ do not self-guard. coro/export.hpp is generate_export_header output the tarball does not carry — without it the first TU dies with a fatal error — so it is snapshotted, and CORO_STATIC_DEFINE goes in defines because it decorates declarations consumers include.

The tests assert behaviour, not compilation

  • glm — cross product, column-major storage, and that GLM_FORCE_DEPTH_ZERO_TO_ONE really maps the near plane to 0 (the assertion that tells Vulkan's clip space from OpenGL's). Both switches are set by the member, not the package: they change the meaning of types crossing a library boundary.
  • gzip-hpp — gzip magic and that a repetitive payload actually shrinks, not just that it round-trips.
  • cpptrace — a four-deep chain must yield ≥4 frames with distinct return addresses; a single-frame stub passes a count check but not that one.
  • libassert — drives the failing path and reads the report back: expression text, message, both decomposed operand values, and a non-empty stacktrace. That last one is the proof cpptrace arrives transitively, since the project declares only libassert.
  • libcoro — 200 coroutines through a thread pool under a coro::mutex.
  • doctest / argparse / mio — subcases run independently; a missing required positional must throw; a write through the mapping must reach the file.

Also in this PR

A separate first commit fixes something I got wrong in #369: vk::to_string IS part of the vulkan module. vulkan.hpp includes vulkan_to_string.hpp itself (guarded only by VULKAN_HPP_NO_TO_STRING), so reading vulkan.cppm's include list as exhaustive was the error. The claim is now an assertion in the member rather than a comment.

Not included: SDL3

Deliberately, and the state of the work is worth recording. The Linux half is already solved — SDL_build_config.h generated with this index's toolchain and the index's own X11 headers (yielding Video drivers: dummy offscreen x11(dynamic), the same shape compat.sdl2 uses), and the selected source set maps to exactly 73 whole directories + 1 partial (src/core/linux, skipping the dbus/IME files — the same set compat.sdl2 skips).

What is missing is the macOS and Windows source sets, which cannot be verified anywhere but CI, and compat.sdl2 — the closest analogue — is a 923-line descriptor. Bundling that here would make eight finished packages hostage to SDL3's iteration cycles. Happy to push it onto this same branch before merge if preferred.

…st says so

The descriptor and the member's module.cpp both claimed `vk::to_string` was
outside the module's surface because `vulkan_to_string.hpp` is not in the list
of headers `vulkan.cppm` includes. That reasoning read the list as exhaustive,
and it is not: `vulkan.hpp` includes `vulkan_to_string.hpp` ITSELF, guarded
only by `VULKAN_HPP_NO_TO_STRING`, which nothing here defines. So it rides into
the module purview and IS exported.

Measured through a plain consumer of the published package:

    to_string(eR8G8B8A8Unorm) = R8G8B8A8Unorm
    to_string(eDeviceLost)    = ErrorDeviceLost

The claim is now an assertion in module.cpp rather than a comment — both
overloads, the enum one and the bitmask one, which come from different
generated sections of that header. `vulkan_static_assertions.hpp` is the one
that really is absent, and the comment now says so and why (nothing includes
it, and it is a self-check for the headers rather than API a consumer calls).
… needs

Adds glm 1.0.2, doctest 2.4.12, argparse 3.2, mio 2023.3.3, gzip-hpp 0.1.0,
cpptrace 1.0.4, libassert 2.2.1 and libcoro 0.16.0, each with a workspace
member that asserts behaviour, and each with a CN mirror.

The list comes from auditing Stehsaer/vulkan-rt's dependencies against this
index: of its thirteen, four were already here (vulkan-headers, vulkan-hpp,
vulkan-memory-allocator, nlohmann_json). These are the rest, minus SDL3, which
is a package-sized job of its own and is not in this commit.

Nothing here is specific to that project. glm, doctest, argparse and libcoro
are the general answers to math, testing, argument parsing and coroutines; the
cpptrace/libassert pair is the general answer to "an assertion that tells you
what the values were".

CN MIRRORS, ALL EIGHT. gitcode.com/mcpp-res/<slug>, upstream re-hosted
byte-identically and verified as such: every mirror was downloaded back and its
sha256 compared against the declared one, 8/8 identical. That equality is what
lets a single `sha256` serve both arms of the url table.

Four of the eight are not "expose the headers and move on":

* compat.gzip-hpp is frozen at a point where it NO LONGER COMPILES: utils.hpp
  names uint8_t six times and decompress.hpp once, and neither includes
  <cstdint> (measured, gcc 16.1.0). It is a CONSUMER-side failure — the package
  has no C++ TU of its own, so `cxxflags = {"-include", "cstdint"}`, the repair
  compat.redis-plus-plus uses, cannot reach the TU that breaks. The one lever a
  descriptor has is a same-named shim first on the include path reaching
  upstream's with #include_next, which is what compat.catch2 does for its <new>
  defect. Two shims, because decompress.hpp and utils.hpp are independent entry
  points, and include_dirs must list mcpp_generated FIRST.

* compat.cpptrace is shape E without the config header. Six symbol backends,
  six unwinders and three demanglers, and EVERY backend file opens with its own
  #ifdef — so the source list is one glob and the configuration is entirely in
  defines, the same way curl's unselected TLS backends compile to nothing. The
  backends chosen are the dependency-free ones on every platform: libgcc's
  _Unwind_Backtrace + dladdr + __cxa_demangle on unix, DbgHelp on windows. The
  cost is documented rather than hidden — dladdr does not read DWARF, so frames
  carry function names and no file:line. A libdwarf feature can add that later
  without changing what a consumer writes.

* compat.libassert takes cpptrace as a real dependency where upstream
  FetchContents a private copy, so a consumer linking both gets one. magic_enum
  turns out not to be needed at all: LIBASSERT_USE_MAGIC_ENUM is switched on
  only under LIBASSERT_BUILD_TESTING, and the library sources never mention it.

* compat.libcoro ships the CORE only, and the tarball decides that: upstream's
  networking half resolves DNS through c-ares, which lives in vendor/c-ares — a
  git submodule, so an empty directory in any GitHub archive (measured: 0
  files). The source list is therefore upstream's LIBCORO_SOURCE_FILES before
  the networking block appends to it, transcribed rather than globbed, because
  scheduler.cpp, poll.cpp, the epoll/kqueue notifiers and all of net/ do not
  self-guard. coro/export.hpp is generate_export_header output that the tarball
  does not carry — without it the first TU dies — so it is snapshotted, and
  CORO_STATIC_DEFINE goes in `defines` because it decorates declarations
  consumers include.

The tests assert behaviour, not compilation: glm checks the cross product,
column-major storage and that GLM_FORCE_DEPTH_ZERO_TO_ONE really maps the near
plane to 0 (the assertion that tells Vulkan's clip space from OpenGL's);
gzip-hpp checks the gzip magic and that a repetitive payload actually shrinks,
not just that it round-trips; cpptrace checks a four-deep chain produces four
frames with DISTINCT return addresses, which a single-frame stub would fail;
libassert drives the FAILING path and reads the report back, asserting the
expression text, the message, both decomposed operand values and a non-empty
stacktrace — that last one is the proof cpptrace arrives transitively, since
the project declares only libassert; libcoro runs 200 coroutines through a
thread pool under a coro::mutex.

Verified: all eight members pass on linux/gcc 16.1.0, and glm was re-resolved
from a cleared store to confirm the rewritten url tables still resolve.
…e-private

The macro decorates DECLARATIONS, so it has to reach every TU that includes
cpptrace's headers — not just cpptrace's own. It was in `cxxflags`, which mcpp
applies to the package alone.

Linux could not see the mistake. There the difference is
`visibility("default")` versus nothing, and the link is unaffected. On the MSVC
ABI it is `dllimport` versus nothing, which is an ABI difference, and
compat.libassert — whose TUs include these headers — failed to link on the
windows CI leg while all seven other new members passed:

    lld-link: warning: locally defined symbol imported:
        cpptrace::v1::runtime_error::runtime_error(...) [LNK4217]
    lld-link: error: undefined symbol: __declspec(dllimport)
        cpptrace::v1::stacktrace_frame::operator!=(...) const

Moved to `defines`, which is the same key compat.libassert already uses for its
own LIBASSERT_STATIC_DEFINE, for the same reason. The backend selectors
(CPPTRACE_UNWIND_WITH_*, GET_SYMBOLS_WITH_*, DEMANGLE_WITH_*) stay in
`cxxflags` deliberately: they decide which .cpp compiles to something and are
nobody else's business.

The measurement is now recorded in the descriptor so the next person does not
have to reach it through a red windows job.
…m, not a key

The previous commit moved CPPTRACE_STATIC_DEFINE from `cxxflags` to `defines`
on the theory that `defines` is interface-visible. It is not, and the compile
database says so plainly: in a build containing both packages, libassert's
eight TUs carry `-DLIBASSERT_STATIC_DEFINE` (their OWN package define) and zero
`-DCPPTRACE_STATIC_DEFINE`, while cpptrace's 45 carry the reverse.
docs/repository-and-schema.md states the same for a feature's `defines`. There
is no interface-define key in this grammar, so that fix would have failed on
windows exactly like the one before it.

What does work is the lever compat.gzip-hpp and compat.catch2 already use: a
shim of the same header name, first on the include path, reaching upstream's
with #include_next. Both libraries make that a one-file job, because in each
the macro is read by exactly one header and every other public header funnels
through it — cpptrace/basic.hpp (included by cpptrace.hpp, exceptions.hpp,
formatting.hpp, from_current.hpp, gdb_jit.hpp, io.hpp, utils.hpp) and
libassert/platform.hpp (included by all nine of its siblings). So one shim per
package serves the package's own TUs, the other package's, and any consumer's
alike. `mcpp_generated` now sorts first in both include_dirs.

Both tests gained a compile-time guard:

    #if !defined(CPPTRACE_STATIC_DEFINE)
    #  error "... the shim was not reached (include_dirs order?)"
    #endif

That is the point of this commit as much as the fix is. The defect is INVISIBLE
on Linux — there the difference is `visibility("default")` versus nothing and
everything links — so it could only ever be caught by the windows leg, after a
push. Now a reordered include_dirs fails to compile on every platform.

Verified locally: both members pass WITH the guards, while the consumer TU's
command line carries neither macro. The macros can therefore only have come
from the shims.
@Sunrisepeak
Sunrisepeak merged commit 0417460 into main Sep 8, 2026
14 checks passed
@Sunrisepeak
Sunrisepeak deleted the feat/vulkan-rt-general-libs branch September 8, 2026 13:34
Sunrisepeak added a commit that referenced this pull request Sep 8, 2026
* feat(compat.sdl3): SDL3 3.4.2, built from source

The last of vulkan-rt's dependencies, and the one that could not ride along in
#370: 266 sources, three platforms, and a config header that has to be
generated. Shape E, sibling to compat.sdl2 rather than its replacement — SDL2
and SDL3 are different APIs with different sonames.

Two things are easier than they were for SDL2, both measured rather than
assumed:

* NO REPACKED TARBALL. SDL2's entry points at an xlings-res re-host because the
  upstream archive carries symlinks that break the windows runner. SDL3's
  carries none (`tar tvzf | grep -c '^l'` is 0), so GLOBAL points straight at
  GitHub.
* THE CONFIG DISPATCHER IS UPSTREAM'S. include/build_config/SDL_build_config.h
  already selects _windows.h / _macos.h by platform and both are checked in.
  Only linux falls through to SDL_build_config_minimal.h, which has no video
  driver at all. So exactly one config is generated, and only linux uses it.

THE CONFIG MAY ONLY CLAIM WHAT THIS INDEX PACKAGES. CMake probes the machine it
runs on, so anything the host happens to have is switched on and then fails to
compile here. Three were found by failing builds — XSCRNSAVER (fatal error:
X11/extensions/scrnsaver.h), LIBTHAI, and HIDAPI_LIBUSB — and XTEST/XSYNC are
off pre-emptively. What remains is checkable rather than hopeful: every library
the finished config says it will dlopen must have a package here. That list is
seven (libfribidi, libX11, libXcursor, libXext, libXfixes, libXi, libXrandr)
and it matches linux.deps exactly.

It was also generated WITH THIS INDEX'S TOOLCHAIN and the index's own X11
headers, which is not a formality: mcpp's gcc has its own sysroot and cannot
see /usr/include, so without them SDL's check fails and configure aborts.
compat.curl records what the shortcut costs — a config generated with the host
cc once asserted that ssize_t does not exist.

The source list is CMake's own answer rather than a reading of CMakeLists:
after configuring, the objects it decided to build were read back out of
CMakeFiles/SDL3-static.dir/build.make and mapped to directories — 266 sources,
73 directories taken whole and exactly one in part (src/core/linux, minus the
six dbus/IME files compat.sdl2 also omits, which do not self-guard).

macOS and Windows do not use the generated config, so their lists are derived
from what the CHECKED-IN configs switch on. The non-obvious one is windows:
SDL_THREAD_GENERIC_COND_SUFFIX and _RWLOCK_SUFFIX mean the windows backend
falls back to the generic condition variable and rwlock, so
thread/generic/SDL_syscond.c and SDL_sysrwlock.c must be linked alongside
thread/windows/* — only those two, since the other four share basenames with
their windows twins.

The test drives SDL's DUMMY video driver, so it runs headless on all three
platforms: header/runtime version agreement, the compiled-in driver list
containing dummy AND offscreen (a config that fell through to minimal has
neither), the dummy driver actually initialising and producing a 320x240 window
with a surface of that size, and the timer advancing.

Verified: linux gcc 16.1.0 and linux llvm 22.1.8 both compile all 267 units and
run the test. macOS and Windows are derived as described and confirmed by CI —
there is no way to compile either from here. CN mirror at
gitcode.com/mcpp-res/sdl3, byte-identical to upstream (verified).

* fix(compat.sdl3): the C++ sources a *.c glob cannot see, and GL/glx.h

Two failures from the first CI run, both from assuming a shape rather than
checking it.

WINDOWS: SDL3 IS NOT ALL C. `core/windows`, `video/windows` and the GameInput
joystick backend (`joystick/gdk/SDL_gameinputjoystick.cpp`) carry C++ sources,
and `*/src/.../*.c` drops them silently — a glob that misses a file produces no
diagnostic at all until the link:

    lld-link: error: undefined symbol: WIN_InitGameInput
    lld-link: error: undefined symbol: SDL_GAMEINPUT_JoystickDriver

`render/direct3d12` has the same problem in the COMMON list: three of its files
are .cpp, so the `*.c` glob was linking half a renderer. Rather than patch only
the symbols the linker happened to name, the whole class was found by asking
the tree which sources are not .c — 34 directories, of which the windows-
relevant ones are these four. macOS was already right because its backends are
.m and the globs say .m.

LINUX/LLVM: `SDL_x11opengl.h` includes <GL/glx.h>, which is not in the Khronos
registry and needs `compat.glx-headers` — the same dependency, for the same
reason, that compat.sdl2 carries. This one is worth the comment it now has,
because it is invisible from two of the three places you would look: the gcc
toolchain's sysroot already carries GL/glx.h, so linux/gcc passed and so did a
local build with either compiler. Only the llvm leg says:

    fatal error: 'GL/glx.h' file not found

Verified after the fix that the header now resolves to the PACKAGE's copy
(.mcpp/.../compat-x-glx-headers/1.7.0/libglvnd-1.7.0/include/GL/glx.h) rather
than to whatever the toolchain happened to have, which is the difference
between a fix and a coincidence. Both linux legs pass; windows is CI's to
confirm.

* fix(compat.sdl3): actually add the windows C++ sources

The previous commit's message described this change; the file did not contain
it. The edit that was supposed to insert three .cpp globs into the windows
source list was written against a pattern with two globs on one line, while the
descriptor has one per line, so it matched nothing and changed nothing — and
because that particular substitution carried no assertion, it failed silently.
Only the direct3d12 line, which was a separate edit, landed.

CI said so immediately and identically: the same five undefined symbols, the
second time round.

    lld-link: error: undefined symbol: WIN_InitGameInput
    lld-link: error: undefined symbol: SDL_GAMEINPUT_JoystickDriver

The globs are in now, and the reasoning is recorded next to them: WIN_* is
defined in video/windows/SDL_windowsgameinput.cpp, the joystick driver in
joystick/gdk/SDL_gameinputjoystick.cpp, and core/windows carries two more C++
TUs. All four are reached because the CHECKED-IN windows config sets
SDL_JOYSTICK_GAMEINPUT — which this descriptor does not get to decide, so the
sources have to be there.

Linux still passes; windows is CI's to confirm.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant