Conversation
jrandolf
marked this pull request as ready for review
September 24, 2026 06:38
Describe executables and ZIPs with one private application specification, and share environment preparation, storage ownership and packaging across ordinary binaries, legacy executable ZIPs and py_zipapp entry points. Use native POSIX exec to preserve the application PID, signals and terminal behavior, with a detached watcher for temporary-file cleanup. On Windows, wait for the application child and its console cleanup before removal. Publish complete persistent images without replacing concurrent entries. Preserve public providers, custom templates and stage-two behavior. Keep historical raw-template adapters available and support Python 3.9 ZIP loaders. Document cleanup lifetime and legacy compatibility limitations. Validation: 727 root targets pass; the launcher matrix passes 313 cases with two skips. Native checks pass on macOS, Windows and Linux arm64/x64, including final loader/ZIP checks on all four pools. Changed-file hooks, documentation, distribution and independent implementation audit pass.
jrandolf
force-pushed
the
codex/bootstrap-runtime-cleanup
branch
from
September 24, 2026 14:54
70c2c88 to
7fac02f
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Temporary virtual environments and executable ZIPs currently leave a launcher
process in front of Python so it can remove files afterward. A termination signal
sent directly to that PID can leave the application running and skip its cleanup.
The different executable and ZIP startup paths also disagree about interpreter
arguments, cache identity, and ownership during failed or concurrent preparation.
Describe the application once, then launch the same prepared image through an
ordinary executable, a legacy executable ZIP, or either
py_zipapp_binaryentrypoint. On POSIX the application replaces the launcher in its original PID, keeping
native signal delivery, process groups, terminal input, and exit status. A detached
watcher removes only temporary files after that PID exits. Windows starts and
reaps the application child before removal, allowing the application's own
Ctrl-C cleanup to finish without forwarding another interrupt.
The implementation separates the private application description, environment
preparation, storage ownership, process launch, and thin entry adapters:
virtual environment, entry point, and explicit links. Both ZIP rule families
use the same packager. Ordinary builds keep their runfiles without copying the
dependency tree.
honoring cancellation. The bundled ZIP shell reaps its preparation children
and validates a fixed six-field result before native execution.
relative directory symlink to an owned backing directory. Publication cannot
replace an existing entry, including one created concurrently. Content, modes,
layout, bootstrap code, and resolved runtime facts contribute to identity.
Completed bundled ZIP cache hits can execute directly from the shell.
precompilation, and coverage behavior. Custom templates preserve their bytes
and historical substitutions; the original raw templates remain usable by
older expanders. Public-only executable providers retain a compatibility path.
Validation uses Bazel 9.1.1 and
--config=fast-testsfor test targets:matrix has 315 cases (313 passed, two skipped), passing under macOS Bash 3.2
and Bash 5.3.20. The final 315-case matrix also passes on Bazel 7.7.1 in
WORKSPACE mode and Bazel 9.1.1, each with the same two expected skips.
targets, Linux arm64 13/13, Linux x86_64 13/13, and Windows x86_64 7/7
passed. Final affected custom/public-provider checks also pass on all four
platform pools.
Execution logs distinguish actual remote test execution from cache hits.
regression covers ZIP loaders that predate
exec_module; all ten affectedlocal ZIP/template targets pass. The final loader and packaged ZIP follow-up
also passes three targets on each of the four native Buildbarn pools.
finally,inherited ignored signals and masks, terminal job control, nesting, arguments,
wrapper runtimes, setup/exec failures, cache publication races, Windows console
handling and exit status, custom templates, and watcher readiness/descriptor
lifetime. Native macOS kqueue and Linux subreaper checks pass. Linux pidfd
and procfs checks pass in a separate Debian guest where those APIs are available;
these backend proofs reject fallback and verify owned-child/descriptor lifetime.
module: 11/11. The Sphinx check privately registers its required Python 3.11;
no dependency configuration changes are included.
documentation and distribution builds pass. Original ordinary and ZIP launchers
reproduce the direct-PID termination failure; the new launchers preserve the
PID, execute application cleanup, and remove their temporary files.
Temporary execution costs one resident watcher, measured at roughly 4.7–6.8 MiB
RSS locally. Runtime-created virtual environments also enter Python for
preparation. In the latest six-sample, 500-file comparison on a loaded shared
host, baseline/current startup medians were 75/73 ms for prepared executables,
144/289 ms for runtime-created venvs, 1724/1559 ms for cold ZIPs, and 77/74 ms for
warm ZIPs. Earlier quieter runs varied substantially; these are measurements,
not performance guarantees. Both builds spawn three build actions; the new
description adds three internal metadata writes.
Removal is asynchronous and follows the original interpreter, including exec;
it does not extend to descendants that outlive it. Namespace PID 1 and subreapers
can adopt the watcher, and namespace/cgroup shutdown or SIGKILL before setup
registration can prevent cleanup. A persistent extract root avoids the temporary
watcher where supported. Its published backing directories belong to the caller;
removing only a cache symlink does not reclaim the backing image. These lifetime
limits and cache cleanup semantics are documented.
Older custom rules exposing only
PyExecutableInfouse the unchanged raw-templateadapter. Its Windows persistent extraction can fail when a repeated launch meets
an existing directory symlink. This also reproduces with the original ZIP rule,
templates, and packaging tools; normal native launch and archive structure pass
in both versions. Temporary extraction is the documented workaround for that
compatibility path. The new default image path passes repeated persistent-launch
checks on Windows.