Problem
CI on main only tests the tree at its current version. Some code depends on the version number itself, for example:
- tests that pin the current minor;
- protocol-major checks;
- scaffold templates;
- docs image tags.
That code is first exercised after pnpm run version, and that runs only on changeset-release/main. So these failures show up at release time, on a PR that nobody can commit to: the version-pr lane rebuilds it with git reset --hard. The release then waits a full cycle:
- file the defect;
- fix it on
main;
- merge the fix;
- wait for the next scheduled or dispatched refresh of the version PR;
- re-run CI.
Precedents of the same class
(Advisory-discovery reds such as #20561 are a different class. The daily OSV scan already covers them, and they are out of scope here.)
Proposal
Add a scheduled job (daily, or on each push to main that adds a changeset) that:
- clones
main and prepares it with scripts/pm/release-rehearsal-clone.mjs --prepare;
- runs
pnpm run version there, with no push and no publish;
- builds, then runs the test suites against the bumped tree. The whole Test Core matrix is the ideal. If that is too expensive, run at least the version-sensitive packages:
create-objectstack, the CLI scaffolds, Scaffold E2E, and the protocol-version sync checks;
- on red, opens or updates one finding card. It never touches
changeset-release/main.
Acceptance
Open question for maintainers
The cost trade-off: run the full Test Core matrix on the bumped tree, or only a curated subset. A curated subset needs its own gate so that the list does not rot.
Generated by Claude Code
Problem
CI on
mainonly tests the tree at its current version. Some code depends on the version number itself, for example:That code is first exercised after
pnpm run version, and that runs only onchangeset-release/main. So these failures show up at release time, on a PR that nobody can commit to: theversion-prlane rebuilds it withgit reset --hard. The release then waits a full cycle:main;Precedents of the same class
.changeset/*.mdpopulation control makesLint & Repo Gatesred on the Version Packages PR — the release cannot be merged, and main goes red the moment it lands #15255: dispatch-gates red on the Version Packages PR.create-objectstack's runtime-image test hard-codes minor 4 and reds on the 17.5.0 bump.(Advisory-discovery reds such as #20561 are a different class. The daily OSV scan already covers them, and they are out of scope here.)
Proposal
Add a scheduled job (daily, or on each push to
mainthat adds a changeset) that:mainand prepares it withscripts/pm/release-rehearsal-clone.mjs --prepare;pnpm run versionthere, with no push and no publish;create-objectstack, the CLI scaffolds, Scaffold E2E, and the protocol-version sync checks;changeset-release/main.Acceptance
mainbefore any Version Packages PR exists.contents: read, plusissues: writefor the finding card only.Open question for maintainers
The cost trade-off: run the full Test Core matrix on the bumped tree, or only a curated subset. A curated subset needs its own gate so that the list does not rot.
Generated by Claude Code