Repository navigation
test(verify): five slow test files pay the boot's first load at module scope, and handle builds its second fixture without a registry reset - #23620
Conversation
…test files, and build handle's second fixture without a registry reset Five of the @objectstack/verify files the hourly timing table lists over 10 s of wall paid their first boot's cold load (lazily imported drivers and better-auth entries, sql.js's WASM compile, cold paths) inside the first clocked window. A shared testkit, boot-warm-up.testkit.ts, runs one throwaway boot, sign-in and sign-up at module scope, so that cost is paid during collection, the clocked-window convention in AGENTS.md. handle.test.ts also built its two "configuration of its own" fixtures with vi.resetModules() plus a re-import, which re-evaluated the fixture's whole module graph and every lazy load of the boot after it. It now imports the fixture module alone under a specifier of its own. Titles and assertions are unchanged. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FLwgbTSmGMrqsXkBDsiwfG
…re comments Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FLwgbTSmGMrqsXkBDsiwfG
…-verify-test-budget
📓 Docs Drift Check2 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run. What this run could not see
Coarse fallback — 5 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 2657ad59816f2cfe19f5449528335d49f67ee082 && git checkout 2657ad59816f2cfe19f5449528335d49f67ee082
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 9f8126048793e9b5760f87130cb83645f4c185ce 1d178dcfcb54760c22c6c01d99bbdb90bd8965d9 && git checkout -B drift-repro 9f8126048793e9b5760f87130cb83645f4c185ce && git merge --no-ff 1d178dcfcb54760c22c6c01d99bbdb90bd8965d9
node scripts/docs-audit/affected-docs.mjs --json 9f8126048793e9b5760f87130cb83645f4c185ce |
✅ ACCEPT, with a patch round owed: PR #23620 at
|
The stand-in erasure guard discovers every exported function in a non-test packages/verify/src file whose first parameter is a type the package declares and publishes. The new test helper warmUpBoot(opts: BootOptions) is one, and an unclassified candidate fails the run by design. It takes the same BootOptions that bootStack takes and checks nothing, so it goes in NOT_A_STAND_IN with that reason. The guard's discovery is unchanged. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FLwgbTSmGMrqsXkBDsiwfG
|
Seat note:
Generated by Claude Code |
✅ ACCEPT, round 2: PR #23620 at
|
Part of #23181
Clause-②: no
Slice 3 of the test-budget card: the
@objectstack/verifyrow, 24 files. This PR measures all 24 and changes five of them, adds one new shared test helper, and classifies that helper in one repo gate script. Nothing published changes:skip-changesetapplies by its criterion, because nothing in the package'sfiles[]moves andscripts/ships in no package.What changed
packages/verify/src/boot-warm-up.testkit.ts.warmUpBoot(opts, { requires })boots a throwaway app (no objects, its own in-memory database), signs the seeded admin in, signs one member up, and stops. A test file calls it once at module scope (await warmUpBoot(...)), so the boot's first load is paid during collection instead of inside the first clockedbeforeAllorit. That is the repo's clocked-window convention (AGENTS.md, Build & Test: "clocked windows measure behaviour, never loading"). The helper shares nothing a case asserts on.requires) of its own first boot:handle.test.ts(automation: true),automation-trigger-elevated-door.test.tsandaction-flow-elevated-door.test.ts(automation: true,requires: ['automation', 'triggers']),harness.served-composition.test.tsanddelete-behavior.test.ts(defaults).handle.test.tsalso drops one redundant arrangement.freshHandleFixtureStack()built its "configuration of its own" withvi.resetModules()and a re-import. That drops the whole module registry, so each call re-evaluated the fixture's entire graph and every lazy load of the boot after it. It now imports the fixture module alone under a specifier of its own (./handle.fixture.js?instance=N). The fixture's builder still runs once more, every nested definition is still new, and its imports resolve to the instances the file already holds. The two cases that use it went from 2045 to 2469 ms to 769 to 1010 ms.scripts/check-verify-stand-in-erasure.mjsclassifies the helper (patch round 1). The stand-in erasure guard discovers every exported function in a non-testpackages/verify/srcfile whose first parameter is a type the package declares and publishes, andwarmUpBoot(opts: BootOptions)is one. It is added toNOT_A_STAND_INwith its reason: it takes the sameBootOptionsthatbootStacktakes,index.tsdoes not re-export it, it is absent fromdist, and it checks nothing. The guard's discovery is unchanged. Unclassified,pnpm check:verify-stand-inexited 1 (reproduced at121ac4d71); classified, the guard and its--self-testexit 0 (2 guarded, 18 exempt, 10 call sites reached).Unchanged: every
it/describetitle line (sha256 identical at1a9d48813and the head for all five files) and everyexpect((77, 10, 13, 12 and 39, before and after). No timeout value moved. No test was removed or moved to another project.The row, measured before changing anything (H1)
Box: 4 vCPU, shared with sibling agents. Every run went through
os-verify-lock. "Clocked" is the file line's duration in vitest's default reporter (hooks plus tests, collection excluded), the number CI'sreport-test-timings.mjsreads. Runs hadAI_AGENTandCLAUDECODEunset, so vitest picks that reporter. "Isolated" meansvitest run --maxWorkers=2on the file alone, at1a9d48813. The whole-package run ispnpm --filter @objectstack/verify test(41 files, 362 tests, 157.4 s). The warm-up probe column is an untracked copy of each file with a plain module-scope warm-up (boot plus sign-in, default options), one run each, deleted before any commit.unique-scope.test.tsparent-write-reach.test.tsdelete-behavior.test.tshandle.test.tsharness.one-composition.test.tsreference-shape.test.tsharness.key-custody.test.tsdelete-behavior-referenceless-tree.test.tsharness.required-providers.test.tsharness.org-context.test.tsharness.served-composition.test.tsharness.app-default-profile.test.tsharness.posture-only.test.tshandle.observe-writes.test.tshandle.update-doors.test.tshandle.system-insert-delete.test.tsautomation-trigger-elevated-door.test.tsharness.host-resolution.test.tshandle.public-form-door.test.tsaction-flow-elevated-door.test.tsharness.artifact-door.test.tshandle.exemplar-deal-lifecycle.test.tserasure-transaction-authorization.test.tshandle.automation-door.test.tsThe card's readings are 2.5× to 5.1× the isolated ones (median about 3.8×), so H1 holds. Three files are over 10 s even isolated:
unique-scope(19.3 s),parent-write-reach(15.9 s) anddelete-behavior(10.4 s).Where the wall goes
A timing probe (untracked copies that time
bootStack,signIn,signUp,seed, each family'srun…Proofs, and everynode:crypto.scryptcall; deleted before any commit) read the top three files:unique-scopeparent-write-reachdelete-behaviordescribe, and every case only reads that run. Eachdescribeboots a different app, posture or database: the walled leg, the controls that seed a duplicate before the declaration arrives, and the attribution apps. A merged boot would change the reports whose summaries the cases assert. So there is no per-case redundant arrangement to share there.handle.test.ts's registry reset, described above.bootStack, isolated) and the families' proof runs. That is source in the published package (harness.ts, the kernel boot, the runners), so it is reported, not edited (Acceptance notes).Which five files. They were ranked by measured removable test-side cost:
handle(3.8 s plus the registry reset),automation-trigger-elevated-door(2.6 s),harness.served-composition(2.5 s) andaction-flow-elevated-door(2.3 s). Four files tie for fifth within 50 ms:reference-shape2242,delete-behavior2229,harness.org-context2212 anderasure2197 ms. The tie goes to the largest card reading,delete-behaviorat 46.4 s.Before and after, per changed file
Interleaved A/B, three rounds: base bytes from an untracked copy of each file at
1a9d48813(blobdf66cfeb96eeforhandle.test.ts), then the changed file, alternating, all through the lock.handle.test.tsautomation-trigger-elevated-door.test.tsaction-flow-elevated-door.test.tsharness.served-composition.test.tsdelete-behavior.test.tsThe projections multiply the median after by the file's own card ÷ isolated-before ratio. They are estimates; the card's own pin, the next refresh list, decides.
What the move costs. The warm-up runs during collection, so each changed file's collection (vitest
import) grew by 2.6 to 6.6 s isolated (median per file). Each file's whole isolated vitestDuration(collection plus clocked) moved by −1.6 s (handle.test.ts, whose registry reset was real work and is gone) to +3.9 s (the trigger door), by median of the three rounds: the warm-up's own boot, sign-in and sign-up are new work, so for four of the five files the total wall rose while the graded clocked time fell. The whole-package run went 157.4 s → 167.9 s, one run each on a shared box. The 36 unchanged files' summed file lines moved 185.6 s → 188.0 s in the same pair of runs, so the box load was similar, and most of the rise is the five warm-up boots in collection.Proof the changed files still fail when their subject breaks
There was one ablation per changed file, with the prediction written down before each run. The verify-source ablations ([A], [D], [E]) went through
scripts/ablation-replace.mjsin wrap mode. Every restore was proven: blob equals HEAD,git diff HEADempty,git status --porcelainclean.handle.test.ts. Mutation: the handle'stenancyreader (packages/verify/src/handle.ts) answers a constant single posture. Predicted: 1 red. Observed: 1 failed / 23 passed. The red case wasreports the walled posture a multi-tenant boot runs under — same reader, other stack, the case that boots the query-imported fresh fixture.automation-trigger-elevated-door.test.ts. Mutation: runtime'srefusesElevatedSelfTriggeredStart(packages/runtime/src/flow-start-admission.ts) always answers false. This suite reads runtime throughdist/, so the run was:--hold, rebuild runtime,ablation-dist-preflight(marker present in 2 built files), test, restore bygit checkout HEAD --, rebuild, preflight--absent(marker absent from all 6 built files, tree clean). Predicted: 7 red. Observed: 7 failed / 9 passed (the six REFUSED rows and the wire envelope).action-flow-elevated-door.test.ts. Same mutation and build legs as [B]: the action door asks the same predicate. Predicted: 7 red. Observed: 7 failed / 9 passed.harness.served-composition.test.ts. Mutation:harness.tsbuilds the analytics service withoutresolveCapabilityArgument('analytics', …). Predicted: 2 red. Observed: 2 failed / 5 passed (the two declared-cube cases; the no-cube control stayed green).delete-behavior.test.ts. Mutation: the family'sjudge()reads the cascade branch inverted (packages/verify/src/delete-behavior.ts). Predicted: 4 red. Observed: 4 failed / 4 passed (positive proof and summary; NEGATIVE control and its count; the derive cases, the observation case and the ATTRIBUTION control stayed green).Tests and gates
The test and typecheck readings below are at
121ac4d71, a merge oforigin/main8d05456cbinto the branch. The gate readings are at1d178dcfc, the patch head, which changes one repo gate script and nothing a@objectstack/verifytest reads.pnpm --filter @objectstack/verify test: 41 files and 362 tests passed, exit 0. In that run the five changed files read 5052, 2444, 2400, 6173 and 9756 ms (handle, the two elevated doors,served-composition,delete-behavior).pnpm --filter @objectstack/verify typecheck: exit 0.tsc --noEmitis clean, andcheck:test-typecheckis OK, with 0 entries in the debt ledger. All six changed files are in thetsconfig.test.jsonprogram (tsc --listFilesOnly: 1 hit each). The helper is also in the build config's program, and it is in no tsup entry.node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstackat1d178dcfcderived 70 commands for the 7 changed paths (the gate script brings 20 families more than the 50 the six test paths derived at121ac4d71). After a fullturbo run build(76/76 tasks, all cached) in the recreated worktree, all 70 exited 0.--ran: "70 derived, 70 run, 0 NOT-MEASURED, 0 UNRUN." At121ac4d71the 50-command derivation had also reconciled 50 of 50 (pnpm check:dual-build-cjs-loadsanswered exit 3, PREREQUISITE NOT MET, until a full build, then exit 0).origin/mainhas since moved eight commits. One of them makesdispatch-gatesnamecheck:durability-log-levelfor some files;pnpm check:durability-log-levelexits 0 at1d178dcfc.dispatch-gatesdoes not name:pnpm exec eslint packages/verify scripts/check-verify-stand-in-erasure.mjs --no-inline-configexits 0. The repo-widepnpm lintis left to CI.skip-changeset: the package'sfiles[]isdist,README.mdandCHANGELOG.md. Strings unique to this change (warmUpBoot,boot-warm-up,freshInstances) have 0 hits there (grep exit 1). The control from source (bootStackOnce) hitsdist/index.js,index.cjs, both.d.tsfiles,README.mdandCHANGELOG.md(grep exit 0).Acceptance notes
automation-trigger-elevated-door.test.tsandaction-flow-elevated-door.test.ts. Shrunk, not claimed:handle.test.ts,harness.served-composition.test.tsanddelete-behavior.test.ts. The other 19 rows are untouched here, and their numbers are in the table above.handle.observe-writes(9.4 s, borderline),handle.system-insert-delete,harness.host-resolution,handle.public-form-door,harness.artifact-door,handle.exemplar-deal-lifecycle,erasure-transaction-authorizationandhandle.automation-door. A ninth,handle.update-doors, projects at 10.0 s. These are one-run probes; a later slice measures them properly. Not filed; the report carries the choice for the seat.unique-scopeis 8 boots plus 7 proof runs, andparent-write-reachis 3 boots plus 3 proof runs that sign up 16 personas and sign in 13. That is the per-boot steady cost and the families' own behaviour, inharness.tsand the kernel boot. Observation only, not filed; the report's open question carries it.parent-write-reachpays 32 production-parameter derivations (3.3 s of 15.7 s) andunique-scopepays 19 (2.8 s of 20.6 s). Each sign-in right after its own sign-up or the admin seed re-derives the same password and salt. plugin-auth's scrypt-memo testkit answers exactly those, but it is package-private, and the seat's standing answer is one shared home, never a copy. Observation, not filed.AGENTS.mdsentence on the 10 s line gained "unless the file declares a measured budget with its class" today (merged into this branch frommain). This PR declares no budget. Whether a stack boot per single-cause control fixture reads as that ruling's first class is a reading for the seat, not for this slice.Generated by Claude Code