ci(lint): tooling self-tests run on a PR only when their own inputs change (#19498) - #19511
Conversation
…hange `Lint & Repo Gates` was the wall clock of a product PR's CI (27.4 min over 184 steps on PR 19314's head, above the longest test shard), and 18.6 of those minutes were the tooling's own self-tests, corpora and censuses. The single `PM dispatch-gates self-test` step was 11.8 min on a three-file `packages/spec` diff, because that family's read-set included the whole-tree censuses its battery runs: the content of every JS/TS and .sh file, the nested .gitignore files, and the tracked NAME set that made any ADDED path anywhere run it. Narrow that family to the tool's own inputs — the workflow tree and composite actions it discovers, every gate source it resolves under scripts/ and a workspace package's own scripts/, the package.json that names a check:* script, the agent configuration its live cases read, and root configuration — and put four more tooling steps behind the same selector with a read-set each: entry_guard 0.62 min scripts/** self_test_workflow_commands 0.48 min scripts/** + the workflow tree declared_population_live 0.40 min the derivation's read-set bare_root_worklist 0.32 min the derivation's read-set Product ratchets and censuses stay unconditional: the two engine ratchets, the tenancy and tenant-audit censuses and everything else that judges product code is untouched, as is ESLint. Steps under ~0.3 min stay unconditional too — the scripts/ shared-module self-tests (0.27) and the changeset-family gate self-tests (0.15) among them. The skip these five carry is a WEAKER claim than a ratchet's and the code and the workflow both say so: a ratchet skips only where it provably reads nothing, while a tooling self-test skips where the tool's own inputs are untouched and its battery may still read the changed path through a census. Gate weakening is a maintainer floor; the authorizing sentence is quoted where the read-sets are declared. What bounds it is unchanged: `push` to main and the hourly scheduled run take the unscoped branch and run the whole battery. The selector's self-test grows with it — the nine ids, the per-family read-sets, an ADDED path inside a read-set, the shape of PR 19314 under both `merge_group` and `pull_request`, and a `pin_step` for each new family against the real lint.yml. Claude-Session: https://claude.ai/code/session_012GcsUbuqFGBibkEDMRC1eE Co-Authored-By: Claude <noreply@anthropic.com>
os-project-manager
added
the
skip-changeset
Claude
Contract reviewIndependent at-tier contract review of PR #19511 (card #19498) at head Served-tier: ① Derived judgmentsAdversarial brief: find what is wrong with the diff. Judged item by item. 1. Scope — RIGHT. 2. The 3. The four new families — RIGHT on all four.
4. Dry runs — reproduced, all RIGHT. Run with THIS PR's selector (absolute path, (a) (b) (c) (d) (e)
(f) (g) This PR's own diff under 5. The selftest's own hazard — RIGHT with one precision. The floor is two literals, One prose inaccuracy, PR body and dev report only, no code change needed: 「The battery grew from 30 cases / 120 checks at its floor to 44 / 228」 — the prior battery was 40 cases / 192 checks (the FLOOR was 30/120). The direction of the claim holds; the number does not. 6. lint.yml header prose — RIGHT. The 7. Expected saving — RIGHT. 8. Nothing else wrong found. The removed ② Semver levelCarriers: none hung. Clause-②: no. Nothing under ③ Boundary flagsTier S — Required contexts on head
Implemented-by: VERDICT: PASS Generated by Claude Code |
|
Provenance — director seat, summon #25 (
Generated by Claude Code |
2 participants
Fixes #19498
Clause-②: no
Gate weakening is a maintainer floor. The sentence that authorizes this one, verbatim (ruling #208 on #19491, part R4):
What this changes
Lint & Repo Gatesset the wall clock of PR #19314's CI — 27.4 minutes over 184 steps for a three-filepackages/specdiff, above the longest test shard — and 18.6 of those minutes were the tooling's own self-tests, corpora and censuses. The singlePM dispatch-gates self-teststep was 11.8 of them, on a PR that changes no PM tool: that family's read-set included the whole-tree censuses its battery runs — the content of every JS/TS and.shfile, the nested.gitignorefiles, and the tracked NAME set, which made an ADDED path anywhere run it.pm_dispatch_gatesis narrowed to the tool's own inputs: the workflow tree and composite actions its discovery reads, every gate source it resolves underscripts/and a workspace package's ownscripts/, thepackage.jsonthat names acheck:*script, the agent configuration its live cases read (.claude/**,skills/**,AGENTS.md,CLAUDE.md), and root configuration. The self-test's own refusal semantics are untouched — an unreadable population still refuses.if:line inlint.yml.pushto main and the hourly scheduled run are unchanged and keep the whole battery: the selector runs everything for any event it does not scope.The census: every unconditional step at or above ~0.3 min
Measured on run 35506407130, job
Lint & Repo Gates(check-run 106066910262) at head7d67e1ee4136aee8f8e6ea838950c7fb71520be2, read step by step fromGET /repos/{owner}/{repo}/actions/jobs/106066910262. 184 steps, 27.4 min.scripts/**entry_guardself_test_workflow_commandsdeclared_population_livebare_root_worklistBelow the line, left unconditional because the list decides and not the principle:
scripts/ shared-module self-tests0.27,Cross-package test inputs0.25,Declared registry log level0.25,Platform-object tenancy census0.20,Merge-driver wiring gate0.20,Documented HTTP status matches the status the runtime emits0.17,ObjectQL double limit gate0.17,Changeset-family gate self-tests0.15. Every other step in the job measured under 0.17 min.Product ratchets and censuses stay unconditional throughout:
query_options_erasure,slot_lookup, the tenancy and tenant-audit censuses, the engine gates.The four families added, and what each reads
entry_guardpnpm check:entry-guardscripts/**— its ownROOT_DIR_WATCH_HINTS, held against the root it walks by its own self-testdeclared_population_livepnpm check:declared-population-livediscoverFamilies+trackedFiles: the workflow tree, every gate source discovery resolves, the tracked NAME set (only a name that DISAPPEARS moves its verdict, and deletions already run everything)bare_root_worklistnode scripts/pm/bare-root-worklist.mjs(self-test only)self_test_workflow_commandsnode scripts/check-self-test-workflow-commands.mjsscripts/**population of.mjs,.mtsand.shfiles, plus the workflow tree and.github/actionsit discovers the runnable self-tests fromWhat is weaker now, said out loud
A ratchet's skip says: no changed path is one this family reads. A tooling self-test's skip now says something weaker: no changed path is one the tool's own inputs name, while the battery behind it may still read that path through a whole-tree census. So a defect these five would have caught can first appear on
maininstead of on the PR that wrote it.Three things bound that, and none of them changed here: every doubt still runs everything (unresolvable base, empty diff, unclassified path, any deletion, rename or type change);
pushon main and the hourlyschedulerun the whole battery; and widening the skip further is again a maintainer call. The selector's header carries this paragraph beside the read-sets, andlint.ymlcarries it on the steps themselves — including the three steps whose prose used to say "unconditional, like every self-test around it", which this change would otherwise have made false.Two previously pinned cases are given up deliberately and are now pinned in the other direction, so the loss is legible: an ADDED file anywhere no longer runs
pm_dispatch_gates(the tracked-NAME sweep), and neither does a nested.gitignoreor a workspace.shoutsidescripts/(the #16769 case).Acceptance — the four dry runs, verbatim
(a)
pull_request, changed files = PR #19314's list. Reproduced in a throwaway detached worktree offorigin/main(c9b23cd066), driven with this branch's selector; the worktree was removed afterwards and nothing underscripts/pm/is touched by this PR.(b)
pull_request, changed filescripts/pm/dispatch-gates.mjs— same throwaway worktree:(c)
push— every family runs:(d) the selector's self-test (
pnpm check:select-gate-families), which also pins the YAML half against the reallint.yml:The battery grew from 30 cases / 120 checks at its floor to 44 / 228, and the floor moves with it (42 / 220). New cases: the nine ids in job order, an ADDED path inside a read-set, a composite action, the PR #19314 shape under both
merge_groupandpull_request, and onepin_stepper new family.For completeness, this PR's own diff under
pull_request— a change to the selector and the workflow runs all five tooling families:Expected wall clock
The five steps carry 13.62 min of the 27.4-min job (11.80 + 0.62 + 0.48 + 0.40 + 0.32). On a diff of #19314's shape all five skip and nothing else changes, so
Lint & Repo Gatesshould read about 13.8 min — arithmetic on one run's step timings, on that runner with its cache state, not a prediction of the next run. The card's bar is ≤ 16 min, and the director seat measures the real number on the first product PR after this lands.Tier S — not on the governed register
Gates
skip-changeset: nothing underpackages/**, nothing published.Every family
node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstackderives for this diff was run locally, each exit code captured before any pipe — 51 commands, 50 exit 0, includingcheck:self-test-wired,check:self-test-workflow-commands,check:step-collectors,check:declared-population-live,check:entry-guard,check:watch-hint-literal,check:required-contexts,check:workflow-status-functions,check:ci-filter-parity,check:bash32-floorandcheck:nul-bytes.pnpm check:pm-dispatch-gateswas run detached per its own header and passed:✓ dispatch-gates self-test: 1883 cases pass./the battery took 1056.1s on this box.The one command that did not return a verdict:
pnpm check:type-check-debtexits 3 = PREREQUISITE NOT MET in a fresh worktree (--re-measure cannot run: 31 workspace dependenc(ies) of the ledgered packages have no built type entry point on disk), which its own remedy text declares is neither a pass nor a finding. This diff touches no TypeScript, and CI builds the closure before that step.Generated by Claude Code