From 9a1ca2bab200fd924323fa4b2af5164641d03359 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 21 Sep 2026 04:46:22 +0000 Subject: [PATCH] ci(lint): tooling self-tests run on a PR only when their own inputs change MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `Lint & Repo Gates` was the wall clock of a product PR's CI (27.4 min over 184 steps on PR 19314's head, above the longest test shard), and 18.6 of those minutes were the tooling's own self-tests, corpora and censuses. The single `PM dispatch-gates self-test` step was 11.8 min on a three-file `packages/spec` diff, because that family's read-set included the whole-tree censuses its battery runs: the content of every JS/TS and .sh file, the nested .gitignore files, and the tracked NAME set that made any ADDED path anywhere run it. Narrow that family to the tool's own inputs — the workflow tree and composite actions it discovers, every gate source it resolves under scripts/ and a workspace package's own scripts/, the package.json that names a check:* script, the agent configuration its live cases read, and root configuration — and put four more tooling steps behind the same selector with a read-set each: entry_guard 0.62 min scripts/** self_test_workflow_commands 0.48 min scripts/** + the workflow tree declared_population_live 0.40 min the derivation's read-set bare_root_worklist 0.32 min the derivation's read-set Product ratchets and censuses stay unconditional: the two engine ratchets, the tenancy and tenant-audit censuses and everything else that judges product code is untouched, as is ESLint. Steps under ~0.3 min stay unconditional too — the scripts/ shared-module self-tests (0.27) and the changeset-family gate self-tests (0.15) among them. The skip these five carry is a WEAKER claim than a ratchet's and the code and the workflow both say so: a ratchet skips only where it provably reads nothing, while a tooling self-test skips where the tool's own inputs are untouched and its battery may still read the changed path through a census. Gate weakening is a maintainer floor; the authorizing sentence is quoted where the read-sets are declared. What bounds it is unchanged: `push` to main and the hourly scheduled run take the unscoped branch and run the whole battery. The selector's self-test grows with it — the nine ids, the per-family read-sets, an ADDED path inside a read-set, the shape of PR 19314 under both `merge_group` and `pull_request`, and a `pin_step` for each new family against the real lint.yml. Claude-Session: https://claude.ai/code/session_012GcsUbuqFGBibkEDMRC1eE Co-Authored-By: Claude --- .github/workflows/lint.yml | 139 +++++++++++++---- scripts/ci/select-gate-families.selftest.sh | 140 ++++++++++++----- scripts/ci/select-gate-families.sh | 165 +++++++++++++++----- 3 files changed, 333 insertions(+), 111 deletions(-) diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml index 5fd83442727..01a87d77814 100644 --- a/.github/workflows/lint.yml +++ b/.github/workflows/lint.yml @@ -31,9 +31,12 @@ on: # the push-on-`main` run of THIS workflow is the only post-merge full-battery # run of the families `scripts/ci/select-gate-families.sh` scopes away on # merge groups (the PM dispatch-gates self-test, both ratchets, the - # verify-lock self-test, the comment-mask corpus). A scoped family that goes - # red on `main` after a queue build skipped it had, until this trigger, no - # run that would notice and no filer that would say so. + # verify-lock self-test, the comment-mask corpus — and since #19498 the + # entry-guard sweep, the declared-population gate, the bare-root worklist + # self-test and the self-test workflow-command gate, whose whole-tree reads + # that ruling took off the PR path). A scoped family that goes red on `main` + # after a queue build skipped it had, until this trigger, no run that would + # notice and no filer that would say so. # # The selector already treats every event that is neither `merge_group` nor # `pull_request` as "run every family", so this trigger alone restores the @@ -245,18 +248,33 @@ jobs: # over the ten merge-group runs measured for #16496; the PM dispatch-gates # self-test alone 597 s), and none of its expensive steps read a # merge group's file surface: a docs-only group paid the full battery. - # So on `merge_group` and `pull_request` the FIVE scoped families below + # So on `merge_group` and `pull_request` the NINE scoped families below # -- each step carrying `if: steps.gate-families.outputs. != 'skip'` # -- run only when the changed paths touch the files that family reads. # `push` on main and the scheduled full run keep the whole battery: the # script runs everything for any event it does not scope. # + # #19498 added four of those nine (`entry_guard`, + # `declared_population_live`, `bare_root_worklist`, + # `self_test_workflow_commands`) and narrowed `pm_dispatch_gates`, on + # ruling #208 on #19491 (R4) and the maintainer's sentence quoted in the + # selector's header: the tooling's self-tests were 18.6 minutes of the + # 27.4-minute job a three-file `packages/spec` PR paid. ⛔ Product + # ratchets and censuses stay unconditional -- the tenancy and + # tenant-audit censuses, the engine gates and everything else here judge + # product code and are not in that ruling. + # # ⭐ The invariant is FAIL-OPEN, and it holds at both layers. The script # runs every family when the base cannot be resolved, the diff fails or # is empty, a path is one it does not classify (a new top-level # directory, an unlisted root file), or any change is a deletion, rename # or type change; a family is skipped ONLY when every changed path is - # positively classified into a class that family provably never reads. + # positively classified into a class that family's DECLARED read-set + # excludes. For the other four -- both ratchets, the corpus walk and the + # verify-lock self-test -- that read-set is what the family provably + # reads. For the five tooling self-tests it is the tool's OWN INPUTS, + # which is a weaker claim; it is argued where it is declared, in the + # selector's header. # The `!= 'skip'` spelling means an ABSENT output -- the selector never # ran, or wrote nothing -- also runs the step. Both halves are pinned by # `scripts/ci/select-gate-families.selftest.sh` (`check:select-gate- @@ -575,8 +593,18 @@ jobs: # `scripts/invoked-as.mjs` may read `process.argv[1]`, and that module's # own self-test drives a real probe through a real symlink. Rationale and # the rejected behavioural-sweep alternative: the gate script's header. - # Scans ~115 files, no spawns; ~0.2s. + # Scans ~115 files, no spawns; ~0.2s of work, measured at 0.62 min as a + # step on run 35506407130. + # + # Scoped (#19498): its population is `scripts/**` — its own + # ROOT_DIR_WATCH_HINTS declaration, held against the root it really walks + # by its own self-test — so a group that changes no file under scripts/ + # (and no root configuration) skips it. The selection step at the top of + # this job decides that, `push` on main and the hourly run keep it + # unconditional, and the ruling that authorizes moving a tooling + # self-test off the PR path is quoted in the selector's header. - name: scripts/ entry guards go through one predicate + if: steps.gate-families.outputs.entry_guard != 'skip' run: pnpm check:entry-guard # Every `scripts/**` TypeScript parse goes through ONE module (#10133 / @@ -911,19 +939,28 @@ jobs: # listed, with no reader of what the test actually reads. What runs # here now is different in kind and was the maintainer's call (#16496, # 「同意你的建议，你负责执行派发所有可行的优化」): the selection step at - # the top of this job classifies every changed path against this - # self-test's MEASURED read-set (every workflow, every gate source under - # `scripts/**` and `packages/*/scripts/**`, every `package.json`, - # `.claude/**`, `skills/**`, `AGENTS.md`, `CLAUDE.md`, `tsconfig.json`, - # every `.gitignore` -- nested ones included -- the CONTENT of every - # JS/TS file in the tree (the compound-anchor census of - # `function ...SelfTest...(` declarations and the exposed-scratch-dir - # sweep of every mkdtempSync/mkdirSync caller both assert over it), and - # the tracked NAME set it sweeps -- so any added, deleted or renamed - # file runs it too), and skips this step only when every path is one - # the test provably never reads: a modified doc, changeset or non-source - # workspace file. Every doubt runs it, the self-test of the selector - # pins that, and `push` on main keeps it unconditional. + # the top of this job classifies every changed path against a declared + # read-set for this step, and skips it when no changed path is in that + # set. Every doubt runs it, the self-test of the selector pins that, and + # `push` on main keeps it unconditional. + # + # ⚠️ Since #19498 that read-set is NARROWER than what the battery reads, + # and the gap is stated rather than papered over. The read-set is the + # tool's own inputs: every workflow and composite action, every gate + # source under `scripts/**` and a workspace package's own `scripts/`, + # every `package.json`, `.claude/**`, `skills/**`, `AGENTS.md`, + # `CLAUDE.md` and root configuration. The battery ALSO reads the CONTENT + # of every JS/TS and `.sh` file in the tree (the compound-anchor census + # of `function ...SelfTest...(` declarations and the exposed-scratch-dir + # sweep of every mkdtempSync/mkdirSync caller, with the nested + # `.gitignore` files it consults) and the tracked NAME set it sweeps — + # which is why, until #19498, any added file anywhere ran it. On PR + # #19314 this step alone was 11.8 minutes of a 27.4-minute job for a + # three-file `packages/spec` diff, and ruling #208 on #19491 (R4) took + # those whole-tree reads off the PR path on the maintainer's sentence, + # quoted in the selector's header. ⛔ A defect in that wider set can now + # first appear on `main`; the push-on-main and hourly full runs are what + # bound it, and widening the skip further is again a maintainer call. # # The gate runs the SELF-TEST only. The live derivation # (`node scripts/pm/dispatch-gates.mjs `) answers a question about a @@ -997,8 +1034,18 @@ jobs: # names — the stronger rule ("a gate that enumerates a directory must # declare one") was implemented, measured at 86 findings over 114 # enumerating gate files, and refused as an allowlist with a verdict - # attached. Reads the derivation once over the tracked corpus; ~5s. + # attached. Reads the derivation once over the tracked corpus; ~5s of + # work, measured at 0.40 min as a step on run 35506407130. + # + # Scoped (#19498): it imports `discoverFamilies` and `trackedFiles` from + # the dispatch derivation, so its inputs are the workflow tree, every + # gate source that discovery resolves, and the tracked NAME set — and + # only a name that DISAPPEARS can turn a live declaration dead, which is + # a structural change the selector already runs everything for. A group + # confined to product source, tests, docs or changesets skips it here and + # pays for it on `push` to main and on the hourly run. - name: A declared gate population reaches the tree + if: steps.gate-families.outputs.declared_population_live != 'skip' run: pnpm check:declared-population-live # ADR-0087 D4's per-release correctness gate (#17080). The REAL run needs @@ -1029,21 +1076,32 @@ jobs: # `dispatch-gates.mjs` reads the worklist, and no verdict in it reaches a # dispatch prompt. # - # Unconditional, for the same reason as the step above: a self-test that - # can be skipped is the gap moving rather than closing. Reads the workflow - # tree and every gate source once; ~0.5s. + # Reads the workflow tree and every gate source once; ~0.5s of work, + # measured at 0.32 min as a step on run 35506407130. + # + # ⚠️ This step WAS unconditional, on the reasoning that a self-test which + # can be skipped is the gap moving rather than closing. #19498 scopes it + # anyway, and the trade is explicit rather than reasoned away: it imports + # the same derivation as the two steps above, so its inputs are the + # workflow tree and the gate sources, and on a group touching neither the + # gap does move — to `push` on main and the hourly full run, which keep + # the whole battery. Gate weakening is a maintainer floor; the sentence + # that authorizes this one is quoted in the selector's header. - name: PM bare-root worklist self-test + if: steps.gate-families.outputs.bare_root_worklist != 'skip' run: node scripts/pm/bare-root-worklist.mjs --self-test # Part-of/closing-keyword guard self-test (#8476). The guard itself is a # PR-scoped blocking check in its own workflow — it needs a pull request # body to judge, which this job does not have — so what runs HERE is its # self-test, which is the half with a verdict independent of any PR. - # Unconditional for the same reason as the two steps above: a self-test - # that runs only when someone remembers is a check whose coverage is a - # function of who remembered, and the failure it hides is quiet — a break - # in the verdict layer lands green and surfaces later as a card silently - # closed by the sentence written to keep it open. + # Unconditional, and not on the two steps above's borrowed reason — they + # are scoped since #19498 and this one is not, because the cost that + # bought that ruling is not here: this step is a tenth of a second, and a + # self-test that runs only when someone remembers is a check whose + # coverage is a function of who remembered. The failure it hides is quiet + # — a break in the verdict layer lands green and surfaces later as a card + # silently closed by the sentence written to keep it open. # # The self-test also pins the WIRING (the guard workflow still invokes # the script, still subscribes to `edited`, still passes the body through @@ -1727,8 +1785,17 @@ jobs: # Invoked as `node scripts/…` rather than through a `pnpm check:*` alias: # see the GATE INVOCATION IDIOM note at the top of this file. Reads # `scripts/` and `.github/workflows/` off disk and spawns the selected - # self-tests; no network. + # self-tests; no network. Measured at 0.48 min as a step on run + # 35506407130. + # + # Scoped (#19498): its declared population is + # `scripts/**/*.mjs`, `scripts/**/*.mts` and `scripts/**/*.sh`, and the + # rest of its read-set is the workflow tree and `.github/actions` it + # discovers the runnable self-tests from — so a group touching none of + # those skips it here and runs it on `push` to main and on the hourly + # full run. - name: Self-test workflow-command gate + if: steps.gate-families.outputs.self_test_workflow_commands != 'skip' run: | node scripts/check-self-test-workflow-commands.mjs --self-test node scripts/check-self-test-workflow-commands.mjs @@ -1806,8 +1873,11 @@ jobs: # silently degrading every gate failure from "here is the command" into # "no substitute available". # - # Unconditional and un-`if:`-ed, like every self-test above it — an - # exemption is precisely what a self-test must not have, or the gap moves. + # Unconditional and un-`if:`-ed — an exemption is precisely what a + # self-test must not have, or the gap moves. (Five self-tests in this job + # do carry one since #19498: each cost minutes on every product PR, and + # each was moved by the ruling quoted in the selector's header. This one + # costs seconds and is not among them.) # # NO NETWORK, measured rather than assumed (#9898), because a self-test # that reached GitHub would put this required context at the mercy of API @@ -1848,9 +1918,10 @@ jobs: # future edit invalidates silently and precisely the rows no reviewer reads. # Nothing but this step is an instrument for them. # - # Unconditional and un-`if:`-ed, like every self-test around it — an - # exemption is precisely what a self-test must not have, or the gap simply - # moves. One `--self-test` per `run:` block, deliberately: the masking shape + # Unconditional and un-`if:`-ed — an exemption is precisely what a + # self-test must not have, or the gap simply moves; the five self-tests + # scoped in #19498 are the ruled exception and this sub-second one is not + # among them. One `--self-test` per `run:` block, deliberately: the masking shape # `check-step-collectors.mjs` guards is a block driving TWO OR MORE distinct # scripts. A discovery collector over `scripts/pm/*.sh --self-test` — which # would also catch the next such script arriving unwired — is ruled out of diff --git a/scripts/ci/select-gate-families.selftest.sh b/scripts/ci/select-gate-families.selftest.sh index ae75c1548ce..ddc3c45e4be 100644 --- a/scripts/ci/select-gate-families.selftest.sh +++ b/scripts/ci/select-gate-families.selftest.sh @@ -13,6 +13,13 @@ # branches around them: empty diff, unresolvable base, structural change, # unspellable path, an event that is not scoped. # +# Since #19498 the same section covers the four families that ruling added -- +# entry_guard, declared_population_live, bare_root_worklist and +# self_test_workflow_commands -- and the cases below pin the NARROWED +# pm_dispatch_gates read-set: the tree shape PR #19314 landed (a spec source +# edit, an added test, an added changeset) must skip every tooling self-test, +# while the tool's own inputs must still run them. +# # The last section reads the REAL lint.yml and pins the YAML half of the # contract: the selector step exists under the id the `if:` lines name, every # scoped step spells `!= 'skip'` (an absent output runs the step), and the set @@ -50,7 +57,7 @@ git_q() { git -c user.name=selftest -c user.email=selftest@example.invalid -c commit.gpgsign=false "$@" } -ALL='pm_dispatch_gates query_options_erasure slot_lookup verify_lock comment_mask_corpus' +ALL='slot_lookup query_options_erasure entry_guard comment_mask_corpus pm_dispatch_gates declared_population_live bare_root_worklist self_test_workflow_commands verify_lock' # ── Fixture repositories ──────────────────────────────────────────────────── # C0 carries one representative file of every class the classifier names, so @@ -247,7 +254,7 @@ cases=$((cases + 1)); RT="$FIX/rt-$cases"; mkdir -p "$RT" rc=$? echo "case: --families prints the family ids in job order" expect_rc 0 -expect_file_is 'the five ids' "$RT/out.txt" "$(printf '%s\n' $ALL)" +expect_file_is 'the nine ids' "$RT/out.txt" "$(printf '%s\n' $ALL)" # ── events that are not scoped ────────────────────────────────────────────── S=$(scenario M:docs/guide.md) @@ -256,7 +263,9 @@ expect_rc 0 expect_warnings '' '' expect_all_run expect_reason pm_dispatch_gates "event 'push' is not scoped" -expect_line 'Gate families: 5 run, 0 skipped' +expect_reason entry_guard "event 'push' is not scoped" +expect_reason bare_root_worklist "event 'push' is not scoped" +expect_line 'Gate families: 9 run, 0 skipped' run_case 'schedule: every family runs (the hourly full run keeps the battery)' "$REPO" schedule '' '' expect_rc 0 @@ -274,21 +283,29 @@ expect_all_run # ── merge_group: the card's four cases ────────────────────────────────────── S=$(scenario M:scripts/pm/tool.mjs) -run_case 'merge_group: a scripts/pm change runs the PM dispatch-gates self-test (and, for a .mjs, the corpus walk)' "$REPO" merge_group '' "$C0" +run_case 'merge_group: a scripts/pm change runs every tooling self-test (and, for a .mjs, the corpus walk)' "$REPO" merge_group '' "$C0" expect_rc 0 expect_warnings '' '' -expect_verdicts pm_dispatch_gates comment_mask_corpus +expect_verdicts entry_guard comment_mask_corpus pm_dispatch_gates declared_population_live bare_root_worklist self_test_workflow_commands expect_reason pm_dispatch_gates 'scripts/pm/tool.mjs (M, scripts)' +expect_reason self_test_workflow_commands 'scripts/pm/tool.mjs (M, scripts)' expect_changed 'one M' "M scripts/pm/tool.mjs" expect_line "Gate-family diff base: $C0 (the merge group's base_sha)" -expect_line 'Gate families: 2 run, 3 skipped' +expect_line 'Gate families: 6 run, 3 skipped' + +S=$(scenario A:scripts/pm/new-tool.mjs) +run_case 'merge_group: an ADDED file inside the read-set still runs it -- the narrowing is about the class, never the status' "$REPO" merge_group '' "$C0" +expect_rc 0 +expect_warnings '' '' +expect_verdicts entry_guard comment_mask_corpus pm_dispatch_gates declared_population_live bare_root_worklist self_test_workflow_commands +expect_reason pm_dispatch_gates 'scripts/pm/new-tool.mjs (A, scripts)' S=$(scenario M:scripts/pm/README.md) -run_case 'merge_group: a scripts/pm prose change runs the PM dispatch-gates self-test alone' "$REPO" merge_group '' "$C0" +run_case 'merge_group: a scripts/pm prose change runs the tooling self-tests, and no ratchet' "$REPO" merge_group '' "$C0" expect_rc 0 expect_warnings '' '' -expect_verdicts pm_dispatch_gates -expect_line 'Gate families: 1 run, 4 skipped' +expect_verdicts entry_guard pm_dispatch_gates declared_population_live bare_root_worklist self_test_workflow_commands +expect_line 'Gate families: 5 run, 4 skipped' S=$(scenario M:docs/guide.md M:content/docs/page.mdx M:.changeset/first.md M:README.md) run_case 'merge_group: a docs-only group skips every family, and prints it' "$REPO" merge_group '' "$C0" @@ -296,21 +313,31 @@ expect_rc 0 expect_warnings '' '' expect_verdicts expect_reason pm_dispatch_gates 'no changed path is in its read-set' -expect_line 'Gate families: 0 run, 5 skipped' +expect_reason entry_guard 'no changed path is in its read-set' +expect_line 'Gate families: 0 run, 9 skipped' expect_line 'skip pm_dispatch_gates' expect_line 'skip comment_mask_corpus' -if grep -q '^| `pm_dispatch_gates` | skip |' "$RT/step-summary" && grep -q '^## Gate families: 0 run, 5 skipped' "$RT/step-summary"; then +expect_line 'skip self_test_workflow_commands' +if grep -q '^| `pm_dispatch_gates` | skip |' "$RT/step-summary" && grep -q '^## Gate families: 0 run, 9 skipped' "$RT/step-summary"; then ok 'the step summary lists the skipped families' else bad 'the step summary lists the skipped families' "$(tr '\n' ' ' < "$RT/step-summary")" fi S=$(scenario M:.github/workflows/lint.yml) -run_case 'merge_group: a workflow change runs the PM dispatch-gates self-test' "$REPO" merge_group '' "$C0" +run_case 'merge_group: a workflow change runs every family built on the dispatch derivation, and not the scripts-only sweep' "$REPO" merge_group '' "$C0" expect_rc 0 expect_warnings '' '' -expect_verdicts pm_dispatch_gates +expect_verdicts pm_dispatch_gates declared_population_live bare_root_worklist self_test_workflow_commands expect_reason pm_dispatch_gates '(M, workflow)' +expect_reason declared_population_live '(M, workflow)' + +S=$(scenario A:.github/actions/setup/action.yml) +run_case 'merge_group: a composite action is part of the workflow tree the derivation discovers' "$REPO" merge_group '' "$C0" +expect_rc 0 +expect_warnings '' '' +expect_verdicts pm_dispatch_gates declared_population_live bare_root_worklist self_test_workflow_commands +expect_reason self_test_workflow_commands '.github/actions/setup/action.yml (A, workflow)' S=$(scenario M:docs/guide.md A:brand-new-dir/thing.txt) run_case 'merge_group: an UNKNOWN path runs every family' "$REPO" merge_group '' "$C0" @@ -342,11 +369,11 @@ expect_changed 'D then A, no R record' "A docs/guide-renamed.md D docs/guide.md" S=$(scenario A:docs/new-page.md) -run_case 'merge_group: an ADDED docs file runs the name-sweeping family only' "$REPO" merge_group '' "$C0" +run_case 'merge_group: an ADDED docs file runs nothing -- the tracked-NAME sweep left the PR path with #19498' "$REPO" merge_group '' "$C0" expect_rc 0 expect_warnings '' '' -expect_verdicts pm_dispatch_gates -expect_reason pm_dispatch_gates 'docs/new-page.md (A, docs)' +expect_verdicts +expect_reason pm_dispatch_gates 'no changed path is in its read-set' git_q -C "$REPO" checkout -q --detach "$C0" run_case 'merge_group: an EMPTY diff runs every family rather than selecting nothing' "$REPO" merge_group '' "$C0" @@ -397,31 +424,43 @@ expect_all_run # ── merge_group: the per-family read-sets ────────────────────────────────── S=$(scenario M:packages/a/src/index.ts) -run_case 'merge_group: a packages TS edit runs both ratchets, the corpus AND the PM self-test (it reads every source), not the lock self-test' "$REPO" merge_group '' "$C0" +run_case 'merge_group: a packages TS edit runs both ratchets and the corpus -- and, since #19498, no tooling self-test' "$REPO" merge_group '' "$C0" expect_rc 0 expect_warnings '' '' -expect_verdicts pm_dispatch_gates query_options_erasure slot_lookup comment_mask_corpus +expect_verdicts query_options_erasure slot_lookup comment_mask_corpus expect_reason query_options_erasure '(M, workspace)' expect_reason comment_mask_corpus packages/a/src/index.ts -expect_reason pm_dispatch_gates packages/a/src/index.ts +expect_reason pm_dispatch_gates 'no changed path is in its read-set' + +S=$(scenario M:packages/a/src/index.ts A:packages/a/src/arm.test.ts A:.changeset/second.md) +run_case 'merge_group: the PR #19314 shape (a spec source edit, an added test, an added changeset) pays the ratchets and the corpus, and no tooling self-test' "$REPO" merge_group '' "$C0" +expect_rc 0 +expect_warnings '' '' +expect_verdicts query_options_erasure slot_lookup comment_mask_corpus +expect_reason pm_dispatch_gates 'no changed path is in its read-set' +expect_reason declared_population_live 'no changed path is in its read-set' +expect_reason bare_root_worklist 'no changed path is in its read-set' +expect_reason self_test_workflow_commands 'no changed path is in its read-set' +expect_reason entry_guard 'no changed path is in its read-set' +expect_line 'Gate families: 3 run, 6 skipped' S=$(scenario M:apps/site/src/page.tsx) -run_case 'merge_group: an apps TSX edit is outside the ratchets (packages/** only) but inside the corpus and the PM census' "$REPO" merge_group '' "$C0" +run_case 'merge_group: an apps TSX edit is outside the ratchets (packages/** only) and inside the corpus alone' "$REPO" merge_group '' "$C0" expect_rc 0 -expect_verdicts pm_dispatch_gates comment_mask_corpus +expect_verdicts comment_mask_corpus S=$(scenario M:packages/a/package.json) -run_case 'merge_group: a package manifest runs the PM self-test and the lock self-test (workspace enumeration), no ratchet' "$REPO" merge_group '' "$C0" +run_case 'merge_group: a package manifest is how the derivation resolves a check:* script, so it runs the derivation families and the lock self-test, no ratchet' "$REPO" merge_group '' "$C0" expect_rc 0 -expect_verdicts pm_dispatch_gates verify_lock +expect_verdicts pm_dispatch_gates declared_population_live bare_root_worklist self_test_workflow_commands verify_lock expect_reason pm_dispatch_gates packages/a/package.json expect_reason verify_lock packages/a/package.json S=$(scenario M:packages/a/.gitignore) -run_case 'merge_group: a nested .gitignore runs the PM self-test (exposed-scratch-dir sweep) alone' "$REPO" merge_group '' "$C0" +run_case 'merge_group: a nested .gitignore is read by the exposed-scratch-dir sweep, which #19498 took off the PR path' "$REPO" merge_group '' "$C0" expect_rc 0 -expect_verdicts pm_dispatch_gates -expect_reason pm_dispatch_gates packages/a/.gitignore +expect_verdicts +expect_reason pm_dispatch_gates 'no changed path is in its read-set' S=$(scenario M:packages/a/src/data.json) run_case 'merge_group: a non-source, non-manifest workspace file skips every family' "$REPO" merge_group '' "$C0" @@ -430,21 +469,21 @@ expect_warnings '' '' expect_verdicts S=$(scenario M:packages/a/foo.sh) -run_case 'merge_group: a workspace shell script outside scripts/ is still gate source, not a skipped non-source workspace file (#16769)' "$REPO" merge_group '' "$C0" +run_case 'merge_group: a workspace shell script outside scripts/ was gate source for the whole-tree census (#16769) and is outside the narrowed read-set (#19498)' "$REPO" merge_group '' "$C0" expect_rc 0 expect_warnings '' '' -expect_verdicts pm_dispatch_gates -expect_reason pm_dispatch_gates packages/a/foo.sh +expect_verdicts +expect_reason pm_dispatch_gates 'no changed path is in its read-set' S=$(scenario M:packages/a/scripts/build.mjs) -run_case 'merge_group: a package-local script is a gate source (PM) and a masked source (corpus)' "$REPO" merge_group '' "$C0" +run_case 'merge_group: a package-local script is a gate source (the derivation families) and a masked source (corpus)' "$REPO" merge_group '' "$C0" expect_rc 0 -expect_verdicts pm_dispatch_gates comment_mask_corpus +expect_verdicts comment_mask_corpus pm_dispatch_gates declared_population_live bare_root_worklist self_test_workflow_commands S=$(scenario M:scripts/pm/os-verify-lock.sh) -run_case 'merge_group: the lock script runs its own self-test and the PM self-test' "$REPO" merge_group '' "$C0" +run_case 'merge_group: the lock script runs its own self-test and every family that reads scripts/' "$REPO" merge_group '' "$C0" expect_rc 0 -expect_verdicts pm_dispatch_gates verify_lock +expect_verdicts entry_guard pm_dispatch_gates declared_population_live bare_root_worklist self_test_workflow_commands verify_lock expect_reason verify_lock '(M, verify-lock)' S=$(scenario M:scripts/helper.mjs) @@ -454,17 +493,17 @@ expect_all_run expect_reason verify_lock scripts/helper.mjs S=$(scenario M:scripts/slot-lookup-baseline.json) -run_case 'merge_group: a ratchet baseline runs the ratchets and the PM self-test' "$REPO" merge_group '' "$C0" +run_case 'merge_group: a ratchet baseline runs the ratchets and every family that reads scripts/' "$REPO" merge_group '' "$C0" expect_rc 0 -expect_verdicts pm_dispatch_gates query_options_erasure slot_lookup +expect_verdicts slot_lookup query_options_erasure entry_guard pm_dispatch_gates declared_population_live bare_root_worklist self_test_workflow_commands S=$(scenario M:scripts/ci/tool.sh) run_case 'merge_group: a scripts/ subdirectory script is a gate source only' "$REPO" merge_group '' "$C0" expect_rc 0 -expect_verdicts pm_dispatch_gates +expect_verdicts entry_guard pm_dispatch_gates declared_population_live bare_root_worklist self_test_workflow_commands S=$(scenario M:.claude/agents/os-dev.md M:skills/x/SKILL.md M:AGENTS.md) -run_case 'merge_group: agent configuration runs the PM self-test only' "$REPO" merge_group '' "$C0" +run_case 'merge_group: agent configuration runs the PM self-test alone -- it is the only battery here that reads it' "$REPO" merge_group '' "$C0" expect_rc 0 expect_verdicts pm_dispatch_gates @@ -483,8 +522,8 @@ expect_all_run S=$(scenario M:docs/guide.md M:packages/a/src/index.test.ts M:.github/workflows/lint.yml) run_case 'merge_group: a mixed group runs the union of what its paths reach' "$REPO" merge_group '' "$C0" expect_rc 0 -expect_verdicts pm_dispatch_gates query_options_erasure slot_lookup comment_mask_corpus -expect_line 'Gate families: 4 run, 1 skipped' +expect_verdicts slot_lookup query_options_erasure comment_mask_corpus pm_dispatch_gates declared_population_live bare_root_worklist self_test_workflow_commands +expect_line 'Gate families: 7 run, 2 skipped' # ── pull_request ──────────────────────────────────────────────────────────── git_q -C "$REPO" checkout -q -B feature "$C0" @@ -498,6 +537,21 @@ expect_verdicts expect_line "Gate-family diff base: $C0 (merge-base of origin/main and HEAD)" expect_changed 'the feature edit only, not C1' "M docs/guide.md" +git_q -C "$REPO" checkout -q -B feature-19314 "$C0" +printf 'export const a = 2;\n' > "$REPO/packages/a/src/index.ts" +mkdir -p "$REPO/packages/a/src" +printf 'export const t = 2;\n' > "$REPO/packages/a/src/arm.test.ts" +printf -- '---\n"a": patch\n---\nthe arm\n' > "$REPO/.changeset/arm.md" +git_q -C "$REPO" add -A +git_q -C "$REPO" commit -q -m 'F2: the PR #19314 shape on a feature branch' +run_case 'pull_request: the PR #19314 shape pays the two ratchets and the corpus, and no tooling self-test (the measurement this narrowing was ruled from)' "$REPO" pull_request main '' +expect_rc 0 +expect_warnings '' '' +expect_verdicts slot_lookup query_options_erasure comment_mask_corpus +expect_reason pm_dispatch_gates 'no changed path is in its read-set' +expect_line 'Gate families: 3 run, 6 skipped' +git_q -C "$REPO" checkout -q -B feature "$F1" + run_case 'pull_request: no base branch in the payload' "$REPO" pull_request '' '' expect_rc 0 expect_warnings 'no base' '::warning::This pull_request event carries no base branch, so the gate-family diff base cannot be computed; every family runs.' @@ -575,6 +629,12 @@ pin_step() { pin_step pm_dispatch_gates 'pnpm check:pm-dispatch-gates' pin_step query_options_erasure 'pnpm check:query-options-erasure' pin_step slot_lookup 'pnpm check:slot-lookup' +pin_step entry_guard 'pnpm check:entry-guard' +pin_step declared_population_live 'pnpm check:declared-population-live' +# The two below are pinned by the script path alone, for the reason spelled at +# verify_lock: this file must not carry another script's self-test flag. +pin_step bare_root_worklist 'node scripts/pm/bare-root-worklist.mjs' +pin_step self_test_workflow_commands 'node scripts/check-self-test-workflow-commands.mjs' # Pinned by the script name alone: spelling the flag that step passes in code # here would read, to check-self-test-wired, as a self-test flag of THIS file # that no workflow passes. @@ -583,7 +643,7 @@ pin_step comment_mask_corpus 'node scripts/check-comment-mask-corpus.mjs' # ── Verdict ───────────────────────────────────────────────────────────────── # #4690: a battery that ran nothing is a failure, never a pass. -if [ "$cases" -lt 30 ] || [ "$checks" -lt 120 ]; then +if [ "$cases" -lt 42 ] || [ "$checks" -lt 220 ]; then echo "SELFTEST FAILED: only $cases case(s) / $checks check(s) ran -- the battery is short" exit 1 fi diff --git a/scripts/ci/select-gate-families.sh b/scripts/ci/select-gate-families.sh index b260018ab6e..0f26db27f88 100644 --- a/scripts/ci/select-gate-families.sh +++ b/scripts/ci/select-gate-families.sh @@ -51,27 +51,27 @@ # nothing -- also runs the step. The YAML half of that contract is pinned by # the self-test, which reads lint.yml and refuses any other spelling. # -# pm_dispatch_gates `pnpm check:pm-dispatch-gates`. Its self-test -# discovers every workflow file, resolves every -# check:* script through the root and package +# pm_dispatch_gates `pnpm check:pm-dispatch-gates`, which spawns +# `scripts/pm/dispatch-gates.mjs --self-test`. Its +# self-test discovers every workflow file, resolves +# every check:* script through the root and package # `package.json`s, reads each gate's source for its # watch hints (so ALL of scripts/** and every -# packages/*/scripts/**), reads .claude/**, +# packages/*/scripts/**), and reads .claude/**, # skills/** (the frame-sync COPIES table), -# `AGENTS.md`, `CLAUDE.md`, `tsconfig.json`, -# .gitignore, and sweeps `git ls-files` for hint -# reachability and test-file residue. It also reads -# the CONTENT of every JS/TS and shell (`.sh`) file in -# the tree: the compound-anchor census of -# `function ...SelfTest...(` declarations asserts none -# is unlisted, and the exposed-scratch-dir sweep reads -# every mkdtempSync/mkdirSync caller and consults -# nested .gitignore files. So any masked source file, -# any `.sh` file, and any .gitignore runs it, and -# because the name sweep reads the tracked NAME set an -# ADDED file anywhere runs it too; only modifications -# of docs, changesets and non-source workspace files -# that are neither a manifest nor a script skip it. +# `AGENTS.md`, `CLAUDE.md`. Those are the tool's OWN +# inputs and they are this family's read-set. +# ⛔ The battery reads MORE than that -- the CONTENT +# of every JS/TS and shell (`.sh`) file in the tree +# (the compound-anchor census of +# `function ...SelfTest...(` declarations, the +# exposed-scratch-dir sweep of every +# mkdtempSync/mkdirSync caller and the nested +# .gitignore files it consults) and the tracked NAME +# set (hint reachability, test-file residue), which is +# why an ADDED path anywhere used to run it. Those +# reads are real and they are DELIBERATELY off the PR +# path: see "The self-test families" below. # query_options_erasure `pnpm check:query-options-erasure`. Lints # packages/**/*.{ts,tsx,mts,cts} under # `eslint.config.mjs`, reads its baseline @@ -91,12 +91,68 @@ # comment_mask_corpus `node scripts/check-comment-mask-corpus.mjs`. Walks # every `.ts .tsx .mts .cts .js .mjs .cjs .jsx` file # in the tree outside build directories. +# entry_guard `pnpm check:entry-guard`. Walks scripts/** -- its +# `ROOT_DIR_WATCH_HINTS` declaration, held against the +# root it really walks by its own self-test -- and +# reads nothing outside it: no workflow, no workspace +# package, no docs. +# declared_population_live +# `pnpm check:declared-population-live`. Imports +# `discoverFamilies` and `trackedFiles` from the +# dispatch derivation, so it reads the workflow tree, +# every gate source that discovery resolves (scripts/**, +# a workspace package's own scripts/ and the +# `package.json` that names its check:* script) and the +# tracked NAME set. Only a name that DISAPPEARS can +# move its verdict from live to refused, and every +# deletion runs every family already. +# bare_root_worklist `node scripts/pm/bare-root-worklist.mjs` with its +# self-test flag -- the self-test only, never the sweep +# as a verdict. Imports the same derivation and reads +# the same set. +# self_test_workflow_commands +# `node scripts/check-self-test-workflow-commands.mjs` +# and its self-test. Declares +# scripts/**/*.mjs, scripts/**/*.mts and scripts/**/*.sh +# as its population, reads the workflow tree and +# .github/actions to learn which self-tests CI runs, +# and spawns those self-tests. # # Root configuration (`package.json`, `pnpm-lock.yaml`, `pnpm-workspace.yaml`, # `turbo.json`, `tsconfig.json`, `eslint.config.mjs`, .gitignore, # .gitattributes, .npmrc, .nvmrc) runs every family: a parser or # dependency bump moves all of them. ESLint itself is not scoped (card # ruling 2), nor is any step whose reads this file cannot name. + +# ## The self-test families are POLICY-scoped, not read-set-scoped (#19498) +# +# Five families above judge the TOOLING rather than the product -- +# pm_dispatch_gates, entry_guard, declared_population_live, bare_root_worklist +# and self_test_workflow_commands -- and their skip is a WEAKER claim than a +# ratchet's. A ratchet's skip says: no changed path is one this family reads. +# A self-test family's skip says: no changed path is one the TOOL'S OWN INPUTS +# name, while the battery behind it may still read that path through a +# whole-tree census. The difference is concentrated in pm_dispatch_gates, whose +# census reads every source file in the tree and whose name sweep read every +# ADDED path. +# +# That weakening is a maintainer call and it was taken: ruling #208 on #19491, +# part R4, under the maintainer's verbatim sentence +# +# 19491 接受你的建议，并立刻派发处理相关任务。 +# +# on the measurement it was ruled from -- on PR #19314 (a three-file +# `packages/spec` diff) the `Lint & Repo Gates` job ran 27.4 minutes, 18.6 of +# them self-tests, ratchets, corpora and censuses of the tooling, and the +# single `PM dispatch-gates self-test` step 11.8 of those. +# +# ⛔ So a defect these families would have caught can now first appear on +# `main` instead of on the PR that wrote it. What keeps that bounded is the +# unscoped half, unchanged by this: `push` on main and the hourly scheduled run +# take the `*)` branch above and run the whole battery, and lint.yml's `on:` +# block owns that argument. Widening this set further -- moving another step +# behind the selector, or making one of these read-sets smaller -- is again a +# maintainer call, taken here, under this script's self-test. # # ## The interface # @@ -116,7 +172,7 @@ set -euo pipefail # The family ids, in the order the job runs them. `--families` prints them so # the self-test can pin the workflow's `if:` set against this list without a # second transcription. -FAMILIES='pm_dispatch_gates query_options_erasure slot_lookup verify_lock comment_mask_corpus' +FAMILIES='slot_lookup query_options_erasure entry_guard comment_mask_corpus pm_dispatch_gates declared_population_live bare_root_worklist self_test_workflow_commands verify_lock' if [ "${1:-}" = '--families' ]; then for id in $FAMILIES; do echo "$id"; done @@ -302,6 +358,30 @@ is_masked_source() { return 1 } +# reads_gate_tree -- exit 0 when this change is inside the +# read-set the dispatch derivation shares with the gates built on it: the +# workflow tree it discovers (.github/**, workflows and composite actions), +# and every gate source it resolves and reads for watch hints (scripts/** at +# any depth, a workspace package's own scripts/, and the package.json that +# names its check:* script). Docs, changesets, agent configuration and every +# other workspace file are outside it. Four families share this helper rather +# than carry four transcriptions of one read-set; each names its own extra +# arms at its `case` below. +reads_gate_tree() { + local path=$1 class=$2 + case "$class" in + workflow|scripts|verify-lock) return 0 ;; + docs|changeset|agent-config) return 1 ;; + workspace) + case "$path" in + */package.json|*/scripts/*) return 0 ;; + *) return 1 ;; + esac + ;; + esac + return 0 +} + # family_reads -- exit 0 when the family must # run for this change. The *) arm of every `case` is "run": nothing here # skips by omission. @@ -310,25 +390,36 @@ family_reads() { case "$class" in unknown|root-config) return 0 ;; esac case "$id" in pm_dispatch_gates) - [ "$status" = M ] || return 0 - # The self-test reads the CONTENT of every JS/TS file in the tree (the - # compound-anchor census of `function ...SelfTest...(` declarations, the - # exposed-scratch-dir sweep of every mkdtempSync/mkdirSync caller) and - # every tracked `.sh` file (the same watch-hint extraction, run through - # the comment mask), and consults nested .gitignore files, so any masked - # source, any `.sh` file, and any .gitignore runs it whatever class it - # sits in. - is_masked_source "$path" && return 0 - case "$path" in *.sh) return 0 ;; esac - case "$path" in */.gitignore) return 0 ;; esac + # NARROWED to the tool's own inputs (#19498). What it keeps: the + # derivation's own read-set above, plus the agent configuration the + # battery's live cases read (.claude/**, skills/**, AGENTS.md, + # CLAUDE.md). + # ⛔ What it gives up, on the ruling this script's header quotes: the + # whole-tree censuses -- every masked source and every `.sh` file read + # for the compound-anchor sweep, the nested .gitignore files the + # exposed-scratch-dir sweep consults, and the tracked NAME sweep that + # made an ADDED path anywhere run it. Those reads are real; they run on + # push-on-main and on the hourly full run, and not on a PR. + case "$class" in agent-config) return 0 ;; esac + if reads_gate_tree "$path" "$class"; then return 0; fi + return 1 + ;; + declared_population_live|bare_root_worklist|self_test_workflow_commands) + # All three import the derivation (discoverFamilies) and judge what it + # finds, so the shared read-set above is theirs exactly. + # declared_population_live also reads the tracked NAME set, where only a + # name that DISAPPEARS can turn a live declaration dead -- and a deletion + # has already run every family before this point. + if reads_gate_tree "$path" "$class"; then return 0; fi + return 1 + ;; + entry_guard) + # Walks scripts/** and nothing else (its ROOT_DIR_WATCH_HINTS, held + # against the root it really walks by its own self-test), so a workflow, + # a workspace package and the agent configuration are all outside it. case "$class" in - docs|changeset) return 1 ;; - workspace) - case "$path" in - */package.json|*/scripts/*) return 0 ;; - *) return 1 ;; - esac - ;; + scripts|verify-lock) return 0 ;; + docs|changeset|workflow|agent-config|workspace) return 1 ;; *) return 0 ;; esac ;;