Skip to content

ci(changeset-gate): a level-axis reading that did not happen no longer concludes success, and the PR body is re-read on edited - #16897

Merged
baozhoutao merged 1 commit into
mainfrom
claude/issue-16776-changeset-level-axis-unmeasured
Sep 8, 2026
Merged

ci(changeset-gate): a level-axis reading that did not happen no longer concludes success, and the PR body is re-read on edited#16897
baozhoutao merged 1 commit into
mainfrom
claude/issue-16776-changeset-level-axis-unmeasured

Conversation

@claude

@claude claude Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Part of #16776 — the two defects the card carries. It does NOT close the card: this PR lands the gate half, and the card's second direction for defect B (the declaration written at PR-creation time) reaches .claude/agents/os-dev.md, a governed surface no seat here may land. See "What is deliberately NOT here".

⛔ Not addressed here: 16361, the same gate's opposite direction (a false red from predicate scope). Untouched by this diff, held by the PM seat behind this PR.

The defect, re-driven before it was fixed

The card's Boundaries block says every reading in it is second-hand and was not re-driven by triage. It was re-driven here, on this branch's base, with the real script and real --event payloads, on the very commit the card names (273247e56f, PR 16342's pre-fix head — @objectstack/spec: patch and @objectstack/runtime: patch beside a diff that moves both packages' src/**).

--event payload exit BEFORE exit AFTER what the level axis says now
Clause-②: yes in the body, no carrier 1 1 enforce (unchanged)
carrier label on, no body line — firing control 1 1 enforce (unchanged) — the instrument fires through either carrier on this same tree
neither 0 1 ⛔ NOT MEASURED, and it is the one reading this PR needed
Clause-②: probablynonsense control 0 1 malformed is still not a declaration, and no longer a silent pass
Clause-②: no 0 0 the explicit opt-out, still a pass on the offending tree
fixed head (cb5d140d70, one word: minor), neither carrier 0 0 NOT MEASURED, and it could not have changed this verdict
no --event at all (the RC-cut / local lane) 0 0 NOT APPLICABLE — no pull request to read a declaration from
pull_request run whose payload will not parse 0 1 the runner writes that file; a run that cannot read it has verified nothing

The firing control is what makes row 3 readable: rows 1 and 2 refuse the byte-identical tree, so row 3's green was about the declaration and never about the diff.

Correction to one inherited reading, re-derived from the file rather than the card: the card and its source comment both say the Check Changeset job "skips label events". On this base it does not — only Check PR Size and Auto Label carry that exclusion. Nothing in defect B depends on it (there was still no edited trigger), but the sentence should not be inherited.

Defect A — what changed, and why it had to be the conclusion

NOT MEASURED was one verdict at exit 0, so the check run concluded success whether the reading had passed or had never happened. It is now split by the only question that matters — could the missing reading have changed the answer?

  • not-measured-moot — no changeset grades patch a package whose packages/*/src/** this diff moves, so yes and no reach the same verdict. Exit 0, and the line now says why it is green rather than printing a tick that means nothing.
  • not-measured-material — a patch sits on a package this diff grew, so the declaration is the difference between clean and enforce, and it was not readable. Exit 1, naming the changeset, the package, and both declarations.

⛔ No tolerance, no allowlist, no comment-only heuristic, and nothing was weakened to make a case go green: the explicit Clause-②: no remains a pass on the very tree the unread reading refuses, and every previously-refusing input still refuses.

Why the conclusion route and not the output/summary route. Measured, not assumed, on a real run of this job (Check Changeset, job 101999797457, 2026-09-08): the check run's output.title is null and its output.summary is the empty string. A GitHub Actions job publishes no check-run output of its own, so "put it in the summary" is not a cheaper spelling of the same fix — it is a request for a second mechanism (a checks: write API call creating a check run) whose only readers would be human.

⚠️ One inherited premise did NOT survive re-driving, and it narrows the claim rather than the fix. The card says the required-check set reads the conclusion. Check Changeset is not in that set: read live from ruleset 12119582 (repository-sourced, active, includes_parents=true returns only it), the required contexts are TypeScript Type Check, Test Core, Dogfood Regression Gate, Build Core, Temporal Conformance (live PG + MySQL), Lint and Repo Gates, Governed Surface Queue Guard. So this gate is advisory at the ruleset layer — and that makes the conclusion route stronger, not weaker: the conclusion is the ONLY authority this gate has (the red X, gh pr checks, every summary view), and it is the same authority through which its "you forgot a changeset" verdict is enforced today. A step log is one layer below the only surface that carries this gate at all. That the set does not include it is recorded in the acceptance notes below, not fixed here.

Defect B — edited, and what it costs

The remedy for the new refusal is a PR-body edit, and a pull_request payload is a snapshot that rerun_failed_jobs replays. Without an edited trigger the red could not be cleared by anything short of pushing a commit — the permanently-red-by-construction shape 5580 and 6378 exist to remove. So the two halves of this card are one change: A's refusal is unshippable without B.

pr-automation.yml now subscribes to edited, exactly as this repo's two other PR-body-scoped blocking checks (duplicate-fix-guard.yml, partof-closing-keyword-guard.yml) already do, and for the argument their headers already carry.

Added run volume, measured (2026-09-01 to 2026-09-08, this repo's own run counts through GET /actions/workflows/FILE/runs?event=pull_request):

workflow subscribed types runs
check-links.yml the default [opened, synchronize, reopened] 1967
duplicate-fix-guard.yml + edited 2524
partof-closing-keyword-guard.yml + edited 2524
pr-automation.yml + labeled, unlabeled 3684

The two edited readings agree exactly, which is the control on the arithmetic; a paths-filtered workflow over the same window returns 1363, which is the control that the endpoint is not answering with a constant. So edited is about 557 events / 8 days, ~70 a day, about +15% on this workflow's own 3684 — and it buys one job, not three: Check PR Size and Auto Label are excluded from edited by the same sentence they already carry for label events (a title or body edit moves no file), which also keeps their behaviour byte-identical to before. One Check Changeset job is ~46s (measured on the same job above).

⚠️ Deliberately NOT filtered to github.event.changes.body. It would trim title-only edits, and its failure direction is this defect returning silently: an expression that misjudges changes.body on an empty previous body stops the gate re-reading and nothing says so. 70 runs a day does not buy that risk.

Blast radius, measured on the population rather than argued

Every one of the 45 most recent merges to main, judged with the real script against each PR's real body and labels, before and after this change (31 judged, 14 exempt by skip-changeset):

  • 25 of 31 already carry a readable declaration — the line is routine, not a new obligation.
  • 6 read NOT MEASURED. Five are moot (a forced yes still reaches clean), so they stay green and now say why.
  • Exactly one flips 0 to 1, and it is the true positive: 16775, whose body writes the declaration inside a sentence rather than at the start of a line, so the gate read nothing and concluded success over @objectstack/spec: patch beside a 124-line move in packages/spec/src/api/rest-server.zod.ts. Its author had done the thinking and written it down; the gate could not read it and said so where nobody with merge authority looks. That is this card, live, after the fact.

The whole population is its own control: every other row is byte-identical before and after.

Two lanes that must NOT be able to red, and are pinned so

cut-rc.yml runs this same script over a whole RC snapshot range on a workflow_dispatch, where there is no pull request and no declaration to read. A rule that reddened there would be unshippable, so the absence of a pull_request payload is its own verdict (no-pull-request, exit 0) rather than an unread declaration — and the same absence ON a pull_request run is a failure instead, because the runner writes that file. The self-test pins both directions, pins that no cut-rc.yml call site grows an --event, and pins that cut-rc.yml stays off pull_request triggers.

The stand-down lane needs a positive payload: false from the reader: a caller that omits the flag falls through to the lanes that can refuse. Unknown provenance enforces.

Verification

  • node scripts/check-changeset-no-major.mjs --self-test — 179 assertions, exit 0. The LEVEL battery grew 41 to 56 cases and the wiring battery 15 to 22; both floors ratcheted to the new counts.
  • Ablation 1 (the materiality split deleted, mutation proved on disk by blob hash and by counting the removed text): 6 assertions red, and the live specimen row returns to exit 0 — the defect itself, restored. Restore verified byte-identical (blob 46edc93a6) and green again.
  • Ablation 2 (edited removed from the trigger list, same proof): the wiring assertion reds by name, quoting the trigger list it read. Restore verified byte-identical (blob 6f349934a) and green again.
  • Full repo lint, not a narrowed one: node --stack-size=4000 node_modules/eslint/bin/eslint.js . --no-inline-config over 6376 files, 0 errors, 0 warnings, exit 0, at commit 675e3d0a50.
  • The gate family derived by node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack from the working change set: 50 commands, 49 green. The one exception is pnpm check:pm-dispatch-gates, which exceeded the container's foreground limit twice while a sibling agent held the same gate; it is reported as NOT MEASURED rather than as a pass or a failure, and CI runs it.
  • No changeset, skip-changeset instead, measured rather than assumed: all 23 published packages ship dist, README.md and CHANGELOG.md and nothing else (positive control: dist present in 23 of 23), so no files[] entry can carry .github/** or scripts/**.

What is deliberately NOT here

The card's second direction for defect B — requiring the Clause-②: line in the PR body from the first push — is partly a change to os-dev's own contract in .claude/agents/os-dev.md. That path is a governed surface (docs/adr/** · .claude/** · skills/** · AGENTS.md · CLAUDE.md), so it is not in this diff and is not this seat's to land. The gate half stands on its own and does not depend on it: this PR refuses only where the missing declaration decides the verdict, which is a much narrower demand than "every PR must declare".

验收备注

  • Check Changeset is not in the repository's required-status-check set (measured above). A gate that carries a maintainer ruling being advisory at the ruleset layer is a fact worth someone's decision, and it is a repository setting no PR can change — noted for the maintainer, not filed and not fixed here.
  • The edited trigger fires on title edits too. Left unfiltered on purpose, with the argument and the volume above.
  • The residual this change does not close: a PR that carries allow-major skips the whole guard step, this axis included. Unchanged, and already recorded in the script's own header.

Generated by Claude Code

…evel verdict now fails, and the body is re-read on edit

`check-changeset-no-major.mjs`'s level axis exited 0 for both "judged and passed"
and "had nothing to judge". A check run concludes `success` or `failure` and has
no third word, so the two published the same conclusion on every surface that
reads conclusions rather than step logs, while `LEVEL AXIS: NOT MEASURED` stayed
in the job log.

`NOT MEASURED` is now split by whether the missing reading could have changed the
answer:

  * not-measured-moot     — no changeset grades `patch` a package whose
                            `packages/*/src/**` the diff moves, so `yes` and `no`
                            reach the same verdict. Still exit 0, and it now says
                            why it is green rather than printing a bare tick.
  * not-measured-material — a `patch` sits on a package the diff grew, so the
                            declaration is the difference between `clean` and
                            `enforce`, and it was not readable. Exit 1, naming the
                            changeset, the package and both declarations.

The explicit `Clause-②: no` stays a pass on the very tree the unread reading
refuses: the opt-out is a declaration, never a tolerance.

Measured over the 45 most recent merges (31 judged, 14 exempt by label): exactly
one flips. Its body declares the clause inside a sentence rather than on a line,
so the gate read nothing and concluded success over an `@objectstack/spec` patch
beside a 124-line move in that package's own src.

Two lanes stay green because they are not pull requests at all — the RC cut
(`cut-rc.yml`, `workflow_dispatch`, no `--event`) and a local run reach
`no-pull-request`. On a real `pull_request` run an unreadable payload is a failure
instead: the runner writes that file, and a gate that could not read the input it
was owed has verified nothing.

The workflow half is the other half of the same fix. The remedy for the new
refusal is a PR-BODY edit, and a `pull_request` payload is a snapshot that
`rerun_failed_jobs` replays, so with no `edited` trigger the red could not be
cleared without pushing a commit. `pr-automation.yml` now subscribes to `edited`,
as this repo's two other PR-body-scoped blocking checks already do. `Check PR
Size` and `Auto Label` are excluded from it by the sentence they already carry
for label events, so the added volume buys exactly one job: measured
2026-09-01..09-08 from this repo's own run counts, about 557 `edited` events over
8 days against this workflow's own 3684, roughly +15%, at ~46s per job.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU
@claude claude Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Sep 8, 2026
@claude

claude Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

ACCEPT — PR #16897, reviewed against GitHub and the tree, ⛔ not against the report

What I re-drove myself

claim how I checked it result
Check Changeset is NOT in the required-check set read ruleset 12119582 on main directly 7 required contexts: Build Core, Dogfood Regression Gate, Governed Surface Queue Guard, Lint & Repo Gates, Temporal Conformance (live PG + MySQL), Test Core, TypeScript Type Check. Check Changeset is absent. The card's merge-authority sentence was wider than the fact
the card's "the Check Changeset job skips label events" read the base pr-automation.yml wrong on this base. Only two jobs carry an action exclusion — :99 (Check PR Size) and :164 (Auto Label). The changeset-check job's if: at :47-48 has none
the live specimen, PR #16775 read its body and files ✅ exact. Clause-②: no appears inside a sentence; line-start matches for ^Clause-②: = 0. It carries .changeset/rest-server-config-embedder-only-reachability.md grading "@objectstack/spec": patch and moves 1 packages/spec/src/** file. ⇒ a patch on a package the PR grew, with the declaration unreadable, merged today at 07:19:53Z under a green check
the edited cost arithmetic re-ran the same run counts myself ✅ reproduced. check-links.yml (default types) 1979; duplicate-fix-guard.yml 2542; partof-closing-keyword-guard.yml 2542; pr-automation.yml 3707. My numbers run ~0.6 % above the report's because ~40 minutes of runs accrued in between — ⭐ but the load-bearing structure reproduces exactly: the two edited workflows agree with each other to the unit, and the delta over the default-types workflow is 563 (report: 557), ~+15 % of this workflow's own volume, buying one job
the verdict machine actually splits at the exit code read the patch ✅ eight verdicts with distinct exits, and the self-test pins the distinction where it counts: renderLevel(notMeasuredMaterial).exitCode === 1 && renderLevel(notMeasuredMoot).exitCode === 0 — asserted on the exit code, not on the verdict name
commit carries no card trailer read it ✅ zero card-relation trailers; the relation is in the PR body

⭐ Why this clears the bar I set in the claim

The brief said: whatever you choose, the distinction must reach the layer that gates a merge, and ⛔ "a careful reviewer could see it" is not an argument. Three things make this the right route rather than the convenient one:

  1. The output/summary route was measured and rejected, not waved off — a real run's check-run output.title is null and its summary empty, because a GitHub Actions job publishes no check-run output of its own. So that route does not exist here; the conclusion is the only surface.
  2. The refusal is narrow by construction. It does not demand that every PR declare. It refuses only where the missing declaration decides the verdict — a patch sitting on a package the diff grew. Where yes and no reach the same answer it still exits 0, and now says why it is green. ⇒ an exit 0 that means "the missing input could not have moved this" is a different object from one that means "I did not look", and the gate now distinguishes them.
  3. A and B are one change, not two sharing a file. The remedy for the new refusal is a PR-body edit, and a pull_request payload is a frozen snapshot that rerun_failed_jobs replays. Without edited, the new red could not be cleared without pushing a commit — the permanently-red-by-construction shape this repo already fixed twice. A is unshippable without B, and the wiring is pinned in the self-test rather than argued in prose.

⭐ And the discovery that Check Changeset is not a required context narrows the card's claim while strengthening the chosen route: the conclusion is not merely the layer merge authority reads — it is the only authority this gate has, and it is the same authority its existing "you forgot a changeset" verdict already rides.

⭐ Two guards I want on the record

The blast radius, and what I am NOT claiming

The report judged the 45 most recent merges with the real script against their real bodies and labels: 31 judged, 14 exempt, 25 already declaring, 6 NOT MEASURED of which 5 moot ⇒ exactly one flips 0 → 1, and it is PR #16775 — this card, live and after the fact. ⛔ I could not re-drive that sweep: this container has no node_modules, so the script refuses with a prerequisite error rather than running — a failed read, not a reading. What I did verify independently is the specimen itself (above), which is the row the whole sweep turns on. The population being its own before/after control is the right shape; I am relying on the dev for the other 30 rows and saying so.

#16361 — fold-or-serial, confirmed

Not folded, and the answer is right. This PR changes the verdict/exit machine and the trigger list; #16361 is about which packages the yes predicate applies to. Same two files, different defect, different repair — gate ① of the fold test fails. #16361 stays pm:queue and wants a rebase after this lands, not a fold. Its hold note is already on that card.

Open question, routed not answered

Should Check Changeset be enrolled as a required context? ⛔ Not this seat's to decide and ⛔ not reachable from any PR — it is a ruleset edit, and enrolment would also oblige the workflow to carry a merge_group: trigger and a REQUIRED_CONTEXTS entry (check-required-contexts.mjs items 6 and 7). Recommendation A for this card (leave advisory — nothing here depends on enrolment) with B as a separate maintainer decision. It goes to the maintainer in the round report; it does not hold this PR.

The cheaper route, correctly declined

Requiring the Clause-②: line from the PR's first push is partly os-dev's own contract in .claude/agents/os-dev.md — a governed surface this seat cannot land. It is not in this diff, and the gate half stands alone without it. ⭐ That is the right call: it refuses only where the missing declaration decides the verdict, which is a far narrower demand than "every PR must declare".

Checks

30 checks, 0 failures, 0 pending. Flipping and arming now.


Generated by Claude Code

@baozhoutao
baozhoutao marked this pull request as ready for review September 8, 2026 15:28
@baozhoutao
baozhoutao added this pull request to the merge queue Sep 8, 2026
Merged via the queue into main with commit 0939ba0 Sep 8, 2026
36 checks passed
@baozhoutao
baozhoutao deleted the claude/issue-16776-changeset-level-axis-unmeasured branch September 8, 2026 15:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants