admin-bypass-repair-bot-thread-pr-840-ea7d365 - #854
EdbertChan wants to merge 23 commits into
Conversation
… a repo has none Scope now comes from the .git boundary, not scripts/create-pr.mjs. The validator is scripts/validate-pr-body.mjs, then engine/skills/draft-pr/scripts/validate-pr-body.mjs. A PR-publishing command in a git repo with neither reports UNCHECKED instead of staying silent. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…eview claim: pr-schema-gate checks PR descriptions in a repo whose validator lives at engine/skills/draft-pr/scripts/validate-pr-body.mjs, and reports UNCHECKED instead of staying silent when a PR-publishing command runs in a repo where no validator can be found. Review lane: behavior Safety invariant: The change can only add a failure or an UNCHECKED notice; a PR description that passes engine/skills/draft-pr/scripts/validate-pr-body.mjs today is never newly blocked. Effectiveness measurement: Hook tests replay the three commands that stayed silent in the incident: `mergify stack push` in a catstack-shaped repo (fires or UNCHECKED, never silent), `gh pr create --body-file <body failing the validator>` in a catstack-shaped repo (fires with the validator's errors), and a passing body in the same repo (clean). The existing Invoker-shaped cases keep their current results. Slice rationale: One hook, one claim: where the hook looks for the validator and what it says when it finds none. Architectural effect: pr-schema-gate stops depending on scripts/create-pr.mjs to decide whether a repo is in scope. Goal: Make the PR-description guard fire in catstack. Motivation: A reflect pass found PR descriptions on catstack PRs #780-#789, #793 and #795 failing the required PR Body check after publication, and the user had to ask for a manual fix of every PR. In catstack the guard stayed silent: fed `mergify stack push` and `gh pr create --body-file <failing body>` it exited 0 with no output, while the same stack push in the Invoker checkout fired. Read at origin/main: engine/hooks/pr-schema-gate/detect.py scopes itself by walking up for scripts/create-pr.mjs (repo_root_with_create_pr_tool) and expects the validator at scripts/validate-pr-body.mjs (VALIDATOR_RELATIVE_PATH); catstack has neither. Alternative considerations: Adding a scripts/create-pr.mjs shim to catstack was set aside because it makes scope depend on an unrelated file again. Copying the validator to scripts/ was set aside because there are already too many copies. Implementation details: In engine/hooks/pr-schema-gate/detect.py, find the repo root from the .git boundary, then look for the validator in a short ordered list: scripts/validate-pr-body.mjs, then engine/skills/draft-pr/scripts/validate-pr-body.mjs. A repo with either is in scope. When a PR-publishing command (gh pr create/edit with a body, gh api PATCH on pulls with a body, mergify stack push) runs in a git repo where no validator is found, return the existing unchecked outcome with a one-line reason instead of None. Keep the existing hit / clean / unchecked outcomes and messages for Invoker-shaped repos unchanged. Non-goals: No change to the validator, preflight, install.sh, or any other hook. Do not edit engine/skills/draft-pr/scripts/validate-pr-body.mjs, scripts/pr/validate-pr-body-local.mjs, or any other file open PR #742 changes; call the validator as it is. If a change would overlap an open PR, stop and report instead. Layer: domain Feature state: active Files: - engine/hooks/pr-schema-gate/detect.py - engine/hooks/pr-schema-gate/tests/test_hooks.py - engine/hooks/pr-schema-gate/README.md Change types: - engine/hooks/pr-schema-gate/detect.py: modify - engine/hooks/pr-schema-gate/tests/test_hooks.py: modify - engine/hooks/pr-schema-gate/README.md: modify Acceptance criteria: - `python3 -m unittest discover -s engine/hooks/pr-schema-gate/tests -v` exits 0. - `python3 scripts/check_hook_test_coverage.py` exits 0. - `python3 scripts/check_no_silent_hook_except.py` exits 0. - `python3 scripts/check_no_new_comments.py` exits 0. Exit code: 0
…view claim: `python3 scripts/check_no_silent_hook_except.py` passes on the finished branch. Review lane: proof Safety invariant: Verification is read-only and alters no repository file. Effectiveness measurement: The command's exit code is the direct measurement. Slice rationale: One check per proof task. Architectural effect: None; verification only. Goal: Prove the slice. Motivation: Running the check is the proof. Alternative considerations: The full suite was not required because the slice touches one component with its own tests. Implementation details: Run `python3 scripts/check_no_silent_hook_except.py`. Non-goals: No mutations. Layer: app_regression Feature state: active Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry. Acceptance criteria: - The command exits 0. Exit code: 0
…view claim: `python3 -m unittest discover -s engine/hooks/pr-schema-gate/tests -v` passes on the finished branch. Review lane: proof Safety invariant: Verification is read-only and alters no repository file. Effectiveness measurement: The command's exit code is the direct measurement. Slice rationale: One check per proof task. Architectural effect: None; verification only. Goal: Prove the slice. Motivation: Running the check is the proof. Alternative considerations: The full suite was not required because the slice touches one component with its own tests. Implementation details: Run `python3 -m unittest discover -s engine/hooks/pr-schema-gate/tests -v`. Non-goals: No mutations. Layer: app_regression Feature state: active Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry. Acceptance criteria: - The command exits 0. Exit code: 0
…view claim: `python3 scripts/check_no_new_comments.py` passes on the finished branch. Review lane: proof Safety invariant: Verification is read-only and alters no repository file. Effectiveness measurement: The command's exit code is the direct measurement. Slice rationale: One check per proof task. Architectural effect: None; verification only. Goal: Prove the slice. Motivation: Running the check is the proof. Alternative considerations: The full suite was not required because the slice touches one component with its own tests. Implementation details: Run `python3 scripts/check_no_new_comments.py`. Non-goals: No mutations. Layer: app_regression Feature state: active Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry. Acceptance criteria: - The command exits 0. Exit code: 1
…view claim: `python3 scripts/check_hook_test_coverage.py` passes on the finished branch. Review lane: proof Safety invariant: Verification is read-only and alters no repository file. Effectiveness measurement: The command's exit code is the direct measurement. Slice rationale: One check per proof task. Architectural effect: None; verification only. Goal: Prove the slice. Motivation: Running the check is the proof. Alternative considerations: The full suite was not required because the slice touches one component with its own tests. Implementation details: Run `python3 scripts/check_hook_test_coverage.py`. Non-goals: No mutations. Layer: app_regression Feature state: active Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry. Acceptance criteria: - The command exits 0. Exit code: 1
…view claim: `python3 scripts/check_hook_test_coverage.py` passes on the finished branch. Review lane: proof Safety invariant: Verification is read-only and alters no repository file. Effectiveness measurement: The command's exit code is the direct measurement. Slice rationale: One check per proof task. Architectural effect: None; verification only. Goal: Prove the slice. Motivation: Running the check is the proof. Alternative considerations: The full suite was not required because the slice touches one component with its own tests. Implementation details: Run `python3 scripts/check_hook_test_coverage.py`. Non-goals: No mutations. Layer: app_regression Feature state: active Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry. Acceptance criteria: - The command exits 0. Exit code: 1
…view claim: `python3 scripts/check_no_new_comments.py` passes on the finished branch. Review lane: proof Safety invariant: Verification is read-only and alters no repository file. Effectiveness measurement: The command's exit code is the direct measurement. Slice rationale: One check per proof task. Architectural effect: None; verification only. Goal: Prove the slice. Motivation: Running the check is the proof. Alternative considerations: The full suite was not required because the slice touches one component with its own tests. Implementation details: Run `python3 scripts/check_no_new_comments.py`. Non-goals: No mutations. Layer: app_regression Feature state: active Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry. Acceptance criteria: - The command exits 0. Exit code: 1
…view claim: `python3 scripts/check_hook_test_coverage.py` passes on the finished branch. Review lane: proof Safety invariant: Verification is read-only and alters no repository file. Effectiveness measurement: The command's exit code is the direct measurement. Slice rationale: One check per proof task. Architectural effect: None; verification only. Goal: Prove the slice. Motivation: Running the check is the proof. Alternative considerations: The full suite was not required because the slice touches one component with its own tests. Implementation details: Run `python3 scripts/check_hook_test_coverage.py`. Non-goals: No mutations. Layer: app_regression Feature state: active Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry. Acceptance criteria: - The command exits 0. Solution: Review claim: `python3 scripts/check_hook_test_coverage.py` passes on the finished branch. Review lane: proof Safety invariant: Verification is read-only and alters no repository file. Effectiveness measurement: The command's exit code is the direct measurement. Slice rationale: One check per proof task. Architectural effect: None; verification only. Goal: Prove the slice. Motivation: Running the check is the proof. Alternative considerations: The full suite was not required because the slice touches one component with its own tests. Implementation details: Run `python3 scripts/check_hook_test_coverage.py`. Non-goals: No mutations. Layer: app_regression Feature state: active Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry. Acceptance criteria: - The command exits 0.
…view claim: `python3 scripts/check_no_new_comments.py` passes on the finished branch. Review lane: proof Safety invariant: Verification is read-only and alters no repository file. Effectiveness measurement: The command's exit code is the direct measurement. Slice rationale: One check per proof task. Architectural effect: None; verification only. Goal: Prove the slice. Motivation: Running the check is the proof. Alternative considerations: The full suite was not required because the slice touches one component with its own tests. Implementation details: Run `python3 scripts/check_no_new_comments.py`. Non-goals: No mutations. Layer: app_regression Feature state: active Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry. Acceptance criteria: - The command exits 0. Solution: Review claim: `python3 scripts/check_no_new_comments.py` passes on the finished branch. Review lane: proof Safety invariant: Verification is read-only and alters no repository file. Effectiveness measurement: The command's exit code is the direct measurement. Slice rationale: One check per proof task. Architectural effect: None; verification only. Goal: Prove the slice. Motivation: Running the check is the proof. Alternative considerations: The full suite was not required because the slice touches one component with its own tests. Implementation details: Run `python3 scripts/check_no_new_comments.py`. Non-goals: No mutations. Layer: app_regression Feature state: active Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry. Acceptance criteria: - The command exits 0.
…view claim: `python3 scripts/check_hook_test_coverage.py` passes on the finished branch. Review lane: proof Safety invariant: Verification is read-only and alters no repository file. Effectiveness measurement: The command's exit code is the direct measurement. Slice rationale: One check per proof task. Architectural effect: None; verification only. Goal: Prove the slice. Motivation: Running the check is the proof. Alternative considerations: The full suite was not required because the slice touches one component with its own tests. Implementation details: Run `python3 scripts/check_hook_test_coverage.py`. Non-goals: No mutations. Layer: app_regression Feature state: active Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry. Acceptance criteria: - The command exits 0. Exit code: 0
…view claim: `python3 scripts/check_no_new_comments.py` passes on the finished branch. Review lane: proof Safety invariant: Verification is read-only and alters no repository file. Effectiveness measurement: The command's exit code is the direct measurement. Slice rationale: One check per proof task. Architectural effect: None; verification only. Goal: Prove the slice. Motivation: Running the check is the proof. Alternative considerations: The full suite was not required because the slice touches one component with its own tests. Implementation details: Run `python3 scripts/check_no_new_comments.py`. Non-goals: No mutations. Layer: app_regression Feature state: active Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry. Acceptance criteria: - The command exits 0. Exit code: 0
…ck-checker-2/g0.t1.a-a5988c815-2d010ac6
…ck-checker-3/g0.t0.a-a6446bd46-2b7ad752
…ck-checker-4/g0.t1.a-ae682b326-22645b22
…o ephemeral inter-task handoff files remain in the worktree before the merge gate. Review lane: cleanup Safety invariant: The scrub script only checks for known handoff artifact names and never touches source, tests, or other repository files. Effectiveness measurement: The script exits non-zero if any handoff artifact remains. Slice rationale: Required terminal scrub for every implementation workflow. Architectural effect: None; hygiene only. Goal: Leave the branch free of handoff artifacts. Motivation: Handoff files must not reach the PR. Alternative considerations: Manual cleanup was set aside as non-deterministic. Implementation details: Run scripts/scrub-handoff-artifacts.sh. Non-goals: No product edits. Layer: app_regression Feature state: active Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry. Acceptance criteria: - The command exits 0. Exit code: 0
…37cebe4c-08bdbf79 — Review claim: No ephemeral inter-task handoff files remain in the worktree before the merge gate. Review lane: cleanup Safety invariant: The scrub script only checks for known handoff artifact names and never touches source, tests, or other repository files. Effectiveness measurement: The script exits non-zero if any handoff artifact remains. Slice rationale: Required terminal scrub for every implementation workflow. Architectural effect: None; hygiene only. Goal: Leave the branch free of handoff artifacts. Motivation: Handoff files must not reach the PR. Alternative considerations: Manual cleanup was set aside as non-deterministic. Implementation details: Run scripts/scrub-handoff-artifacts.sh. Non-goals: No product edits. Layer: app_regression Feature state: active Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry. Acceptance criteria: - The command exits 0.
A PR body the repo's validator accepted was reported to the agent as "could not check", and an owed stack follow-up stayed armed. The hook treats exit 0 alongside an UNCHECKED/SKIPPED/not-installed line as a vacuous pass, so the validator gets no credit for a run that never judged the body. The catstack validator prints exactly such a line for a sub-check it skipped -- "Summary reading grade unchecked: Summary has N words; under 30 the score is too noisy to trust" -- while still accepting the body and exiting 0. Every accepted body with a short Summary was therefore reported unchecked. check_body_file now lifts the vacuous reading when the run states its own verdict on the body (VALIDATOR_PASS_VERDICT_RE, "PR body validation passed"). Only a pass verdict counts, so a "failed" banner beside exit 0 stays unchecked. The scan also reads the whole output rather than the first VALIDATOR_OUTPUT_MAX_LINES: truncation shortens what the agent is shown, never what is judged. Two tests in tests/test_advisory.py drive a stub validator with the catstack shape (skip note on stderr, verdict on stdout, exit 0): one asserts the write is silent, one asserts it clears the pending follow-up. Both fail before this change. test_validator_exit_zero_with_an_unchecked_ line_is_not_clean still passes, so the guard against a validator that really did not run is unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… its head did not move Exit code: 0
…ead PRRT_kwDOT3uYWs6lQWRD after PR head changes Exit code: 0
…1-36089247 — Resolve bot review thread PRRT_kwDOT3uYWs6lQWRD after PR head changes
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 6315464. Configure here.
| with open(os.path.join(tmp.name, "scripts", "create-pr.mjs"), "w") as f: | ||
| f.write("// stub\n") | ||
| with open(os.path.join(tmp.name, "scripts", "validate-pr-body.mjs"), "w") as f: | ||
| f.write("// stub\n") |
There was a problem hiding this comment.
Stub validator flakes in-scope write tests
Medium Severity
_repo_with_tool now writes a validate-pr-body.mjs stub that exits 0 with no output. check_body_file treats that silent pass as clean, so gh pr edit --body-file /tmp/body.md emits nothing when the file already exists. test_never_exits_nonzero_for_any_pr_write_in_scope and test_codex_nested_workdir_can_select_repo_with_tool then fail because they require an advisory line.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit 6315464. Configure here.
|
This pull request does not currently match the merge queue conditions, so it cannot be queued from here. The box comes back if it matches again. |


Summary
A local checker can compare pull request description text with the repository's own rules before an assistant publishes it.
Some repositories keep their rule runner in more than one common location.
This slice treats the repository root as the boundary and searches the supported rule-runner locations.
When no runner exists, publishing advice says unchecked instead of staying quiet.
Accepted descriptions that skip one noisy subcheck still clear the outstanding stack reminder.
Review Claim
Approve that pull request description writes use the matching repository's validator, or receive an explicit unchecked warning.
Review Lane
behavior
Review Unit
engine-runtime
Safety Invariant
The checker still exits zero for every advisory path; this slice only changes which message is shown and whether the stack reminder remains pending.
Assumption: non-interactive request; no separate safety confirmation.
Slice Rationale
This keeps the detector, hook entrypoint, documentation, and focused tests together because they describe one publishing-advice behavior.
Non-goals
Architecture
Before
The advisory path only treated repositories with one publishing helper as in scope, and stack publication reminders named one validator path.
After
The advisory path first finds the git repository root, then chooses the first validator path that exists. If no validator exists, publishing commands get an unchecked warning and stack publication does not arm reminder state.
Test Plan
Test Plan
python3 scripts/ci/check_hook_test_coverage.py engine/hooks/pr-schema-gate- passed withcheck_hook_test_coverage: OK (1 hook(s) checked).python3 scripts/ci/check_no_new_comments.py --base main- passed withok no new comments.CATSTACK_LLM_JUDGE_RUNNERS='[["codex", ["codex", "exec", "--skip-git-repo-check", "--sandbox", "read-only", "-c", "notify=[]", "{prompt}"]]]' PATH="$HOME/.local/bin:$PATH" python3 engine/skills/make-pr/scripts/preflight.py --base origin/main --body-file .pr-body-draft.md- passed withok preflight passed.node scripts/pr/validate-pr-body-local.mjs --body-file .pr-body-draft.md --base main- passed withPR body validation passed.PRRT_kwDOT3uYWs6lQWRD.python3 -m unittest discover -s engine/hooks/pr-schema-gate/tests -v- run locally; failed intest_codex_nested_workdir_can_select_repo_with_toolandtest_never_exits_nonzero_for_any_pr_write_in_scope.Revert Plan
Revert Plan
git revert 6315464.Note
Medium Risk
Changes which repos get PR-style advisories and when stack follow-up state arms, affecting agent guidance on publishing flows; the hook still never blocks commands.
Overview
pr-schema-gate no longer keys scope on
scripts/create-pr.mjs. It walks to the git root (including worktrees with a.gitfile), picks the first existing validator (scripts/validate-pr-body.mjsor catstack’sengine/skills/draft-pr/scripts/validate-pr-body.mjs), and runs that on file-backed PR writes. Advisory text names the validator path that was used or sought.Git repos without either validator now get an explicit unchecked warning on PR-publishing commands (
gh pr create/edit,gh apibody writes,mergify stack push) instead of staying silent;mergify stack pushno longer arms the stack follow-up reminder when no validator exists. Directories outside any git repo remain silent; unparseable shell in a repo with no validator stays silent.Validator exit-0 handling is tightened: a real pass requires
PR body validation passed.in the full output, so catstack’s “Summary reading grade unchecked” on stderr no longer flips an accepted body to unchecked or leaves a stale stack reminder. Vacuous exit-0 (UNCHECKED/SKIPPED/not-installed) without that verdict still reports unchecked.Tests and README document the new scope, catstack-shaped repos, and no-validator behavior. Two CI helpers resolve script paths with
os.path.realpathinstead ofabspath.Reviewed by Cursor Bugbot for commit 6315464. Bugbot is set up for automated code reviews on this repo. Configure here.