Skip to content

admin-bypass-repair-bot-thread-pr-840-ea7d365 - #854

Closed
EdbertChan wants to merge 23 commits into
mainfrom
plan/admin-bypass-repair-bot-thread-pr-840-ea7d365
Closed

EdbertChan wants to merge 23 commits into
mainfrom
plan/admin-bypass-repair-bot-thread-pr-840-ea7d365

Conversation

@EdbertChan

@EdbertChan EdbertChan commented Sep 23, 2026 •

Copy link
Copy Markdown
Owner

Summary

A local checker can compare pull request description text with the repository's own rules before an assistant publishes it.

Some repositories keep their rule runner in more than one common location.

This slice treats the repository root as the boundary and searches the supported rule-runner locations.

When no runner exists, publishing advice says unchecked instead of staying quiet.

Accepted descriptions that skip one noisy subcheck still clear the outstanding stack reminder.

Review Claim

Approve that pull request description writes use the matching repository's validator, or receive an explicit unchecked warning.

Review Lane

behavior

Review Unit

engine-runtime

Safety Invariant

The checker still exits zero for every advisory path; this slice only changes which message is shown and whether the stack reminder remains pending.

Assumption: non-interactive request; no separate safety confirmation.

Slice Rationale

This keeps the detector, hook entrypoint, documentation, and focused tests together because they describe one publishing-advice behavior.

Non-goals

  • No blocking behavior is added.
  • No pull request body rules are reimplemented in the hook.
  • No user interface behavior changes.

Architecture

Before

The advisory path only treated repositories with one publishing helper as in scope, and stack publication reminders named one validator path.

After

The advisory path first finds the git repository root, then chooses the first validator path that exists. If no validator exists, publishing commands get an unchecked warning and stack publication does not arm reminder state.

Test Plan

Test Plan
  • python3 scripts/ci/check_hook_test_coverage.py engine/hooks/pr-schema-gate - passed with check_hook_test_coverage: OK (1 hook(s) checked).
  • python3 scripts/ci/check_no_new_comments.py --base main - passed with ok no new comments.
  • CATSTACK_LLM_JUDGE_RUNNERS='[["codex", ["codex", "exec", "--skip-git-repo-check", "--sandbox", "read-only", "-c", "notify=[]", "{prompt}"]]]' PATH="$HOME/.local/bin:$PATH" python3 engine/skills/make-pr/scripts/preflight.py --base origin/main --body-file .pr-body-draft.md - passed with ok preflight passed.
  • node scripts/pr/validate-pr-body-local.mjs --body-file .pr-body-draft.md --base main - passed with PR body validation passed.
  • Workflow safe-push guard for PR PR body gate (1) the PR-description guard hook runs in catstack #840.
  • Workflow review-thread resolver for PRRT_kwDOT3uYWs6lQWRD.
  • python3 -m unittest discover -s engine/hooks/pr-schema-gate/tests -v - run locally; failed in test_codex_nested_workdir_can_select_repo_with_tool and test_never_exits_nonzero_for_any_pr_write_in_scope.

Revert Plan

Revert Plan
  • Safe to revert? Yes.
  • Revert command: git revert 6315464.
  • Post-revert steps: rerun the hook coverage check and the no-new-comments check.
  • Data migration? No.

Note

Medium Risk
Changes which repos get PR-style advisories and when stack follow-up state arms, affecting agent guidance on publishing flows; the hook still never blocks commands.

Overview
pr-schema-gate no longer keys scope on scripts/create-pr.mjs. It walks to the git root (including worktrees with a .git file), picks the first existing validator (scripts/validate-pr-body.mjs or catstack’s engine/skills/draft-pr/scripts/validate-pr-body.mjs), and runs that on file-backed PR writes. Advisory text names the validator path that was used or sought.

Git repos without either validator now get an explicit unchecked warning on PR-publishing commands (gh pr create/edit, gh api body writes, mergify stack push) instead of staying silent; mergify stack push no longer arms the stack follow-up reminder when no validator exists. Directories outside any git repo remain silent; unparseable shell in a repo with no validator stays silent.

Validator exit-0 handling is tightened: a real pass requires PR body validation passed. in the full output, so catstack’s “Summary reading grade unchecked” on stderr no longer flips an accepted body to unchecked or leaves a stale stack reminder. Vacuous exit-0 (UNCHECKED/SKIPPED/not-installed) without that verdict still reports unchecked.

Tests and README document the new scope, catstack-shaped repos, and no-validator behavior. Two CI helpers resolve script paths with os.path.realpath instead of abspath.

Reviewed by Cursor Bugbot for commit 6315464. Bugbot is set up for automated code reviews on this repo. Configure here.

EdbertChan and others added 23 commits September 24, 2026 00:55
… a repo has none

Scope now comes from the .git boundary, not scripts/create-pr.mjs. The
validator is scripts/validate-pr-body.mjs, then
engine/skills/draft-pr/scripts/validate-pr-body.mjs. A PR-publishing
command in a git repo with neither reports UNCHECKED instead of staying
silent.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…eview claim: pr-schema-gate checks PR descriptions in a repo whose validator lives at engine/skills/draft-pr/scripts/validate-pr-body.mjs, and reports UNCHECKED instead of staying silent when a PR-publishing command runs in a repo where no validator can be found.

Review lane: behavior
Safety invariant: The change can only add a failure or an UNCHECKED notice; a PR description that passes engine/skills/draft-pr/scripts/validate-pr-body.mjs today is never newly blocked.
Effectiveness measurement: Hook tests replay the three commands that stayed silent in the incident: `mergify stack push` in a catstack-shaped repo (fires or UNCHECKED, never silent), `gh pr create --body-file <body failing the validator>` in a catstack-shaped repo (fires with the validator's errors), and a passing body in the same repo (clean). The existing Invoker-shaped cases keep their current results.
Slice rationale: One hook, one claim: where the hook looks for the validator and what it says when it finds none.
Architectural effect: pr-schema-gate stops depending on scripts/create-pr.mjs to decide whether a repo is in scope.
Goal: Make the PR-description guard fire in catstack.
Motivation: A reflect pass found PR descriptions on catstack PRs #780-#789, #793 and #795 failing the required PR Body check after publication, and the user had to ask for a manual fix of every PR. In catstack the guard stayed silent: fed `mergify stack push` and `gh pr create --body-file <failing body>` it exited 0 with no output, while the same stack push in the Invoker checkout fired. Read at origin/main: engine/hooks/pr-schema-gate/detect.py scopes itself by walking up for scripts/create-pr.mjs (repo_root_with_create_pr_tool) and expects the validator at scripts/validate-pr-body.mjs (VALIDATOR_RELATIVE_PATH); catstack has neither.
Alternative considerations: Adding a scripts/create-pr.mjs shim to catstack was set aside because it makes scope depend on an unrelated file again. Copying the validator to scripts/ was set aside because there are already too many copies.
Implementation details: In engine/hooks/pr-schema-gate/detect.py, find the repo root from the .git boundary, then look for the validator in a short ordered list: scripts/validate-pr-body.mjs, then engine/skills/draft-pr/scripts/validate-pr-body.mjs. A repo with either is in scope. When a PR-publishing command (gh pr create/edit with a body, gh api PATCH on pulls with a body, mergify stack push) runs in a git repo where no validator is found, return the existing unchecked outcome with a one-line reason instead of None. Keep the existing hit / clean / unchecked outcomes and messages for Invoker-shaped repos unchanged.
Non-goals: No change to the validator, preflight, install.sh, or any other hook. Do not edit engine/skills/draft-pr/scripts/validate-pr-body.mjs, scripts/pr/validate-pr-body-local.mjs, or any other file open PR #742 changes; call the validator as it is. If a change would overlap an open PR, stop and report instead.
Layer: domain
Feature state: active
Files:
- engine/hooks/pr-schema-gate/detect.py
- engine/hooks/pr-schema-gate/tests/test_hooks.py
- engine/hooks/pr-schema-gate/README.md
Change types:
- engine/hooks/pr-schema-gate/detect.py: modify
- engine/hooks/pr-schema-gate/tests/test_hooks.py: modify
- engine/hooks/pr-schema-gate/README.md: modify
Acceptance criteria:
- `python3 -m unittest discover -s engine/hooks/pr-schema-gate/tests -v` exits 0.
- `python3 scripts/check_hook_test_coverage.py` exits 0.
- `python3 scripts/check_no_silent_hook_except.py` exits 0.
- `python3 scripts/check_no_new_comments.py` exits 0.

Exit code: 0
…view claim: `python3 scripts/check_no_silent_hook_except.py` passes on the finished branch.

Review lane: proof
Safety invariant: Verification is read-only and alters no repository file.
Effectiveness measurement: The command's exit code is the direct measurement.
Slice rationale: One check per proof task.
Architectural effect: None; verification only.
Goal: Prove the slice.
Motivation: Running the check is the proof.
Alternative considerations: The full suite was not required because the slice touches one component with its own tests.
Implementation details: Run `python3 scripts/check_no_silent_hook_except.py`.
Non-goals: No mutations.
Layer: app_regression
Feature state: active
Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry.
Acceptance criteria:
- The command exits 0.

Exit code: 0
…view claim: `python3 -m unittest discover -s engine/hooks/pr-schema-gate/tests -v` passes on the finished branch.

Review lane: proof
Safety invariant: Verification is read-only and alters no repository file.
Effectiveness measurement: The command's exit code is the direct measurement.
Slice rationale: One check per proof task.
Architectural effect: None; verification only.
Goal: Prove the slice.
Motivation: Running the check is the proof.
Alternative considerations: The full suite was not required because the slice touches one component with its own tests.
Implementation details: Run `python3 -m unittest discover -s engine/hooks/pr-schema-gate/tests -v`.
Non-goals: No mutations.
Layer: app_regression
Feature state: active
Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry.
Acceptance criteria:
- The command exits 0.

Exit code: 0
…view claim: `python3 scripts/check_no_new_comments.py` passes on the finished branch.

Review lane: proof
Safety invariant: Verification is read-only and alters no repository file.
Effectiveness measurement: The command's exit code is the direct measurement.
Slice rationale: One check per proof task.
Architectural effect: None; verification only.
Goal: Prove the slice.
Motivation: Running the check is the proof.
Alternative considerations: The full suite was not required because the slice touches one component with its own tests.
Implementation details: Run `python3 scripts/check_no_new_comments.py`.
Non-goals: No mutations.
Layer: app_regression
Feature state: active
Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry.
Acceptance criteria:
- The command exits 0.

Exit code: 1
…view claim: `python3 scripts/check_hook_test_coverage.py` passes on the finished branch.

Review lane: proof
Safety invariant: Verification is read-only and alters no repository file.
Effectiveness measurement: The command's exit code is the direct measurement.
Slice rationale: One check per proof task.
Architectural effect: None; verification only.
Goal: Prove the slice.
Motivation: Running the check is the proof.
Alternative considerations: The full suite was not required because the slice touches one component with its own tests.
Implementation details: Run `python3 scripts/check_hook_test_coverage.py`.
Non-goals: No mutations.
Layer: app_regression
Feature state: active
Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry.
Acceptance criteria:
- The command exits 0.

Exit code: 1
…view claim: `python3 scripts/check_hook_test_coverage.py` passes on the finished branch.

Review lane: proof
Safety invariant: Verification is read-only and alters no repository file.
Effectiveness measurement: The command's exit code is the direct measurement.
Slice rationale: One check per proof task.
Architectural effect: None; verification only.
Goal: Prove the slice.
Motivation: Running the check is the proof.
Alternative considerations: The full suite was not required because the slice touches one component with its own tests.
Implementation details: Run `python3 scripts/check_hook_test_coverage.py`.
Non-goals: No mutations.
Layer: app_regression
Feature state: active
Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry.
Acceptance criteria:
- The command exits 0.

Exit code: 1
…view claim: `python3 scripts/check_no_new_comments.py` passes on the finished branch.

Review lane: proof
Safety invariant: Verification is read-only and alters no repository file.
Effectiveness measurement: The command's exit code is the direct measurement.
Slice rationale: One check per proof task.
Architectural effect: None; verification only.
Goal: Prove the slice.
Motivation: Running the check is the proof.
Alternative considerations: The full suite was not required because the slice touches one component with its own tests.
Implementation details: Run `python3 scripts/check_no_new_comments.py`.
Non-goals: No mutations.
Layer: app_regression
Feature state: active
Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry.
Acceptance criteria:
- The command exits 0.

Exit code: 1
…view claim: `python3 scripts/check_hook_test_coverage.py` passes on the finished branch.

Review lane: proof
Safety invariant: Verification is read-only and alters no repository file.
Effectiveness measurement: The command's exit code is the direct measurement.
Slice rationale: One check per proof task.
Architectural effect: None; verification only.
Goal: Prove the slice.
Motivation: Running the check is the proof.
Alternative considerations: The full suite was not required because the slice touches one component with its own tests.
Implementation details: Run `python3 scripts/check_hook_test_coverage.py`.
Non-goals: No mutations.
Layer: app_regression
Feature state: active
Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry.
Acceptance criteria:
- The command exits 0.

Solution:
  Review claim: `python3 scripts/check_hook_test_coverage.py` passes on the finished branch.
Review lane: proof
Safety invariant: Verification is read-only and alters no repository file.
Effectiveness measurement: The command's exit code is the direct measurement.
Slice rationale: One check per proof task.
Architectural effect: None; verification only.
Goal: Prove the slice.
Motivation: Running the check is the proof.
Alternative considerations: The full suite was not required because the slice touches one component with its own tests.
Implementation details: Run `python3 scripts/check_hook_test_coverage.py`.
Non-goals: No mutations.
Layer: app_regression
Feature state: active
Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry.
Acceptance criteria:
- The command exits 0.
…view claim: `python3 scripts/check_no_new_comments.py` passes on the finished branch.

Review lane: proof
Safety invariant: Verification is read-only and alters no repository file.
Effectiveness measurement: The command's exit code is the direct measurement.
Slice rationale: One check per proof task.
Architectural effect: None; verification only.
Goal: Prove the slice.
Motivation: Running the check is the proof.
Alternative considerations: The full suite was not required because the slice touches one component with its own tests.
Implementation details: Run `python3 scripts/check_no_new_comments.py`.
Non-goals: No mutations.
Layer: app_regression
Feature state: active
Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry.
Acceptance criteria:
- The command exits 0.

Solution:
  Review claim: `python3 scripts/check_no_new_comments.py` passes on the finished branch.
Review lane: proof
Safety invariant: Verification is read-only and alters no repository file.
Effectiveness measurement: The command's exit code is the direct measurement.
Slice rationale: One check per proof task.
Architectural effect: None; verification only.
Goal: Prove the slice.
Motivation: Running the check is the proof.
Alternative considerations: The full suite was not required because the slice touches one component with its own tests.
Implementation details: Run `python3 scripts/check_no_new_comments.py`.
Non-goals: No mutations.
Layer: app_regression
Feature state: active
Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry.
Acceptance criteria:
- The command exits 0.
…view claim: `python3 scripts/check_hook_test_coverage.py` passes on the finished branch.

Review lane: proof
Safety invariant: Verification is read-only and alters no repository file.
Effectiveness measurement: The command's exit code is the direct measurement.
Slice rationale: One check per proof task.
Architectural effect: None; verification only.
Goal: Prove the slice.
Motivation: Running the check is the proof.
Alternative considerations: The full suite was not required because the slice touches one component with its own tests.
Implementation details: Run `python3 scripts/check_hook_test_coverage.py`.
Non-goals: No mutations.
Layer: app_regression
Feature state: active
Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry.
Acceptance criteria:
- The command exits 0.

Exit code: 0
…view claim: `python3 scripts/check_no_new_comments.py` passes on the finished branch.

Review lane: proof
Safety invariant: Verification is read-only and alters no repository file.
Effectiveness measurement: The command's exit code is the direct measurement.
Slice rationale: One check per proof task.
Architectural effect: None; verification only.
Goal: Prove the slice.
Motivation: Running the check is the proof.
Alternative considerations: The full suite was not required because the slice touches one component with its own tests.
Implementation details: Run `python3 scripts/check_no_new_comments.py`.
Non-goals: No mutations.
Layer: app_regression
Feature state: active
Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry.
Acceptance criteria:
- The command exits 0.

Exit code: 0
…o ephemeral inter-task handoff files remain in the worktree before the merge gate.

Review lane: cleanup
Safety invariant: The scrub script only checks for known handoff artifact names and never touches source, tests, or other repository files.
Effectiveness measurement: The script exits non-zero if any handoff artifact remains.
Slice rationale: Required terminal scrub for every implementation workflow.
Architectural effect: None; hygiene only.
Goal: Leave the branch free of handoff artifacts.
Motivation: Handoff files must not reach the PR.
Alternative considerations: Manual cleanup was set aside as non-deterministic.
Implementation details: Run scripts/scrub-handoff-artifacts.sh.
Non-goals: No product edits.
Layer: app_regression
Feature state: active
Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry.
Acceptance criteria:
- The command exits 0.

Exit code: 0
…37cebe4c-08bdbf79 — Review claim: No ephemeral inter-task handoff files remain in the worktree before the merge gate.

Review lane: cleanup
Safety invariant: The scrub script only checks for known handoff artifact names and never touches source, tests, or other repository files.
Effectiveness measurement: The script exits non-zero if any handoff artifact remains.
Slice rationale: Required terminal scrub for every implementation workflow.
Architectural effect: None; hygiene only.
Goal: Leave the branch free of handoff artifacts.
Motivation: Handoff files must not reach the PR.
Alternative considerations: Manual cleanup was set aside as non-deterministic.
Implementation details: Run scripts/scrub-handoff-artifacts.sh.
Non-goals: No product edits.
Layer: app_regression
Feature state: active
Layer exception: allowed. Verification and the terminal scrub run after the docs commit so they check the final branch the PR will carry.
Acceptance criteria:
- The command exits 0.
A PR body the repo's validator accepted was reported to the agent as
"could not check", and an owed stack follow-up stayed armed.

The hook treats exit 0 alongside an UNCHECKED/SKIPPED/not-installed line
as a vacuous pass, so the validator gets no credit for a run that never
judged the body. The catstack validator prints exactly such a line for a
sub-check it skipped -- "Summary reading grade unchecked: Summary has N
words; under 30 the score is too noisy to trust" -- while still accepting
the body and exiting 0. Every accepted body with a short Summary was
therefore reported unchecked.

check_body_file now lifts the vacuous reading when the run states its own
verdict on the body (VALIDATOR_PASS_VERDICT_RE, "PR body validation
passed"). Only a pass verdict counts, so a "failed" banner beside exit 0
stays unchecked. The scan also reads the whole output rather than the
first VALIDATOR_OUTPUT_MAX_LINES: truncation shortens what the agent is
shown, never what is judged.

Two tests in tests/test_advisory.py drive a stub validator with the
catstack shape (skip note on stderr, verdict on stdout, exit 0): one
asserts the write is silent, one asserts it clears the pending follow-up.
Both fail before this change. test_validator_exit_zero_with_an_unchecked_
line_is_not_clean still passes, so the guard against a validator that
really did not run is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ead PRRT_kwDOT3uYWs6lQWRD after PR head changes

Exit code: 0
…1-36089247 — Resolve bot review thread PRRT_kwDOT3uYWs6lQWRD after PR head changes

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 6315464. Configure here.

with open(os.path.join(tmp.name, "scripts", "create-pr.mjs"), "w") as f:
f.write("// stub\n")
with open(os.path.join(tmp.name, "scripts", "validate-pr-body.mjs"), "w") as f:
f.write("// stub\n")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stub validator flakes in-scope write tests

Medium Severity

_repo_with_tool now writes a validate-pr-body.mjs stub that exits 0 with no output. check_body_file treats that silent pass as clean, so gh pr edit --body-file /tmp/body.md emits nothing when the file already exists. test_never_exits_nonzero_for_any_pr_write_in_scope and test_codex_nested_workdir_can_select_repo_with_tool then fail because they require an advisory line.

Additional Locations (2)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 6315464. Configure here.

@mergify

mergify Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

This pull request does not currently match the merge queue conditions, so it cannot be queued from here. The box comes back if it matches again.

@EdbertChan EdbertChan closed this Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant