Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/ecosystem.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ again.
| `bound-tool-result` | hook (PreToolUse on native shell for Claude, Cursor, and Codex; rewrites through capture_tool_result.py so parent-visible stdout/stderr never exceeds 16KiB; full bytes on disk; MCP/Read/screenshots/hosted tools are named V1 gaps) |
| `bug-complaint-leak` | hook |
| `publish-act-guard` | hook |
| `serial-option-guard` | hook (PreToolUse on `AskUserQuestion`; blocks a `(Recommended)` option that works several PR stacks, rebases, or repairs one at a time in the parent chat, unless cat-mode's `route_execution.py` ran with `units=N` this session; the meaning is judged by llm-judge's `serial-option-guard` phrase dictionary; fails open as unchecked when the transcript or judge is unavailable) |
| `categorical-scope-guard` | hook (PreToolUse on `Bash`; blocks a status-narrowed mutation when the live turn said all/every/each) |
| `cat-mode-default` | hook (UserPromptSubmit + PreToolUse on `Agent`; applies `cat-mode` on every prompt and on subagent prompts when `CATSTACK_CAT_MODE_DEFAULT=on`) |
| `demo-freeze` | hook |
Expand Down
5 changes: 5 additions & 0 deletions engine/hooks/hooks.toml
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,11 @@ mode = "stop"
why_mode = "outward"
summary = "Stops two agents writing one temp file."

[hooks.serial-option-guard]
mode = "stop"
why_mode = "attention"
summary = "Stops recommending many PR stacks one at a time here."

[hooks.skill-usage-log]
mode = "off"
why_mode = "habit"
Expand Down
5 changes: 4 additions & 1 deletion engine/hooks/llm-judge/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,10 @@ the dictionary's `on_hit` text.

`ask(prompt)` tries these in order and stops at the first one that answers:

1. **codex**: `codex exec --skip-git-repo-check -m gpt-5.3-codex-spark --sandbox read-only -c notify=[] PROMPT`
1. **codex**: `codex exec --skip-git-repo-check --sandbox read-only -c notify=[] PROMPT`
(no `-m`: codex runs the model set in `~/.codex/config.toml`, so the judge
uses a model the account can already call; a ChatGPT-account login refuses
API-only models)
2. **claude**: `claude -p --model haiku --settings '{"disableAllHooks": true}' PROMPT`
3. **cursor**: `cursor-agent -p --output-format text PROMPT`

Expand Down
2 changes: 1 addition & 1 deletion engine/hooks/llm-judge/judge.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
RUNNERS_ENV = "CATSTACK_LLM_JUDGE_RUNNERS"
STATE_ENV = "CATSTACK_LLM_JUDGE_STATE_DIR"
DEFAULT_RUNNERS = (
("codex", ["codex", "exec", "--skip-git-repo-check", "-m", "gpt-5.3-codex-spark", "--sandbox", "read-only", "-c", "notify=[]", PROMPT_SLOT]),
("codex", ["codex", "exec", "--skip-git-repo-check", "--sandbox", "read-only", "-c", "notify=[]", PROMPT_SLOT]),
("claude", ["claude", "-p", "--model", "haiku", "--settings", '{"disableAllHooks": true}', PROMPT_SLOT]),
("cursor", ["cursor-agent", "-p", "--output-format", "text", PROMPT_SLOT]),
)
Expand Down
23 changes: 23 additions & 0 deletions engine/hooks/llm-judge/phrases/restated-rule.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"checker": "restated-rule",
"reads": "user",
"meaning": "The person reminds the agent of a rule, skill, convention, or standing decision that already exists, because the agent's last action did not follow it.",
"match": [
"I thought that was a skill that we enforce in catstack? /reflect",
"I believe that when we do this, we should be handling these in separate worktrees in subagent for each stack?",
"isn't there already a rule that says to run it under bash?",
"we already decided DO1 is production, why are you asking again",
"the pr skill says to do this, did you not read it?"
],
"not_match": [
"please land all prs labeled admin-bypass in both invoker and in catstack. make sure they pass CI",
"make that fix for catmode",
"Yes do it locally",
"go",
"the UI is messed up on the workers page",
"why did you choose Saturday for the deploy window?",
"we don't have a rule for this yet, can you add one?",
"1. yes 2. hold 3. all 4. yes. 5. how much tokens is this all costing me so far?"
],
"on_hit": "restated-rule: the person had to restate a rule or skill that already exists. Count it as an intervention; a restated rule the agent should already follow is automate-me evidence."
}
20 changes: 20 additions & 0 deletions engine/hooks/llm-judge/phrases/serial-option-guard.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
{
"checker": "serial-option-guard",
"reads": "reply",
"meaning": "A menu option the assistant marked as recommended proposes that the assistant itself works through several independent publishing units (PR stacks, PR rebases, PR repairs, branches to fix and push) one after another in this chat, instead of running them as parallel isolated units.",
"match": [
"Question: ~22 PRs need rebases or test fixes. Where should that work run?\nRecommended option: Here, one at a time (Recommended)\nDescription: I rebase/fix and force-push each branch myself, requeue it, and keep watching until the queue drains. Slow (hours) but you see every step.",
"Recommended option: Sequentially in this chat (Recommended)\nDescription: I work through the 9 stacks one after another and push each as I go.",
"Recommended option: Stack by stack, myself (Recommended)\nDescription: Rebase each of the six stacks here in turn and requeue it before starting the next.",
"Recommended option: Loop here (Recommended)\nDescription: Go down the list of 15 failing PRs and fix each in this session."
],
"not_match": [
"Recommended option: Invoker workflows (Recommended)\nDescription: Submit one fix workflow per stuck PR to Invoker; I watch and requeue.",
"Recommended option: One worktree subagent per stack, in parallel (Recommended)\nDescription: Each stack gets its own isolated worker; I collect the reports.",
"Recommended option: Here, one at a time (Recommended)\nDescription: I fix the one failing test on this branch and push it.",
"Recommended option: Do it here (Recommended)\nDescription: Read the 5 failing logs and report which ones share a cause.",
"Recommended option: Close as superseded (Recommended)\nDescription: Check each has no content vs master first, then close; retarget any child PR onto the real base.",
"Recommended option: Fix bot, ask on human (Recommended)\nDescription: Fix real bot findings in code, reply+resolve bot nits. Leave human threads for you and list them."
],
"on_hit": "serial-option-guard: the (Recommended) option works several independent publishing units (PR stacks, rebases, repairs) one at a time in this chat, and this session has not run the routing check. Run cat-mode's scripts/route_execution.py with units set to the number of units first, for example python3 route_execution.py '{\"units\": 22, \"work_kind\": \"durable_parallel\", \"tools\": [...]}', then recommend the route it returns (an Invoker workflow per unit, else one worktree subagent per unit). A serial option may stay in the menu, but not as the recommended one until that check has run."
}
7 changes: 7 additions & 0 deletions engine/hooks/llm-judge/tests/test_judge.py
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,13 @@ def test_default_runner_order_is_codex_then_claude_then_cursor(self):
os.environ.pop(judge.RUNNERS_ENV)
self.assertEqual([name for name, _ in judge.runners()], ["codex", "claude", "cursor"])

def test_default_codex_runner_uses_the_account_model_not_a_pinned_one(self):
with patch.dict(os.environ):
os.environ.pop(judge.RUNNERS_ENV)
codex_argv = dict(judge.runners())["codex"]
self.assertNotIn("-m", codex_argv)
self.assertNotIn("--model", codex_argv)

def test_investigate_runner_argv_is_read_only_and_excludes_cursor(self):
os.environ.pop(judge.RUNNERS_ENV)
self.assertEqual(
Expand Down
72 changes: 72 additions & 0 deletions engine/hooks/serial-option-guard/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# serial-option-guard

Blocks an `AskUserQuestion` menu whose `(Recommended)` option works several
independent publishing units (PR stacks, rebases, repairs) one at a time in
the parent chat, unless the routing check already ran this session.

Owning rule: cat-mode's "Many PR stacks: one parallel unit per stack, never
serial", with `corpus/skills/cat-mode/scripts/route_execution.py` as the
routing check.

## How it decides

Claude `PreToolUse` on `AskUserQuestion` (`claude_pretooluse.py` ->
`detect.py`).

1. Parse the menu. Only options whose label carries the fixed
`(Recommended)` marker are considered. No marker, no check.
2. Parse the transcript's tool calls. If a successful `Bash` call ran
`route_execution` with a `units` value (`units=9`, `"units": 22`) and its
output names a route, the menu is allowed. Reading the script with `cat`
or `grep` does not count; a call that errored does not count.
3. Otherwise ask the shared llm-judge whether the recommended option means
"the assistant works through many publishing units one after another in
this chat". The meaning lives in the phrase dictionary
[`engine/hooks/llm-judge/phrases/serial-option-guard.json`](../llm-judge/phrases/serial-option-guard.json),
not in a word list here. The hook waits for that one verdict (default 40
seconds, `SERIAL_OPTION_GUARD_WAIT_SECONDS`) so a hit blocks the menu
before the user sees it. It takes only its own verdict file, so other
hooks' verdicts stay in the llm-judge inbox.

## Fires on

The real menu option `Here, one at a time (Recommended)` for "~22 PRs need
rebases or test fixes. Where should that work run?"
(`tests/fixtures/real_serial_recommended.json`), when the judge says it
matches.

## Silent on

- a serial option with no `(Recommended)` marker;
- a recommended parallel option (`Invoker workflows (Recommended)`), a
single-unit option, or a read-only option, when the judge says clean;
- any menu after `route_execution.py` ran with `units=N` this session;
- subagent payloads and other tools.

## Block message

The dictionary's `on_hit` text: run `route_execution.py` with `units` set to
the number of units, then recommend the route it returns (an Invoker
workflow per unit, else one worktree subagent per unit). A serial option may
stay in the menu, but not as the recommended one until the check has run.

## Fail direction

Fails open, and says so. A missing or unreadable transcript, a judge with no
working runner, or a judge that does not answer in time lets the menu
through and writes `serial-option-guard: unchecked, ...` to stderr. A late
verdict still lands in the llm-judge inbox and reaches the agent after the
next tool call, before the serial work starts. A crash in the detector is
logged by the hook SDK and allows.

## Escape hatch

`CATSTACK_HOOK_MODE_SERIAL_OPTION_GUARD=off` (or `warn`). Grow coverage by
adding the real text of a miss to the dictionary's `match` list, or of a
false alarm to `not_match`.

## Tests

```
python3 -m unittest discover -s engine/hooks/serial-option-guard/tests -v
```
16 changes: 16 additions & 0 deletions engine/hooks/serial-option-guard/claude.hook.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"hooks": {
"PreToolUse": [
{
"matcher": "AskUserQuestion",
"hooks": [
{
"type": "command",
"command": "python3 $HOME/.claude/hooks/serial-option-guard/claude_pretooluse.py",
"timeout": 60
}
]
}
]
}
}
20 changes: 20 additions & 0 deletions engine/hooks/serial-option-guard/claude_pretooluse.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#!/usr/bin/env python3
"""Claude Code PreToolUse entrypoint for serial-option-guard."""
from __future__ import annotations

import os
import sys

sys.path.insert(0, os.path.join(
os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "_sdk"))

from detect import detect # noqa: E402
from runtime import run_hook # noqa: E402


def main() -> None:
run_hook("serial-option-guard", "claude", detect, "PreToolUse")


if __name__ == "__main__":
main()
Loading
Loading