Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 17 additions & 9 deletions corpus/skills/principle-explicit-errors/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,21 @@ disable-model-invocation: true
# Make errors and failures explicit

Every failure path must make its disposition visible: fail fast, propagate,
translate into a named domain result, or suppress only a narrow, documented
case whose absence is safe and observable when that assumption changes.
translate into a named domain result, or log the exception with its context.
Never suppress an exception. Silence is not a policy.
Each rule below restates standard practice (see Grounding).

## Exceptions

An empty or comments-only handler is not an error policy. It hides failures
from callers, logs, tests, and maintainers. Do not write `catch {}`,
`except: pass`, ignored promise rejections, or equivalent silent fallbacks.
Replace them with an explicit action and keep the original error context.
An empty handler, a comment-only handler, and a handler whose only action
is to discard the error (`catch {}`, `void err`, `except: pass`,
`.catch(() => {})`) hide the failure from callers, logs, tests, and
maintainers. Do not write them.

If the exception must not propagate — a diagnostic callback, a metric
write, a best-effort side path — log it, then continue. The log names the
operation and includes the original exception. A comment that says why you
caught it is not a log. `void err` is not a log.

Mechanical enforcement: `engine/hooks/explicit-failures` (advisory PreToolUse hook, on by default; its README lists the shapes it catches).

Expand Down Expand Up @@ -85,9 +90,12 @@ The repo context is a batch data pipeline (filings in, CSV grids out), not a
long-running service; each line says how the source fits that.

- Silent handlers → Tim Peters, PEP 20 "The Zen of Python" (2004): "Errors
should never pass silently. Unless explicitly silenced."
<https://peps.python.org/pep-0020/>; Joshua Bloch, *Effective Java* 3rd ed.
(2018), Item 77 "Don't ignore exceptions". A batch job that swallows an
should never pass silently." <https://peps.python.org/pep-0020/>. This
skill does not take the next sentence ("Unless explicitly silenced") as
permission to drop an exception. Joshua Bloch, *Effective Java* 3rd ed.
(2018), Item 77 "Don't ignore exceptions". When the exception must not
propagate, log it with the original exception attached. A comment, an
empty catch, or `void err` is not that log. A batch job that swallows an
error ships a wrong CSV under a green run, so this applies as written.
- Fail immediately and visibly → Jim Shore, "Fail Fast", *IEEE Software*
21(5) (2004) <https://martinfowler.com/ieeeSoftware/failFast.pdf>. Shore's
Expand Down
10 changes: 7 additions & 3 deletions corpus/skills/principle-explicit-errors/tests/fires_example.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
An agent is adding a fallback around an optional dependency and proposes
`try { loadOptionalModule() } catch {}`. The agent invokes
`/principle-explicit-errors`, names the expected import failure, and replaces
the silent handler with an explicit fallback plus a test for unexpected errors.
`try { loadOptionalModule() } catch {}`. A review bot then "fixes" a
diagnostic callback by replacing `void reportingFailure` with an empty
`catch {}` so the callback cannot change the caller's outcome. The agent
invokes `/principle-explicit-errors`: neither shape is allowed. If the
exception must not propagate, the handler logs the operation and the
original exception, and a test asserts that log. An empty catch, a
comment-only catch, and `void err` are all suppression.

A second shape, same skill, no exception in sight. A realized-gains tab
passes "every value traces to a source", then a review pass finds a sell
Expand Down
28 changes: 28 additions & 0 deletions corpus/skills/principle-explicit-errors/tests/test_void_discard.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
"""The skill's discard rule is the same check the failure hook runs."""
from __future__ import annotations

import os
import sys
import unittest

HOOK_DIR = os.path.abspath(
os.path.join(os.path.dirname(__file__), "..", "..", "..", "..", "engine", "hooks", "explicit-failures")
)
sys.path.insert(0, HOOK_DIR)

import detect # noqa: E402


class VoidDiscardTest(unittest.TestCase):
def test_void_only_catch_is_reported(self):
text = "try { a() } catch (err) {\n void err;\n}\n"
hits = detect.scan_js(text)
self.assertEqual(hits, [(1, "catch block that only discards the error with `void`")])

def test_logged_catch_is_quiet(self):
text = "try { a() } catch (err) {\n console.error('failed', err);\n}\n"
self.assertEqual(detect.scan_js(text), [])


if __name__ == "__main__":
unittest.main()
5 changes: 5 additions & 0 deletions engine/hooks/hooks.toml
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,11 @@ summary = "Stops drift after the user narrowed the task twice."
enabled_by = "CATSTACK_REFLECT_ENFORCEMENT"
rule_modes = { "scope-lock.prompt-instruction" = "warn", "scope-lock.reflection-acknowledged" = "warn" }

[hooks.spreadsheet-completeness-guard]
mode = "stop"
why_mode = "attention"
summary = "Blocks spreadsheet completion claims without source-backed entity coverage."

[hooks.scratchpad-collision]
mode = "stop"
why_mode = "outward"
Expand Down
8 changes: 8 additions & 0 deletions engine/hooks/spreadsheet-completeness-guard/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# spreadsheet-completeness-guard

Stop hook for spreadsheet completion claims. A completion claim must carry a
source-backed `SPREADSHEET_COVERAGE: PASS` receipt from the expected-entity
validator. Missing observations must be explicitly classified; estimates and
zeros are not substitutes for absent competitor data.

Tests: `python3 -m unittest discover -s engine/hooks/spreadsheet-completeness-guard/tests -v`
5 changes: 5 additions & 0 deletions engine/hooks/spreadsheet-completeness-guard/claude.hook.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"hooks": {
"Stop": [{"matcher": "*", "hooks": [{"type": "command", "command": "python3 $HOME/.claude/hooks/spreadsheet-completeness-guard/claude_stop_check.py", "timeout": 10}]}]
}
}
10 changes: 10 additions & 0 deletions engine/hooks/spreadsheet-completeness-guard/claude_stop_check.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/usr/bin/env python3
from __future__ import annotations
import os
import sys
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "_sdk"))
from detect import detect # noqa: E402
from runtime import run_hook # noqa: E402

if __name__ == "__main__":
run_hook("spreadsheet-completeness-guard", "claude", detect, "Stop", json_error_stderr=False)
5 changes: 5 additions & 0 deletions engine/hooks/spreadsheet-completeness-guard/codex.hook.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"hooks": {
"Stop": [{"matcher": "*", "hooks": [{"type": "command", "command": "python3 $HOME/.codex/hooks/spreadsheet-completeness-guard/codex_stop_check.py", "timeout": 10}]}]
}
}
10 changes: 10 additions & 0 deletions engine/hooks/spreadsheet-completeness-guard/codex_stop_check.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/usr/bin/env python3
from __future__ import annotations
import os
import sys
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "_sdk"))
from detect import detect # noqa: E402
from runtime import run_hook # noqa: E402

if __name__ == "__main__":
run_hook("spreadsheet-completeness-guard", "codex", detect, "Stop")
5 changes: 5 additions & 0 deletions engine/hooks/spreadsheet-completeness-guard/cursor.hook.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"hooks": {
"stop": [{"command": "python3 $HOME/.cursor/hooks/spreadsheet-completeness-guard/cursor_stop_check.py", "timeout": 10}]
}
}
10 changes: 10 additions & 0 deletions engine/hooks/spreadsheet-completeness-guard/cursor_stop_check.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/usr/bin/env python3
from __future__ import annotations
import os
import sys
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "_sdk"))
from detect import detect # noqa: E402
from runtime import run_hook # noqa: E402

if __name__ == "__main__":
run_hook("spreadsheet-completeness-guard", "cursor", detect, "stop")
55 changes: 55 additions & 0 deletions engine/hooks/spreadsheet-completeness-guard/detect.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
"""Stop completion claims for spreadsheet work without a coverage receipt."""
from __future__ import annotations

import re
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "_sdk"))
from finding import Finding # noqa: E402

RULE_INCOMPLETE = "spreadsheet-completeness-guard.incomplete"
SPREADSHEET_RE = re.compile(r"\b(?:spreadsheet|google sheet|google sheets|workbook|raw tab|aggregation table|chart)\b", re.I)
COMPLETION_RE = re.compile(r"\b(?:complete|completed|done|filled|populated|regenerated|ready|all competitors)\b", re.I)
RECEIPT_RE = re.compile(r"SPREADSHEET_COVERAGE\s*:\s*PASS\b", re.I)
FAIL_RE = re.compile(r"SPREADSHEET_COVERAGE\s*:\s*FAIL\b", re.I)


def claims_completion(message: str) -> bool:
return bool(SPREADSHEET_RE.search(message or "") and COMPLETION_RE.search(message or ""))


def has_receipt(message: str) -> bool:
return bool(RECEIPT_RE.search(message or ""))


def decide(payload: dict) -> Finding | None:
message = str(payload.get("last_assistant_message") or payload.get("message") or "")
if not claims_completion(message) or has_receipt(message):
return None
try:
retry_count = int(payload.get("spreadsheet_retry_count") or 0)
except (TypeError, ValueError):
retry_count = 0
retry_count = max(0, min(retry_count, 3))
if FAIL_RE.search(message):
return Finding(
rule_id=RULE_INCOMPLETE,
subject="spreadsheet coverage failure",
message="Coverage failed after the bounded retry loop. Do not generate or declare the table/chart complete; list the unresolved entity/period keys.",
evidence=f"SPREADSHEET_COVERAGE: FAIL; retry {retry_count}/3",
)
return Finding(
rule_id=RULE_INCOMPLETE,
subject="spreadsheet completion claim",
message=(f"Do not declare spreadsheet data complete until the expected-entity coverage "
f"validator has passed. Run source collection retry {min(retry_count + 1, 3)}/3; "
"add source-backed rows or state unresolved entity/period keys. "
"Only SPREADSHEET_COVERAGE: PASS permits completion."),
evidence=f"completion claim without a coverage receipt; retry {retry_count}/3",
)


def detect(event: dict[str, object]) -> list[Finding]:
finding = decide(event)
return [] if finding is None else [finding]
38 changes: 38 additions & 0 deletions engine/hooks/spreadsheet-completeness-guard/tests/test_hooks.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
import os
import sys
import unittest

sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import detect # noqa: E402


class SpreadsheetCompletenessTests(unittest.TestCase):
def test_blocks_completion_without_coverage_receipt(self):
finding = detect.decide({"last_assistant_message": "The spreadsheet is complete and all competitors are populated."})
self.assertIsNotNone(finding)
self.assertEqual(finding.rule_id, "spreadsheet-completeness-guard.incomplete")

def test_stays_silent_with_pass_receipt(self):
finding = detect.decide({"last_assistant_message": "SPREADSHEET_COVERAGE: PASS — the workbook is complete."})
self.assertIsNone(finding)

def test_stays_silent_for_non_spreadsheet_completion(self):
self.assertIsNone(detect.decide({"last_assistant_message": "The README edit is complete."}))

def test_requests_bounded_retry_when_coverage_is_missing(self):
finding = detect.decide({
"last_assistant_message": "The spreadsheet is complete.",
"spreadsheet_retry_count": 1,
})
self.assertIn("retry 2/3", finding.message)

def test_keeps_blocking_after_three_failed_retries(self):
finding = detect.decide({
"last_assistant_message": "SPREADSHEET_COVERAGE: FAIL — the spreadsheet is complete.",
"spreadsheet_retry_count": 3,
})
self.assertIn("after the bounded retry loop", finding.message)


if __name__ == "__main__":
unittest.main()
Loading