The guard that refuses a file-set rule carrying a stray tests field throws a CLIError without { reported: true }, from inside the command's try and outside any fail() helper. Under --json that produces no envelope at all.
// packages/cli/src/commands/rules.ts:263-268 (create), and again ~:540-545 (improve)
throw new CLIError(
`Rule "${rule.id}" was delivered as a file set and also carries \`tests\`; ...`,
"RULE_GENERATION_FAILED"
);
index.ts:136-141 handles a CLIError that is not reported by writing Error: <message> to stderr and setting exitCode = 1. Nothing is written to stdout.
What an agent sees
create-remote-rule.txt:171-182 promises { ok: false, code, message } on stdout and lists RULE_GENERATION_FAILED as a code to branch on. The agent gets:
- stdout: empty
- stderr: prose
- exit: 1
which is indistinguishable from a crash, and gives it nothing to branch on. Rules already written earlier in the same loop are also never reported, so the agent does not learn what landed on disk before the refusal.
Two copies
The same guard is duplicated at commands/rules.ts:253-267 and around :536-550. Neither has any test. If one stopped firing, or the two drifted, a file set's fixtures would be silently dropped.
Suggested fix
Route both through the command's existing fail() helper, which writes the envelope under --json and marks the error reported. Then add a test that drives rule create --json and rule improve --json against a mocked payload carrying both files and tests, asserting the envelope's code rather than just a non-zero exit.
This is the same seam that produced #264's fix today: a unit test proving the function throws correctly says nothing about whether the command reports it correctly.
Found in a pre-release review of v0.11.0..main.
The guard that refuses a file-set rule carrying a stray
testsfield throws aCLIErrorwithout{ reported: true }, from inside the command'stryand outside anyfail()helper. Under--jsonthat produces no envelope at all.index.ts:136-141handles aCLIErrorthat is notreportedby writingError: <message>to stderr and settingexitCode = 1. Nothing is written to stdout.What an agent sees
create-remote-rule.txt:171-182promises{ ok: false, code, message }on stdout and listsRULE_GENERATION_FAILEDas a code to branch on. The agent gets:which is indistinguishable from a crash, and gives it nothing to branch on. Rules already written earlier in the same loop are also never reported, so the agent does not learn what landed on disk before the refusal.
Two copies
The same guard is duplicated at
commands/rules.ts:253-267and around:536-550. Neither has any test. If one stopped firing, or the two drifted, a file set's fixtures would be silently dropped.Suggested fix
Route both through the command's existing
fail()helper, which writes the envelope under--jsonand marks the error reported. Then add a test that drivesrule create --jsonandrule improve --jsonagainst a mocked payload carrying bothfilesandtests, asserting the envelope'scoderather than just a non-zero exit.This is the same seam that produced #264's fix today: a unit test proving the function throws correctly says nothing about whether the command reports it correctly.
Found in a pre-release review of
v0.11.0..main.