fix(skill-evals): security eval fixtures that failed correct answers, and the triage suite's old taxonomy

[potiuk]

Follow-ups from the security-family optimization (#1344): eval fixtures that failed correct answers, and the triage suite's outdated taxonomy.

Fixture fixes

eval	problem	fix
issue-fix 5b case 3	The expected description endorsed an injected BYPASS_AUTH = True snippet, which the model rightly rejects	The description now says the snippet is untrusted and not adopted
issue-fix 5c case 3	Expected both security fix and an overlapping bare security	The overlap is dropped
issue-fix 5g	The spec asked for "forbidden strings", but grading used occurrences	The spec asks for each occurrence as it appears
invalidate step 4	Graded the open-ended other_applicable list against []	No longer graded; the selected section still is
import 2b	Unclear when the summary is null, and #NNN vs URL	The spec says when it is null and asks for #NNN
import 5	Unclear whether a receipt counts as a canned response	Null for an imported report
cve-allocate step 1 case 2	The input has two hard blockers; the expected answer listed one	Both listed
cve-allocate step 2	The model miscounted words	The spec says to count tokens, with an example
forwarder step 1	The snippet length varied with "~80 characters"	The snippet is the matched text up to the first line break

Triage suite taxonomy

The security-issue-triage suite still used the old five-class set (NOT-CVE-WORTHY). It now uses the skill's six classes, INVALID plus the added FIX-ALREADY-PUBLIC, across 28 files: system prompts, expectations, distribution keys, the step-2.6 signal enum and the README. The label and heading text "not CVE worthy" is unchanged, and fixture directories keep their names.

Test plan

• Hooks on commit
• The nine fixed steps, three runs each after the change:
  • Seven pass 3/3.
  • issue-fix 5c and forwarder step 1 each failed once in three runs, on different cases (model variance).
• Triage suite, once, after the migration:
  • Six steps fully green; the step-3 case that failed once passed on re-run.
  • step-2.6 case 2 and step-2a case 1 fail the same way on main. They predate this change and are left for a separate look.
• No new case exercises FIX-ALREADY-PUBLIC yet.

Part of #1344.

Generated-by: Claude Code (Opus 5.5)

🤖 Generated with Claude Code