tests: run every suite hermetic, with none of this machine's settings

[EdbertChan]

Summary

Catstack comes with automatic self-checks that prove its safety checkers work. They should give the same result on any machine.

The problem: the self-checks read the personal settings of whoever ran them. On one machine, seven failed while the code was fine.

The cause: the run kept the person's home folder, shell settings and project settings, and the self-checks read them.

The fix: before anything runs, the runner clears those out and hands over an empty home folder. A new self-check fails if anything gets through.

Review Claim

A full self-check run gives the same result no matter whose machine runs it.

Review Lane

behavior

Review Unit

engine-runtime

Safety Invariant

Hooks on a real machine behave exactly as before: the flag reader's new skip list is empty unless set, and only the test runner and the ledger tests set it.

Slice Rationale

One claim, hermetic tests. The setup script, the reader's skip list, the guard test, and the four test files the stricter run exposed ship together because the runner fails on those four until they stop reading the machine.

Non-goals

• No flag changes its default, values, or lookup order.
• A suite run on its own with python3 -m unittest outside the runner still sees the shell; only the ledger suites protect themselves there.
• The self-referencing node_modules/node_modules link in one local checkout is not touched.

Test Plan

Test Plan

• Fail before, with CATSTACK_UNVERIFIED_TAG_BEHAVIOR=do_not_emit in ~/.catstack.env: python3 -m unittest discover -s engine/hooks/unverified-tag-ledger/tests on origin/main → FAILED (failures=7); python3 -m unittest tests/test_hermetic_test_env.py → FAILED (failures=4) (setup script absent).
• Full run under the stricter setup before fixing the exposed tests: 14 failures (test_judge read the real codex install; runner tests lacked the new script; install tests hit a local node_modules link loop, cleared by a real npm ci).
• Pass after, launched from a polluted shell: CATSTACK_UNVERIFIED_TAG_BEHAVIOR=do_not_emit CATSTACK_HOOK_MODE_DIU_STOP=off bash scripts/test/run_all_tests.sh → exit=0, 0 FAIL:/ERROR: lines, 3365 tests run, no catstack-test-home.* folder left behind.

Revert Plan

Revert Plan

• Safe to revert? Yes
• Revert command: git revert <sha>
• Post-revert steps: None
• Data migration? No

🤖 Generated with Claude Code

---

Note

Low Risk
Production hook behavior is unchanged unless CATSTACK_SKIP_ENV_FILES is set; only the test runner and explicit test setup use it.

Overview
Makes scripts/test/run_all_tests.sh source a new hermetic_env.sh that strips inherited CATSTACK_*, GIT_*, and related vars, points HOME at a temp directory with isolated git config, and sets CATSTACK_SKIP_ENV_FILES so the real ~/.catstack.env and repo .env are not consulted during the full run.

The shared flag reader gains optional CATSTACK_SKIP_ENV_FILES (path-separated, realpath-matched) so listed env files drop out of env_file_candidates without changing default lookup order when unset. Ledger hook suites set the same skip when run standalone; tests/test_hermetic_test_env.py guards that the setup actually isolates the runner.

Supporting fixes: LLM judge tests stub Codex config/catalog so defaults do not depend on the host install; runner fake-repo tests copy hermetic_env.sh; exit traps also remove CATSTACK_TEST_HOME.

^{Reviewed by Cursor Bugbot for commit 20ae939. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_ca4a2015-8f21-46d5-88a7-f6b89a918a95)

[mergify]

Queued — the merge queue status continues in this comment ↓.

[cursor]

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_a6c23fc1-7aa4-43b7-b8d9-91e940fc14bd)

[EdbertChan]

@Mergifyio queue

[mergify]

Merge Queue Status

• ✅ Entered queue — 2026-09-28 05:03 UTC · Rule: admin-bypass · triggered by @EdbertChan with the @mergifyio queue command
• ✅ Checks passed · on draft merge queue: checking #1206 on main (3e0b253) #1208
• ✅ Merged — 2026-09-28 05:35 UTC · at d26ab66b85fda485d807122439212ddd4e694e53 · squash

This pull request spent 32 minutes 32 seconds in the queue, including 32 minutes 14 seconds running CI.

Required conditions to merge

• check-success = lint
• check-success = test
• check-success = validate