Skip to content

test(dogfood): every test file runs in its own temporary working directory - #21919

Merged
objectstack-fleet[bot] merged 2 commits into
mainfrom
claude/issue-21914-dogfood-package-dir-state
Oct 6, 2026
Merged

objectstack-fleet[bot] merged 2 commits into
mainfrom
claude/issue-21914-dogfood-package-dir-state

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Fixes #21914
Clause-②: no

What changes

Every dogfood test file now runs in its own temporary working directory. The suite fails when any file leaves .objectstack/data in packages/qa/dogfood.

  • test/per-file-cwd.setup.ts (new) is a setupFiles entry, wired explicitly in BOTH projects of vitest.config.ts, because inline projects inherit nothing from the root block. shared-showcase keeps isolate: false; the module still runs once per file there.
    • At module top level, before the test file's own imports, it creates a directory under the run's temporary root and chdirs into it.
    • In afterAll it restores the previous cwd. That afterAll is also the guard: it THROWS when packages/qa/dogfood/.objectstack/data exists. The message names the directory, its entries and the remedy. It also says the named file may be a concurrent one on another worker rather than the writer, and whether the directory was already present when the file started.
  • test/per-file-cwd.global-setup.ts (new) is a root-level globalSetup. It runs once per run, covering both projects and each OS_TEST_SHARD slice (measured).
    • At the START it clears a stale packages/qa/dogfood/.objectstack, so a developer's earlier run never reds the suite.
    • It creates one temporary root for the run and hands it to the workers with provide / inject.
    • At the END it removes that root, which is where the per-file directories are removed. The removal is run-level, not per file, because the memoized shared-showcase boot keeps its SQLite handles open in the directory of the file that booted it.
    • The teardown judges nothing (see Evidence: a throwing teardown is a false green).
  • vitest.config.ts wires the two modules. A header section explains why there are two halves and why the guard is not in the teardown.
  • test/enterprise-organizations.ts: the module-level probeOrganizations() now passes this package's root as hostRoot, resolved from the module's location (new URL('..', import.meta.url)), not the cwd. This was measured to be needed; see Evidence.

No per-file edits. The five files the card names, and the other 87 measured writers, are covered by the module with no change of their own. Test isolation only: @objectstack/dogfood is private: true, so no published package moves and there is no changeset (skip-changeset).

The invariant for every dogfood author

  • Each test file runs in its own temporary cwd. Anything it writes relative to the cwd is its own, no other file sees it, and it is removed at the end of the run. A file needs no mkdtemp / chdir of its own.
  • A package-relative read must resolve from the module's location (new URL('..', import.meta.url), import.meta.dirname), never from process.cwd(). The cwd is a temporary directory.
  • A file that writes into packages/qa/dogfood/.objectstack/data fails the run. That happens through an absolute path built from the package root, or through a process.chdir() back to the package directory before a boot. The fix is to write relative to the file's own cwd.
  • Files that already chdir into a temp dir of their own still work, because they restore to the per-file directory. Their own chdir is now redundant and harmless.

Why (measured)

A per-file probe over the whole suite measured 92 test files leaving .objectstack/data/showcase_external.db in the package directory, not the five the card names:

  • 7 leave the populated federated fixture (24576 B, 2 tables): the card's five, plus showcase-demo-personas-loginable and showcase-demo-personas-membership, which pass onEnable in the bundle.
  • 85 leave an empty SQLite file (4096 B, 0 tables). The showcase's declared external datasource has a cwd-relative filename, and its auto-connect creates the file on every showcase boot, onEnable or not.

A later boot on the same runner found or missed the federated tables depending on which files ran before it, and that ordering is how PR #21905 went red only on dogfood shard 3/3. The seat chose this route (one module) and this guard (comment 6004950414 on #21914), on the dev's measurement (comment 6004909676).

Evidence

All runs are at head 967ce88a, under the shared verify lock, from a clean package directory.

  • Whole suite: pnpm --filter @objectstack/dogfood test gave Test Files 205 passed | 1 skipped (206) and Tests 1591 passed | 9 skipped (1600). Afterwards packages/qa/dogfood/.objectstack does not exist, and no os-dogfood-run-* root is left in the temp dir.

  • CI's three-shard split: CI's dogfood leg exports OS_TEST_SHARD=k/3 and vitest.config.ts turns it into vitest's shard. Here each shard ran as OS_TEST_SHARD=k/3 pnpm --filter @objectstack/dogfood test: the same vitest selection, without turbo, so no cached replay. Each exited 0 and left no .objectstack:

    shard Test Files Tests
    1/3 69 passed (69) 507 passed (507)
    2/3 69 passed (69) 461 passed, 1 skipped (462)
    3/3 67 passed, 1 skipped (68) 623 passed, 8 skipped (631)

    The three add up to the whole run: 206 files, 1600 tests.

  • Ablation (H4) through scripts/ablation-replace.mjs, wrap mode. The central process.chdir(...) was replaced by the bare mkdtempSync(...): anchor count 1 to 0, blob 0991eb9c to ee5a65be.

    • With the chdir dropped, showcase-external-autoconnect and showcase-search ran: Test Files 2 failed (2), Tests 8 passed (8), exit 1. Each failed in the guard: .../packages/qa/dogfood/.objectstack/data exists after this test file ran. Entries: showcase_external.db (plus -shm / -wal for the shared-showcase file).
    • Restore was proven by the tool: blob after restore equals HEAD (0991eb9c), and git diff HEAD is empty.
    • The same two files then gave 2 passed, exit 0, and left nothing.
    • No build step is involved: vitest loads the mutated module from source.
  • Stale directory: .objectstack/data/x.db was planted, then 9 files were run. Result: Test Files 9 passed (9), exit 0, nothing left (the globalSetup cleared it).

  • Census: those 9 files are the 7 populated-fixture writers plus showcase-search and showcase-permission-zoo, both shared-showcase files.

  • hostRoot line, measured both ways, running rls-multitenant, org-create-default-team and enterprise-organizations.test:

    • Without the line (commit 4d07dc29), the skip text read not resolvable from /tmp/os-dogfood-run-.../file-... and told the reader to declare the package in that temp directory's package.json.
    • With it (967ce88a), the text names packages/qa/dogfood/.
    • The verdict is the same both ways (skipped), because no framework package declares @objectstack/organizations.
  • Guard placement: a throwing globalSetup teardown was measured on vitest 4.1.11 to print error during close and still exit 0, a false green. So the guard is the per-file afterAll. (A teardown that sets process.exitCode = 1 does exit 1, but the summary still reads all-passed.)

  • Typecheck and lint: pnpm --filter @objectstack/dogfood typecheck is green, and tsc --listFiles includes both new modules and enterprise-organizations.ts. pnpm lint exits 0.

  • Gates: 130 commands at 967ce88a, the dispatch list plus pnpm check:dispatcher-error-vocabulary from dispatch-gates --commands. dispatch-gates --ran: 48 derived famil(ies) accounted for — 48 run, 0 NOT-MEASURED.

    • check:dual-build-cjs-loads and check:published-readme-exports first exited 3 (dist prerequisite: 7 packages unbuilt). After building those 7, both exit 0.
    • The three PR-context scripts (check-closing-target-claim, check-partof-closing-keyword, check-single-claim-paths) are re-run with this PR's context; the results are in the report on the card.

Open PRs that add dogfood files

PR new file boots the showcase own chdir under this PR
#21864 showcase-public-form-withdrawal-layers.dogfood.test.ts yes no Covered with no author action. Without this PR it would leave .objectstack/data in the package directory.
#21917 organization-delete-federated-fixture.dogfood.test.ts yes, with onEnable yes Unaffected; its own chdir is redundant.
#21906 external-import-code-datasource-namespace.dogfood.test.ts (also edits three external-* files) yes, with onEnable yes Unaffected. None of its files is edited here.
#21877 datasource-contractless-credentials.dogfood.test.ts yes yes Unaffected.
#21897 flow-node-config-values-at-registration.dogfood.test.ts no (fixture stack) no Runs in its own temp cwd; it reads nothing relative to the cwd.

None of these files reads a package-relative path through process.cwd(). Only their own prevCwd captures do.

Acceptance notes

  • Observation, not filed. The showcase's external datasource is declared read-only (schemaMode: 'external', allowWrites: false). Its auto-connect CREATES a missing .objectstack/data/showcase_external.db, plus -wal / -shm (measured on 85 harness boots).
    • The declaration's own comment in showcase-external.datasource.ts says that if the fixture file cannot be opened, "the boot stops with that as the reason rather than serving a showcase whose federation pages are quietly dead".
    • It was measured only through the verify harness's bootStack, never at a public door (os start / os dev), so it stays here.
  • Latent, unreachable today. bootStack(..., { multiTenant: true }) also defaults its hostRoot to the cwd: rls-multitenant.dogfood.test.ts:79, and attachments-permission-matrix.dogfood.test.ts:766 through bootFixture. Both are gated on organizationsAvailable, which is false in this repository because no framework package may declare @objectstack/organizations (ADR-0132). A run that declares it in this package would need those boots to pass the package root too. Carrier: whoever declares it.
  • The own chdir in external-validate-sees-runtime-save, external-import-destructive-remedy, PR fix(runtime,service-datasource): an import over a code-defined datasource is held to its package's ADR-0028 namespace #21906's file and PR fix(objectql): the cascade skips a federated object's injected tenant anchor #21917's file is now redundant. It is left untouched and can be removed once fix(runtime,service-datasource): an import over a code-defined datasource is held to its package's ADR-0028 namespace #21906 lands. Carrier: the domain:cli seat.
  • Attribution limit: under parallel workers, the guard can name a file that ran at the same time as the writer. The message says so, and says whether the directory was already present when the named file started.

Generated by Claude Code

claude added 2 commits October 5, 2026 23:04
…ctory

Showcase boots write .objectstack/data/showcase_external.db relative to the
cwd: the declared external datasource auto-connects (SQLite creates the
file) and onEnable provisions its federated tables there. From the package
directory 92 files left it behind, 7 of them populated, so a later boot's
federated state depended on shard composition and file order.

A setupFiles module, wired in both projects, chdirs each file into its own
directory under a run-level temporary root and restores the cwd in
afterAll, where it throws when packages/qa/dogfood/.objectstack/data
exists. A root globalSetup clears a stale .objectstack at the start and
removes the run root at the end; its teardown judges nothing.

Claude-Session: https://claude.ai/code/session_01RWZbGvPFcRKvUqASZtunCU
Co-authored-by: Claude <noreply@anthropic.com>
… root

Under the per-file temporary cwd, probeOrganizations() defaulted its host
root to that temp directory, so its skip text told the reader to declare
@objectstack/organizations in a /tmp directory's package.json (measured).
The module-level probe now passes this package's root, resolved from the
module's own location.

Claude-Session: https://claude.ai/code/session_01RWZbGvPFcRKvUqASZtunCU
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added the size/m label Oct 5, 2026
@objectstack-fleet objectstack-fleet Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Oct 5, 2026
@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

1 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run.

What this run could not see

Coarse fallback — 2 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json e6dc7a240617eaeef9a64e788bf6e5561c107f1b → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 76c0afd216a96d371d1e40d30d8a3b31dd5ddc40 — the merge of head 967ce88af337772d0a40b7d6dfaebfac1b901207 into base e6dc7a240617eaeef9a64e788bf6e5561c107f1b, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 76c0afd216a96d371d1e40d30d8a3b31dd5ddc40 && git checkout 76c0afd216a96d371d1e40d30d8a3b31dd5ddc40
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin e6dc7a240617eaeef9a64e788bf6e5561c107f1b 967ce88af337772d0a40b7d6dfaebfac1b901207 && git checkout -B drift-repro e6dc7a240617eaeef9a64e788bf6e5561c107f1b && git merge --no-ff 967ce88af337772d0a40b7d6dfaebfac1b901207

node scripts/docs-audit/affected-docs.mjs --json e6dc7a240617eaeef9a64e788bf6e5561c107f1b

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

@github-actions github-actions Bot added the tests label Oct 5, 2026
@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review October 5, 2026 23:55
@objectstack-fleet
objectstack-fleet Bot enabled auto-merge October 5, 2026 23:55
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Oct 5, 2026
Merged via the queue into main with commit be97cf3 Oct 6, 2026
41 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-21914-dogfood-package-dir-state branch October 6, 2026 00:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

2 participants