Skip to content

[finding] check:test-source-alias's clocked-window rule cannot see a load routed through a helper — the defect it documents sat green in packages/cli for the whole of #17180 #17658

Description

@os-sales

The clocked-window rule in scripts/check-test-source-alias.mjs documents a hazard, prescribes a remedy, and names the right population — and then cannot see an instance of that hazard when the dynamic load reaches import() through a helper rather than as a literal specifier. Found while fixing #17180, which was exactly such an instance.

Two-leg measurement (worktree at origin/main merge base 155b875d0, branch claude/issue-17180-optional-package-probe-transform)

Same file, same line, same defect — only the SPELLING of the load differs. Each leg was written to disk, the landing proved by occurrence count and git hash-object, and the file restored to its HEAD blob afterwards (git diff HEAD empty).

leg how the load is spelled at optional-package.test.ts:112 pnpm check:test-source-alias
B loadOptionalPackage('@objectstack/cloud-connection') — the real pre-fix code exit 0, and the gate prints nothing about the file
C an added literal await import('@objectstack/cloud-connection'); on the line above, nothing else changed exit 1

Leg C's diagnostic, verbatim:

✗ packages/cli/src/utils/optional-package.test.ts:112: `import('@objectstack/cloud-connection')` is paid inside a function body — a CLOCKED window.
    THE CONVENTION: clocked windows measure behaviour, never loading — a test that boots a real
    Add a module-top side-effect import so the transform is paid during COLLECTION, which vitest

So this is not "the gate does not cover this shape". It covers it precisely: @objectstack/cloud-connection is already in KNOWN_UNALIASED_TEST_IMPORTS['@objectstack/cli'], and the remedy the gate printed in leg C is the exact one-line fix that took the probe from 5005ms to 5.7ms. The call simply escapes the predicate.

Where the predicate loses it

moduleLoadSites() classifies loads by running IMPORT_PATTERNS over a comment-masked projection of the source text, and keeps match[2] ?? match[3] ?? match[4] ?? match[5] as the specifier. That is a text scan for a literal specifier sitting inside import(...) or require(...). In the #17180 file the specifier is an ordinary string argument:

const load = await loadOptionalPackage('@objectstack/cloud-connection');

and the only import() in the chain is one frame down in optional-package.ts, where it imports a variable:

const module = await import(specifier);

Neither site offers a literal specifier inside an import form, so neither becomes a clocked entry, and the file has no moduleScope load either — the two conditions the diagnostic requires. The result is silence, with CI green, over a probe that was blowing its 5000ms budget on a built worktree.

Why it is worth a card

The blind spot is not exotic. Any test that loads an optional or conditional dependency through a wrapper has this shape, and a wrapper is the normal way this repo loads one (loadOptionalPackage exists precisely so os doctor can tell "absent" from "installed and broken"). The rule's cost model is unchanged by the indirection — the transform is identical — so the population it means to protect is strictly larger than the population it can currently read.

Precedent for the class, both closed: #10452 (check-cross-package-test-inputs' literal collector cannot see an escaping relative import specifier) and #8020 (this same gate could not read a template-literal alias replacement). Same failure mode each time — a text-level collector whose silence is indistinguishable from a pass.

Deliberately NOT proposed here

⛔ No widening in #17180's PR — that PR is a one-line test fix and the gate is out of its file surface. Whoever takes this one should also decide the hard part, which is not the detection but the scope: teaching the rule to follow one hop into a same-package helper is a different kind of reader than a masked-text scan, and the rule's own header already refuses to widen its population casually ("widening the population is a different card", near :266). A cheaper variant worth pricing first: flag a test file that names a registered specifier as a string literal anywhere while having no module-scope load of it.

Two files in packages/cli already reason about this mechanism and are worth reading before touching the predicate: test/vitest-resolution-base-collapse.e2e.test.ts and vitest.config.ts.

Refs: #17180 (the instance, fixed by the gate's own prescribed remedy) · #10126 (the card that added the clocked-window rule) · #16497 (sibling instance of the same cost, still open) · #10452 · #8020

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions