Skip to content

ci: every Test Core run publishes the slowest test files and packages beside their pinned weights (maintainer-directed, part B measurement) #16454

Description

@claude

Filed by the skills lane seat (session session_019RfFHiRCSs3JXLK4cwcfox, os-steve) on the maintainer's direction, 2026-09-07T02:1xZ, and dispatched by the same seat on that direction. Surface owner stays domain:devx.

Authority (maintainer, verbatim, live PM chat 2026-09-07): 「我想的是测试会不会太多,是否都是必要的,是不是应该砍,每次修改都要完整的测试吗」 → 「同意,这几个都处理,你现在就负责派发」. This card is the measurement half of part B: cutting is decided per test file by cost, so the cost has to be visible first.

What

Every Test Core run publishes the slowest test files and packages, so the maintainer and the seats read a list instead of guessing which tests to cut.

Measured

  • 3,388 test files against 2,183 source files (1.55 : 1); 48,230 it() cases; 35–89 new test files a day over the last fourteen days.
  • The expensive kinds are concentrated: packages/cli has 60 *.e2e.test.ts files and its own vitest.config.ts records that the e2e tier is ~75% of the package's test time (781 s of ~1,040 s); runtime carries 41 integration files, drivers 36 conformance, services 14 conformance + 8 live.
  • Nothing today reports per-file durations; the turbo run summaries per shard give per-package durations only.

Ruling

  1. Each shard job already streams vitest output (--log-order=stream). Capture the per-file result lines vitest prints (file, test count, milliseconds) into a shard artifact; the Test Core aggregator merges the six, prints the slowest 20 files and the slowest 10 packages to the job summary, and publishes the merged table as an artifact with 14-day retention.
  2. The same table carries each package's measured seconds beside its pinned weight from scripts/test-shard-timings.json, so the drift CI: the shard-timings file is stale for the CLI package — 672s predicted vs 28m46s measured against a 30-minute timeout, so Test Core shard 1/6 is one slow run from being killed on any PR touching the CLI #16173 describes is readable on every run. Report only — no red here; the alarm is CI: the shard-timings file is stale for the CLI package — 672s predicted vs 28m46s measured against a 30-minute timeout, so Test Core shard 1/6 is one slow run from being killed on any PR touching the CLI #16173's remedy.
  3. Pure tooling: ci.yml steps plus one script under scripts/ with a self-test. No test is changed or removed by this card.

Acceptance

  • The parser has a self-test over real vitest output lines (green, failed, skipped, the summary line) and refuses an empty capture as "not measured" rather than an empty table.
  • The PR's own CI run shows the table in the job summary; the seat reads it.

Refs #16173.


Generated by Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions