Skip to content

Add job size benchmark script + on-demand CI workflow - #8495

Open
TomasVotruba wants to merge 2 commits into
mainfrom
tv-jobsize
Open

TomasVotruba wants to merge 2 commits into
mainfrom
tv-jobsize

Conversation

@TomasVotruba

@TomasVotruba TomasVotruba commented Sep 14, 2026

Copy link
Copy Markdown
Member

Adds a benchmark that measures Rector wall time across parallel job sizes, plus a manual CI workflow to run it.

Script

scripts/rector-jobsize-benchmark.php runs cold Rector dry-runs across a list of job sizes and prints a markdown table of wall time, slowdown vs fastest, exit code, and file counts. It is a standalone dev tool and is excluded from PHPStan.

CI workflow

.github/workflows/jobsize_benchmark.yaml runs on workflow_dispatch only (no per-PR cost). Sizes default to 50-200 by 25 and are overridable from the run dialog. Results go to the job summary and an uploaded artifact.

Measurements

Both runs analysed the same 2620 files, every run cold, one sample per size.

Remote (GitHub runner, 4 cores)

jobSize wall vs fastest
16 48.8 s 1.40x
50 39.3 s 1.13x
100 34.9 s 1.00x
150 35.6 s 1.02x
300 36.2 s 1.04x

Local (24 cores)

jobSize wall vs fastest
50 49.1 s 1.16x
100 44.7 s 1.06x
130 42.2 s 1.00x
200 47.9 s 1.13x
300 64.2 s 1.52x
400 71.5 s 1.69x

Side by side, shared sizes

jobSize local (24c) remote (4c)
50 49.1 s 39.3 s
100 44.7 s 34.9 s
150 49.8 s 35.6 s
300 64.2 s 36.2 s

Notes:

  • On 24 cores, wall time climbs past job size ~250 (300 is 1.5x the fastest, 400 is 1.7x) as large chunks starve cores and the last worker drags the tail. The sweet spot sits around 60-130.
  • On 4 cores the curve is flat from 100 upward, because with few workers the chunk size matters much less.
  • The local box was under concurrent load during the sweep, so its absolute numbers are noisier and higher than the runner here; the shape of the curve is the point, not the absolute seconds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant