Rebalance CodSpeed shards and build CPU benchmarks off metal

[robert3005]

Just like macro benchmarks build binaries for codspeed on smaller machines. We also rebalance benchmark targets across machines to have more uniform run duration across all of them.

We avoid building all benchmark targets every time by enumerating the actual binaries we are benchmarking

First-run timings

Simulation shard	Build	Benchmark step	Combined
Shard #1 - Array compute	34s	54s	88s
Shard #2 - Array selection	30s	70s	100s
Shard #3 - Array dictionaries	22s	53s	75s
Shard #4 - Array builders & views	41s	53s	94s
Shard #5 - Core & encodings	33s	42s	75s
Shard #6 - Main library & storage	189s	37s	226s
Shard #7 - FastLanes & decimals	117s	49s	166s
Shard #8 - Encodings & layout	46s	56s	102s
Shard #9 - Strings, tensor & spatial	189s	82s	271s

The four array shards each take 75–100s to build and run, versus 436s for the previous single array shard in the baseline run. The slowest simulation shard is now 271s, but build times still vary substantially: storage and strings each take 189s to compile. This first run demonstrates improvement, not an evenly balanced steady state; cache state and changed package feature combinations can affect the comparison.

CPU features	VM build	Metal measurement step	Entire metal job	Previous entire metal job
AVX2	25s	34s	65s	210s
AVX-512	32s	31s	62s	207s
NEON	34s	29s	73s	261s

Combined metal job duration fell from 678s to 200s (about 70%). VM artifact uploads took another 30–47s, with downloads on metal taking 6–8s. These are observed job durations, not billing measurements.

[codspeed]

Merging this PR will degrade performance by 19.88%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

❌ 6 regressed benchmarks
✅ 2238 untouched benchmarks
⏩ 176 skipped benchmarks¹
🗄️ 1 archived benchmark run²

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	WallTime	arrow_checked_add_u32_neon[16384]	12.3 µs	20.5 µs	-39.83%
❌	Simulation	random_i8[0.5]	70.6 µs	94.5 µs	-25.31%
❌	Simulation	decompress[u64, (4000, 1024)]	72 µs	86.7 µs	-16.97%
❌	WallTime	words_gather_scalar_avx2[65536]	8.3 µs	9.4 µs	-11.83%
❌	WallTime	mul_u32_nonnull_avx512	5.7 µs	6.4 µs	-10.64%
❌	WallTime	mul_i32_nonnull_avx512	7.2 µs	8 µs	-10.05%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing rk/codspeedbuild (f028bb0) with develop (313bc4f)}

Footnotes

1. 176 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. 1 benchmark was run, but is now archived. If it was deleted in another branch, consider rebasing to remove it from the report. Instead if it was added back, click here to restore it. ↩