Skip to content

Publish a reproducible, generated benchmark report - #377

Merged
sgerbino merged 3 commits into
developfrom
pr/benchmark-report
Oct 6, 2026
Merged

Publish a reproducible, generated benchmark report#377
sgerbino merged 3 commits into
developfrom
pr/benchmark-report

Conversation

@sgerbino

@sgerbino sgerbino commented Oct 2, 2026

Copy link
Copy Markdown
Collaborator

Closes #352.

The published benchmark page is now fully generated from fresh runs on the three dedicated machines, replacing the hand-written February report.

What's here

  • .github/workflows/benchmark-report.yml — dispatch-only workflow that runs the full comparison suite (corosio / asio coroutines / asio callbacks) on the Linux, Windows, and macOS runners and uploads raw per-iteration JSON plus an environment capture. Linux builds asio twice (epoll and io_uring reactors) so every corosio configuration is compared against a same-reactor asio.
  • .github/bench/report_page.py — stdlib-only generator: median-across-iterations aggregation with CV-based noise classification, theme-aware inline SVG charts (diverging bars vs the asio-coroutines baseline), four Antora pages (methodology landing + per-platform Summary / Test Environment / Results).
  • bench/README.md — how to run the suite, dispatch the workflow, and regenerate the pages.
  • Scenario descriptions in the harness (describe() / category_description()) rendered as category intros and chart hover text.
  • Generated pages + charts from run 36946358854 (7 iterations × 2 s, commit 90bf46a): Linux epoll median +1.3%, io_uring +2.1%, Windows IOCP +1.4%, macOS kqueue +60.3% vs Boost.Asio coroutines. The old benchmark-report.adoc URL stays alive via a page alias.

Bench fixes surfaced by production runs

  • The asio local-socket suites registered on Windows but local::connect_pair relies on socketpair(), which asio stubs out as operation_not_supported there — the thrown error aborted every full-suite run. They are now POSIX-only, matching the corosio suites.
  • Churn benchmarks RST-closed only the client side; on loopback a FIN racing the RST could still park connections in TIME_WAIT, which at production duration overflowed the macOS PCB table mid-benchmark. Both sides now RST-close, in all three implementations.

Reproducing

Dispatch benchmark-report from the Actions tab (defaults: 7 × 2 s), gh run download the artifacts, run report_page.py, review the Antora build, open a PR — see bench/README.md.

@github-project-automation github-project-automation Bot moved this to Backlog in Beast2 Oct 2, 2026
@sgerbino
sgerbino force-pushed the pr/benchmark-report branch from 5947101 to 5e36210 Compare October 2, 2026 03:31
@cppalliance-bot

cppalliance-bot commented Oct 2, 2026

Copy link
Copy Markdown

GCOVR code coverage report https://377.corosio.prtest3.cppalliance.org/gcovr/index.html
LCOV code coverage report https://377.corosio.prtest3.cppalliance.org/genhtml/index.html
Coverage Diff Report https://377.corosio.prtest3.cppalliance.org/diff-report/index.html

Build time: 2026-10-02 03:41:39 UTC

@cppalliance-bot

cppalliance-bot commented Oct 2, 2026 •
edited
Loading

Copy link
Copy Markdown

GCOVR code coverage report https://377.corosio.prtest3.cppalliance.org/gcovr/index.html
LCOV code coverage report https://377.corosio.prtest3.cppalliance.org/genhtml/index.html
Coverage Diff Report https://377.corosio.prtest3.cppalliance.org/diff-report/index.html

Build time: 2026-10-02 22:17:12 UTC

@github-actions

github-actions Bot commented Oct 2, 2026 •
edited
Loading

Copy link
Copy Markdown

Benchmark report

a80ce002fd8c (base) vs 99008d23794f (head)

  • ✅ linux — clean (epoll, uring)
  • ⚠️ windows — 2 flagged (iocp)
  • ✅ macos — clean (kqueue)

⚠️ Flagged

Platform Backend Benchmark Metric Base Head Δ Noise
windows iocp accept_churn/burst/100 ops_per_sec 152.2 ops/s 149.1 ops/s 🔴▼ -2.52% 0.60%
windows iocp accept_churn/burst_lockless/100 ops_per_sec 156.7 ops/s 151.9 ops/s 🔴▼ -3.02% 0.71%
linux — full results (226 benchmarks, 5 iterations, 1.5s each)

accept_churn

Benchmark Backend Metric Base Head Δ Noise
burst/10 epoll ops_per_sec 1.86K ops/s 1.88K ops/s 🟢▲ +0.78% 0.91%
burst/10 uring ops_per_sec 1.72K ops/s 1.70K ops/s 🟢▲ +0.08% 1.25%
burst/100 epoll ops_per_sec 187.7 ops/s 189.4 ops/s 🟢▲ +0.35% 0.99%
burst/100 uring ops_per_sec 174.5 ops/s 173.5 ops/s 🔴▼ -0.09% 0.76%
burst_lockless/10 epoll ops_per_sec 1.88K ops/s 1.89K ops/s 🟢▲ +0.25% 0.95%
burst_lockless/10 uring ops_per_sec 1.83K ops/s 1.82K ops/s 🔴▼ -0.26% 0.75%
burst_lockless/100 epoll ops_per_sec 189.1 ops/s 190.4 ops/s 🟢▲ +0.26% 0.88%
burst_lockless/100 uring ops_per_sec 187.0 ops/s 185.6 ops/s 🔴▼ -0.39% 0.77%
concurrent/1 epoll ops_per_sec 14.86K ops/s 14.89K ops/s 🟢▲ +0.21% 0.50%
concurrent/1 uring ops_per_sec 12.70K ops/s 12.68K ops/s 🟢▲ +0.13% 1.08%
concurrent/16 epoll ops_per_sec 14.85K ops/s 14.97K ops/s 🟢▲ +0.38% 0.92%
concurrent/16 uring ops_per_sec 12.54K ops/s 12.56K ops/s 🟢▲ +0.37% 1.30%
concurrent/4 epoll ops_per_sec 14.87K ops/s 14.90K ops/s 🟢▲ +0.36% 0.45%
concurrent/4 uring ops_per_sec 12.66K ops/s 12.63K ops/s 🟢▲ +0.36% 1.08%
sequential epoll ops_per_sec 14.83K ops/s 14.84K ops/s 🟢▲ +0.07% 0.53%
sequential uring ops_per_sec 12.76K ops/s 12.70K ops/s 🔴▼ -0.40% 1.03%
sequential_lockless epoll ops_per_sec 14.96K ops/s 14.96K ops/s 🔴▼ -0.14% 0.55%
sequential_lockless uring ops_per_sec 13.68K ops/s 13.62K ops/s 🔴▼ -1.24% 1.03%

fan_out

Benchmark Backend Metric Base Head Δ Noise
concurrent_parents/1 epoll ops_per_sec 3.68K ops/s 3.70K ops/s 🔴▼ -0.01% 0.99%
concurrent_parents/1 uring ops_per_sec 3.45K ops/s 3.44K ops/s 🔴▼ -0.02% 1.20%
concurrent_parents/16 epoll ops_per_sec 3.57K ops/s 3.59K ops/s 🟢▲ +0.20% 0.98%
concurrent_parents/16 uring ops_per_sec 3.32K ops/s 3.30K ops/s 🔴▼ -0.24% 0.47%
concurrent_parents/4 epoll ops_per_sec 3.59K ops/s 3.60K ops/s 🟢▲ +0.11% 0.97%
concurrent_parents/4 uring ops_per_sec 3.36K ops/s 3.33K ops/s 🔴▼ -0.03% 1.18%
concurrent_parents_lockless/1 epoll ops_per_sec 3.72K ops/s 3.73K ops/s 🟢▲ +0.28% 0.23%
concurrent_parents_lockless/1 uring ops_per_sec 3.66K ops/s 3.65K ops/s 🔴▼ -0.21% 0.34%
concurrent_parents_lockless/16 epoll ops_per_sec 3.61K ops/s 3.61K ops/s 🔴▼ -0.06% 0.14%
concurrent_parents_lockless/16 uring ops_per_sec 3.53K ops/s 3.51K ops/s 🔴▼ -0.41% 0.37%
concurrent_parents_lockless/4 epoll ops_per_sec 3.62K ops/s 3.62K ops/s 🔴▼ -0.03% 0.15%
concurrent_parents_lockless/4 uring ops_per_sec 3.55K ops/s 3.55K ops/s 🔴▼ -0.42% 0.43%
fork_join/1 epoll ops_per_sec 54.57K ops/s 54.93K ops/s 🟢▲ +0.20% 0.92%
fork_join/1 uring ops_per_sec 53.22K ops/s 53.18K ops/s 🔴▼ -0.26% 0.33%
fork_join/16 epoll ops_per_sec 3.69K ops/s 3.68K ops/s 🟢▲ +0.01% 0.29%
fork_join/16 uring ops_per_sec 3.43K ops/s 3.43K ops/s 🔴▼ -0.23% 0.27%
fork_join/4 epoll ops_per_sec 14.62K ops/s 14.66K ops/s 🟢▲ +0.14% 0.74%
fork_join/4 uring ops_per_sec 13.73K ops/s 13.72K ops/s 🔴▼ -0.17% 0.33%
fork_join/64 epoll ops_per_sec 897.2 ops/s 896.8 ops/s 🟢▲ +0.04% 0.21%
fork_join/64 uring ops_per_sec 812.8 ops/s 812.5 ops/s 🔴▼ -0.12% 0.17%
fork_join_lockless/1 epoll ops_per_sec 55.32K ops/s 55.54K ops/s 🟢▲ +0.42% 0.30%
fork_join_lockless/1 uring ops_per_sec 56.00K ops/s 55.69K ops/s 🔴▼ -0.61% 0.24%
fork_join_lockless/16 epoll ops_per_sec 3.72K ops/s 3.73K ops/s 🟢▲ +0.09% 0.18%
fork_join_lockless/16 uring ops_per_sec 3.67K ops/s 3.66K ops/s 🔴▼ -0.43% 1.04%
fork_join_lockless/4 epoll ops_per_sec 14.78K ops/s 14.81K ops/s 🟢▲ +0.19% 0.16%
fork_join_lockless/4 uring ops_per_sec 14.64K ops/s 14.52K ops/s 🔴▼ -0.53% 1.08%
fork_join_lockless/64 epoll ops_per_sec 907.5 ops/s 907.0 ops/s 🔴▼ -0.25% 0.22%
fork_join_lockless/64 uring ops_per_sec 870.3 ops/s 867.6 ops/s 🔴▼ -0.05% 1.04%
nested/16 epoll ops_per_sec 891.0 ops/s 895.0 ops/s 🟢▲ +0.17% 1.05%
nested/16 uring ops_per_sec 809.4 ops/s 805.0 ops/s 🟢▲ +0.01% 1.38%
nested/4 epoll ops_per_sec 3.66K ops/s 3.67K ops/s 🟢▲ +0.29% 0.57%
nested/4 uring ops_per_sec 3.43K ops/s 3.40K ops/s 🔴▼ -0.25% 1.47%
nested_lockless/16 epoll ops_per_sec 900.4 ops/s 904.4 ops/s 🟢▲ +0.11% 0.93%
nested_lockless/16 uring ops_per_sec 860.7 ops/s 856.4 ops/s 🔴▼ -0.24% 1.19%
nested_lockless/4 epoll ops_per_sec 3.70K ops/s 3.72K ops/s 🟢▲ +0.20% 1.07%
nested_lockless/4 uring ops_per_sec 3.65K ops/s 3.62K ops/s 🔴▼ -0.32% 1.22%

http_server

Benchmark Backend Metric Base Head Δ Noise
concurrent/1 epoll ops_per_sec 55.10K ops/s 55.11K ops/s 🔴▼ -0.05% 0.45%
concurrent/1 uring ops_per_sec 53.62K ops/s 53.73K ops/s 🟢▲ +0.29% 0.27%
concurrent/16 epoll ops_per_sec 57.75K ops/s 57.81K ops/s 🟢▲ +0.01% 0.25%
concurrent/16 uring ops_per_sec 53.22K ops/s 53.04K ops/s 🔴▼ -0.07% 0.98%
concurrent/32 epoll ops_per_sec 57.33K ops/s 57.44K ops/s 🟢▲ +0.25% 0.20%
concurrent/32 uring ops_per_sec 53.14K ops/s 52.60K ops/s 🔴▼ -0.33% 1.37%
concurrent/4 epoll ops_per_sec 57.16K ops/s 57.21K ops/s 🟢▲ +0.08% 0.16%
concurrent/4 uring ops_per_sec 53.11K ops/s 53.21K ops/s 🟢▲ +0.20% 0.24%
multithread/1 epoll ops_per_sec 57.23K ops/s 57.40K ops/s 🟢▲ +0.41% 0.26%
multithread/1 uring ops_per_sec 53.10K ops/s 52.68K ops/s 🔴▼ -0.28% 1.31%
multithread/16 epoll ops_per_sec 161.6K ops/s 162.1K ops/s 🟢▲ +0.05% 0.61%
multithread/16 uring ops_per_sec 74.91K ops/s 75.17K ops/s 🟢▲ +0.54% 19.53%
multithread/2 epoll ops_per_sec 86.61K ops/s 87.19K ops/s 🟢▲ +0.84% 0.32%
multithread/2 uring ops_per_sec 90.67K ops/s 90.44K ops/s 🔴▼ -0.12% 1.02%
multithread/4 epoll ops_per_sec 157.3K ops/s 158.7K ops/s 🔴▼ -0.47% 2.29%
multithread/4 uring ops_per_sec 79.40K ops/s 79.11K ops/s 🔴▼ -0.69% 1.14%
multithread/8 epoll ops_per_sec 161.0K ops/s 161.2K ops/s 🔴▼ -0.17% 0.54%
multithread/8 uring ops_per_sec 77.16K ops/s 77.23K ops/s 🟢▲ +0.01% 0.19%
single_conn epoll ops_per_sec 55.20K ops/s 54.99K ops/s 🔴▼ -0.30% 1.05%
single_conn uring ops_per_sec 53.56K ops/s 53.70K ops/s 🟢▲ +0.29% 0.13%
single_conn_lockless epoll ops_per_sec 55.97K ops/s 55.71K ops/s 🔴▼ -0.22% 1.03%
single_conn_lockless uring ops_per_sec 56.18K ops/s 56.35K ops/s 🟢▲ +0.32% 0.24%

local_socket_latency

Benchmark Backend Metric Base Head Δ Noise
concurrent/1 epoll latency_mean_ns 7.12 µs 7.25 µs 🟢▲ +0.17% 4.28%
concurrent/1 uring latency_mean_ns 6.86 µs 6.73 µs 🟢▲ +0.29% 5.00%
concurrent/16 epoll latency_mean_ns 110.10 µs 110.03 µs 🔴▼ -0.22% 1.02%
concurrent/16 uring latency_mean_ns 108.19 µs 108.68 µs 🔴▼ -0.01% 0.93%
concurrent/4 epoll latency_mean_ns 27.87 µs 27.62 µs 🟢▲ +0.06% 1.91%
concurrent/4 uring latency_mean_ns 27.11 µs 27.24 µs 🔴▼ -0.78% 0.89%
concurrent_lockless/1 epoll latency_mean_ns 7.14 µs 7.12 µs 🔴▼ -0.14% 4.57%
concurrent_lockless/1 uring latency_mean_ns 6.56 µs 6.83 µs 🟢▲ +0.10% 5.43%
concurrent_lockless/16 epoll latency_mean_ns 108.51 µs 108.23 µs 🔴▼ -0.06% 0.95%
concurrent_lockless/16 uring latency_mean_ns 108.30 µs 107.98 µs 🟢▲ +0.58% 0.82%
concurrent_lockless/4 epoll latency_mean_ns 27.45 µs 27.24 µs 🟢▲ +1.88% 1.28%
concurrent_lockless/4 uring latency_mean_ns 27.01 µs 26.93 µs 🟢▲ +0.56% 1.95%
pingpong/1 epoll latency_mean_ns 7.06 µs 7.18 µs 🟢▲ +0.34% 4.37%
pingpong/1 uring latency_mean_ns 6.65 µs 6.52 µs 🟢▲ +0.84% 2.62%
pingpong/1024 epoll latency_mean_ns 7.90 µs 8.00 µs 🟢▲ +0.43% 3.99%
pingpong/1024 uring latency_mean_ns 7.38 µs 7.20 µs 🟢▲ +1.01% 4.94%
pingpong/64 epoll latency_mean_ns 7.27 µs 7.37 µs 🟢▲ +0.47% 5.13%
pingpong/64 uring latency_mean_ns 6.89 µs 6.58 µs 🟢▲ +1.10% 5.15%
pingpong_lockless/1 epoll latency_mean_ns 6.96 µs 6.95 µs 🔴▼ -0.09% 0.36%
pingpong_lockless/1 uring latency_mean_ns 6.67 µs 6.52 µs 🟢▲ +0.49% 3.73%
pingpong_lockless/1024 epoll latency_mean_ns 7.40 µs 7.46 µs 🟢▲ +0.13% 4.40%
pingpong_lockless/1024 uring latency_mean_ns 7.21 µs 7.21 µs 🔴▼ -0.11% 4.90%
pingpong_lockless/64 epoll latency_mean_ns 7.01 µs 7.01 µs 🟢▲ +0.03% 0.35%
pingpong_lockless/64 uring latency_mean_ns 6.84 µs 6.70 µs 🟢▲ +0.58% 4.88%

local_socket_throughput

Benchmark Backend Metric Base Head Δ Noise
bidirectional/1024 epoll bytes_per_sec 305.4 MB/s 304.9 MB/s 🟢▲ +0.32% 1.26%
bidirectional/1024 uring bytes_per_sec 302.4 MB/s 303.2 MB/s 🟢▲ +0.14% 0.62%
bidirectional/1048576 epoll bytes_per_sec 3.17 GB/s 3.18 GB/s 🔴▼ -0.25% 2.31%
bidirectional/1048576 uring bytes_per_sec 3.28 GB/s 3.27 GB/s 🟢▲ +1.77% 3.82%
bidirectional/16384 epoll bytes_per_sec 3.08 GB/s 3.01 GB/s 🟢▲ +0.48% 4.88%
bidirectional/16384 uring bytes_per_sec 2.42 GB/s 2.41 GB/s 🔴▼ -0.22% 0.68%
bidirectional/262144 epoll bytes_per_sec 3.16 GB/s 3.13 GB/s 🟢▲ +0.26% 2.91%
bidirectional/262144 uring bytes_per_sec 3.30 GB/s 3.28 GB/s 🔴▼ -1.54% 1.98%
bidirectional/4096 epoll bytes_per_sec 995.4 MB/s 999.7 MB/s 🟢▲ +0.20% 0.61%
bidirectional/4096 uring bytes_per_sec 996.2 MB/s 986.2 MB/s 🔴▼ -0.36% 1.26%
bidirectional/65536 epoll bytes_per_sec 2.95 GB/s 2.94 GB/s 🟢▲ +0.33% 1.86%
bidirectional/65536 uring bytes_per_sec 3.02 GB/s 3.01 GB/s 🔴▼ -0.63% 1.25%
bidirectional_lockless/1024 epoll bytes_per_sec 307.3 MB/s 310.2 MB/s 🟢▲ +0.54% 1.21%
bidirectional_lockless/1024 uring bytes_per_sec 303.9 MB/s 304.4 MB/s 🔴▼ -0.44% 1.29%
bidirectional_lockless/1048576 epoll bytes_per_sec 3.17 GB/s 3.20 GB/s 🔴▼ -0.15% 2.52%
bidirectional_lockless/1048576 uring bytes_per_sec 3.33 GB/s 3.32 GB/s 🔴▼ -1.96% 4.43%
bidirectional_lockless/16384 epoll bytes_per_sec 3.12 GB/s 3.07 GB/s 🟢▲ +1.20% 5.06%
bidirectional_lockless/16384 uring bytes_per_sec 2.64 GB/s 2.64 GB/s 🟢▲ +0.91% 1.97%
bidirectional_lockless/262144 epoll bytes_per_sec 3.16 GB/s 3.14 GB/s 🟢▲ +0.93% 3.52%
bidirectional_lockless/262144 uring bytes_per_sec 3.38 GB/s 3.35 GB/s 🔴▼ -1.66% 2.33%
bidirectional_lockless/4096 epoll bytes_per_sec 1.01 GB/s 1.01 GB/s 🔴▼ -0.34% 1.42%
bidirectional_lockless/4096 uring bytes_per_sec 998.4 MB/s 993.6 MB/s 🔴▼ -1.36% 1.45%
bidirectional_lockless/65536 epoll bytes_per_sec 2.97 GB/s 2.96 GB/s 🟢▲ +0.74% 2.21%
bidirectional_lockless/65536 uring bytes_per_sec 3.19 GB/s 3.22 GB/s 🔴▼ -0.81% 2.22%
unidirectional/1024 epoll bytes_per_sec 296.3 MB/s 298.2 MB/s 🟢▲ +0.39% 1.13%
unidirectional/1024 uring bytes_per_sec 302.3 MB/s 301.8 MB/s 🟢▲ +0.09% 0.64%
unidirectional/1048576 epoll bytes_per_sec 3.27 GB/s 3.28 GB/s 🔴▼ -0.13% 3.20%
unidirectional/1048576 uring bytes_per_sec 3.40 GB/s 3.36 GB/s 🔴▼ -0.91% 4.60%
unidirectional/16384 epoll bytes_per_sec 3.08 GB/s 3.05 GB/s 🟢▲ +0.63% 3.79%
unidirectional/16384 uring bytes_per_sec 2.66 GB/s 2.65 GB/s 🔴▼ -0.21% 1.47%
unidirectional/262144 epoll bytes_per_sec 3.23 GB/s 3.28 GB/s 🟢▲ +2.46% 2.18%
unidirectional/262144 uring bytes_per_sec 3.44 GB/s 3.44 GB/s 🔴▼ -1.08% 2.44%
unidirectional/4096 epoll bytes_per_sec 976.1 MB/s 980.8 MB/s 🟢▲ +0.68% 0.53%
unidirectional/4096 uring bytes_per_sec 997.5 MB/s 994.1 MB/s 🔴▼ -0.34% 0.73%
unidirectional/65536 epoll bytes_per_sec 3.22 GB/s 3.23 GB/s 🔴▼ -0.93% 3.63%
unidirectional/65536 uring bytes_per_sec 3.29 GB/s 3.33 GB/s 🟢▲ +1.93% 2.30%
unidirectional_lockless/1024 epoll bytes_per_sec 298.3 MB/s 303.1 MB/s 🟢▲ +1.28% 1.45%
unidirectional_lockless/1024 uring bytes_per_sec 305.4 MB/s 304.7 MB/s 🔴▼ -0.32% 1.29%
unidirectional_lockless/1048576 epoll bytes_per_sec 3.33 GB/s 3.31 GB/s 🔴▼ -1.16% 3.39%
unidirectional_lockless/1048576 uring bytes_per_sec 3.35 GB/s 3.33 GB/s 🟢▲ +0.42% 4.35%
unidirectional_lockless/16384 epoll bytes_per_sec 3.09 GB/s 3.07 GB/s 🟢▲ +2.01% 3.33%
unidirectional_lockless/16384 uring bytes_per_sec 2.68 GB/s 2.64 GB/s 🔴▼ -0.21% 2.34%
unidirectional_lockless/262144 epoll bytes_per_sec 3.28 GB/s 3.30 GB/s 🟢▲ +1.15% 1.92%
unidirectional_lockless/262144 uring bytes_per_sec 3.41 GB/s 3.38 GB/s 🔴▼ -1.27% 2.76%
unidirectional_lockless/4096 epoll bytes_per_sec 989.2 MB/s 993.7 MB/s 🟢▲ +0.52% 0.77%
unidirectional_lockless/4096 uring bytes_per_sec 994.2 MB/s 996.1 MB/s 🟢▲ +0.30% 1.77%
unidirectional_lockless/65536 epoll bytes_per_sec 3.26 GB/s 3.28 GB/s 🔴▼ -1.40% 3.45%
unidirectional_lockless/65536 uring bytes_per_sec 3.28 GB/s 3.30 GB/s 🔴▼ -0.05% 2.13%

socket_latency

Benchmark Backend Metric Base Head Δ Noise
concurrent/1 epoll latency_mean_ns 14.81 µs 14.79 µs 🟢▲ +0.05% 0.22%
concurrent/1 uring latency_mean_ns 14.46 µs 14.54 µs 🔴▼ -0.07% 1.17%
concurrent/16 epoll latency_mean_ns 232.67 µs 233.55 µs 🟢▲ +0.07% 1.19%
concurrent/16 uring latency_mean_ns 234.36 µs 235.20 µs 🔴▼ -0.03% 1.08%
concurrent/4 epoll latency_mean_ns 58.24 µs 58.26 µs 🔴▼ -0.06% 0.17%
concurrent/4 uring latency_mean_ns 58.15 µs 58.35 µs 🟢▲ +0.04% 1.05%
concurrent_lockless/1 epoll latency_mean_ns 14.62 µs 14.70 µs 🔴▼ -0.38% 1.18%
concurrent_lockless/1 uring latency_mean_ns 14.41 µs 14.52 µs 🔴▼ -0.32% 1.19%
concurrent_lockless/16 epoll latency_mean_ns 230.89 µs 232.47 µs 🔴▼ -0.40% 1.23%
concurrent_lockless/16 uring latency_mean_ns 234.90 µs 234.95 µs 🟢▲ +0.05% 0.17%
concurrent_lockless/4 epoll latency_mean_ns 57.65 µs 58.02 µs 🔴▼ -0.05% 1.23%
concurrent_lockless/4 uring latency_mean_ns 58.23 µs 58.40 µs 🔴▼ -0.20% 0.36%
pingpong/1 epoll latency_mean_ns 14.65 µs 14.75 µs 🔴▼ -0.23% 1.10%
pingpong/1 uring latency_mean_ns 14.41 µs 14.32 µs 🟢▲ +0.32% 1.24%
pingpong/1024 epoll latency_mean_ns 15.06 µs 15.11 µs 🔴▼ -0.41% 0.27%
pingpong/1024 uring latency_mean_ns 14.75 µs 14.73 µs 🔴▼ -0.21% 1.03%
pingpong/64 epoll latency_mean_ns 14.77 µs 14.84 µs 🔴▼ -0.54% 0.24%
pingpong/64 uring latency_mean_ns 14.47 µs 14.42 µs ⚪ 0.00% 1.10%
pingpong_lockless/1 epoll latency_mean_ns 14.61 µs 14.62 µs 🟢▲ +0.02% 0.23%
pingpong_lockless/1 uring latency_mean_ns 14.42 µs 14.44 µs 🟢▲ +0.02% 0.44%
pingpong_lockless/1024 epoll latency_mean_ns 14.96 µs 14.97 µs 🔴▼ -0.08% 0.23%
pingpong_lockless/1024 uring latency_mean_ns 14.69 µs 14.76 µs 🔴▼ -0.05% 1.10%
pingpong_lockless/64 epoll latency_mean_ns 14.70 µs 14.70 µs 🔴▼ -0.16% 0.33%
pingpong_lockless/64 uring latency_mean_ns 14.43 µs 14.52 µs 🔴▼ -0.15% 1.28%

socket_throughput

Benchmark Backend Metric Base Head Δ Noise
bidirectional/1024 epoll bytes_per_sec 133.9 MB/s 134.1 MB/s 🟢▲ +0.15% 0.46%
bidirectional/1024 uring bytes_per_sec 141.1 MB/s 140.7 MB/s 🔴▼ -0.01% 0.91%
bidirectional/1048576 epoll bytes_per_sec 3.33 GB/s 3.24 GB/s 🔴▼ -2.70% 2.24%
bidirectional/1048576 uring bytes_per_sec 3.24 GB/s 3.20 GB/s 🔴▼ -0.78% 1.47%
bidirectional/16384 epoll bytes_per_sec 1.64 GB/s 1.64 GB/s 🟢▲ +0.16% 0.86%
bidirectional/16384 uring bytes_per_sec 1.55 GB/s 1.51 GB/s 🔴▼ -2.29% 1.17%
bidirectional/262144 epoll bytes_per_sec 3.20 GB/s 3.18 GB/s 🔴▼ -0.78% 1.38%
bidirectional/262144 uring bytes_per_sec 3.18 GB/s 3.16 GB/s 🔴▼ -0.69% 0.95%
bidirectional/4096 epoll bytes_per_sec 506.1 MB/s 506.0 MB/s 🟢▲ +0.40% 0.93%
bidirectional/4096 uring bytes_per_sec 495.7 MB/s 492.9 MB/s 🔴▼ -0.40% 0.91%
bidirectional/65536 epoll bytes_per_sec 2.50 GB/s 2.53 GB/s 🟢▲ +0.57% 2.07%
bidirectional/65536 uring bytes_per_sec 2.76 GB/s 2.74 GB/s 🔴▼ -0.75% 2.02%
bidirectional_lockless/1024 epoll bytes_per_sec 135.6 MB/s 134.5 MB/s 🔴▼ -0.12% 1.26%
bidirectional_lockless/1024 uring bytes_per_sec 141.0 MB/s 141.0 MB/s 🟢▲ +0.32% 1.01%
bidirectional_lockless/1048576 epoll bytes_per_sec 3.24 GB/s 3.23 GB/s 🟢▲ +0.71% 1.96%
bidirectional_lockless/1048576 uring bytes_per_sec 3.31 GB/s 3.32 GB/s 🟢▲ +0.41% 1.32%
bidirectional_lockless/16384 epoll bytes_per_sec 1.63 GB/s 1.64 GB/s 🟢▲ +0.99% 1.51%
bidirectional_lockless/16384 uring bytes_per_sec 1.54 GB/s 1.53 GB/s 🔴▼ -0.13% 0.94%
bidirectional_lockless/262144 epoll bytes_per_sec 3.20 GB/s 3.19 GB/s 🔴▼ -0.55% 1.03%
bidirectional_lockless/262144 uring bytes_per_sec 3.22 GB/s 3.23 GB/s 🟢▲ +0.18% 1.19%
bidirectional_lockless/4096 epoll bytes_per_sec 510.0 MB/s 509.2 MB/s 🟢▲ +0.28% 1.37%
bidirectional_lockless/4096 uring bytes_per_sec 492.9 MB/s 495.6 MB/s 🟢▲ +0.55% 0.43%
bidirectional_lockless/65536 epoll bytes_per_sec 2.53 GB/s 2.53 GB/s 🟢▲ +0.28% 2.97%
bidirectional_lockless/65536 uring bytes_per_sec 2.82 GB/s 2.81 GB/s 🟢▲ +0.47% 2.09%
multithread/2 epoll bytes_per_sec 3.41 GB/s 3.42 GB/s 🟢▲ +0.37% 0.79%
multithread/2 uring bytes_per_sec 3.67 GB/s 3.66 GB/s 🔴▼ -0.05% 0.37%
multithread/4 epoll bytes_per_sec 4.84 GB/s 4.86 GB/s 🟢▲ +0.91% 0.86%
multithread/4 uring bytes_per_sec 5.06 GB/s 5.07 GB/s 🟢▲ +0.17% 0.33%
multithread/8 epoll bytes_per_sec 4.95 GB/s 4.95 GB/s ⚪ 0.00% 0.28%
multithread/8 uring bytes_per_sec 5.12 GB/s 5.12 GB/s 🔴▼ -0.11% 0.40%
unidirectional/1024 epoll bytes_per_sec 125.3 MB/s 125.8 MB/s 🟢▲ +0.23% 0.58%
unidirectional/1024 uring bytes_per_sec 138.9 MB/s 140.1 MB/s 🟢▲ +0.11% 1.15%
unidirectional/1048576 epoll bytes_per_sec 3.33 GB/s 3.35 GB/s 🔴▼ -0.54% 1.38%
unidirectional/1048576 uring bytes_per_sec 3.40 GB/s 3.41 GB/s 🔴▼ -0.01% 1.03%
unidirectional/16384 epoll bytes_per_sec 1.55 GB/s 1.56 GB/s 🟢▲ +0.35% 0.97%
unidirectional/16384 uring bytes_per_sec 1.56 GB/s 1.54 GB/s 🔴▼ -0.39% 2.96%
unidirectional/262144 epoll bytes_per_sec 3.31 GB/s 3.32 GB/s 🟢▲ +0.35% 1.13%
unidirectional/262144 uring bytes_per_sec 3.20 GB/s 3.18 GB/s 🔴▼ -0.95% 1.66%
unidirectional/4096 epoll bytes_per_sec 474.3 MB/s 472.7 MB/s 🟢▲ +0.32% 1.24%
unidirectional/4096 uring bytes_per_sec 493.9 MB/s 496.1 MB/s 🔴▼ -0.02% 1.26%
unidirectional/65536 epoll bytes_per_sec 2.73 GB/s 2.60 GB/s 🔴▼ -5.56% 4.75%
unidirectional/65536 uring bytes_per_sec 2.70 GB/s 2.68 GB/s 🔴▼ -2.09% 4.58%
unidirectional_lockless/1024 epoll bytes_per_sec 127.1 MB/s 126.1 MB/s 🟢▲ +0.12% 1.62%
unidirectional_lockless/1024 uring bytes_per_sec 139.3 MB/s 139.8 MB/s 🟢▲ +0.26% 0.54%
unidirectional_lockless/1048576 epoll bytes_per_sec 3.35 GB/s 3.33 GB/s 🔴▼ -0.99% 0.74%
unidirectional_lockless/1048576 uring bytes_per_sec 3.32 GB/s 3.32 GB/s 🔴▼ -0.03% 0.73%
unidirectional_lockless/16384 epoll bytes_per_sec 1.56 GB/s 1.57 GB/s 🟢▲ +0.61% 1.09%
unidirectional_lockless/16384 uring bytes_per_sec 1.61 GB/s 1.60 GB/s 🟢▲ +0.58% 2.41%
unidirectional_lockless/262144 epoll bytes_per_sec 3.34 GB/s 3.32 GB/s 🔴▼ -0.56% 1.13%
unidirectional_lockless/262144 uring bytes_per_sec 3.23 GB/s 3.21 GB/s 🔴▼ -1.11% 1.03%
unidirectional_lockless/4096 epoll bytes_per_sec 476.1 MB/s 473.8 MB/s 🔴▼ -0.14% 1.33%
unidirectional_lockless/4096 uring bytes_per_sec 492.6 MB/s 489.7 MB/s 🔴▼ -0.26% 0.95%
unidirectional_lockless/65536 epoll bytes_per_sec 2.77 GB/s 2.76 GB/s 🔴▼ -0.35% 1.19%
unidirectional_lockless/65536 uring bytes_per_sec 2.65 GB/s 2.71 GB/s 🟢▲ +1.24% 2.23%
windows — full results (77 benchmarks, 5 iterations, 1.5s each)

accept_churn

Benchmark Backend Metric Base Head Δ Noise
burst/10 iocp ops_per_sec 1.56K ops/s 1.53K ops/s 🔴▼ -1.57% 0.33%
burst/100 iocp ops_per_sec 152.2 ops/s 149.1 ops/s 🔴▼ -2.52% 0.60%
burst_lockless/10 iocp ops_per_sec 1.57K ops/s 1.54K ops/s 🔴▼ -1.85% 0.48%
burst_lockless/100 iocp ops_per_sec 156.7 ops/s 151.9 ops/s 🔴▼ -3.02% 0.71%
concurrent/1 iocp ops_per_sec 12.41K ops/s 12.24K ops/s 🔴▼ -1.62% 1.51%
concurrent/16 iocp ops_per_sec 13.33K ops/s 13.25K ops/s 🔴▼ -0.43% 0.34%
concurrent/4 iocp ops_per_sec 13.10K ops/s 12.94K ops/s 🔴▼ -1.23% 0.32%
sequential iocp ops_per_sec 12.37K ops/s 12.14K ops/s 🔴▼ -1.73% 1.58%
sequential_lockless iocp ops_per_sec 12.41K ops/s 12.20K ops/s 🔴▼ -1.81% 1.50%

fan_out

Benchmark Backend Metric Base Head Δ Noise
concurrent_parents/1 iocp ops_per_sec 7.19K ops/s 7.18K ops/s 🔴▼ -0.15% 0.26%
concurrent_parents/16 iocp ops_per_sec 6.42K ops/s 6.43K ops/s 🟢▲ +0.05% 1.27%
concurrent_parents/4 iocp ops_per_sec 6.90K ops/s 6.90K ops/s 🔴▼ -0.21% 0.77%
concurrent_parents_lockless/1 iocp ops_per_sec 7.19K ops/s 7.18K ops/s 🔴▼ -0.18% 0.36%
concurrent_parents_lockless/16 iocp ops_per_sec 6.46K ops/s 6.44K ops/s 🔴▼ -0.23% 1.37%
concurrent_parents_lockless/4 iocp ops_per_sec 6.94K ops/s 6.94K ops/s 🟢▲ +0.12% 0.48%
fork_join/1 iocp ops_per_sec 113.0K ops/s 112.8K ops/s 🔴▼ -0.10% 0.43%
fork_join/16 iocp ops_per_sec 7.18K ops/s 7.14K ops/s 🔴▼ -0.83% 0.69%
fork_join/4 iocp ops_per_sec 29.24K ops/s 29.25K ops/s 🔴▼ -0.06% 0.25%
fork_join/64 iocp ops_per_sec 1.72K ops/s 1.73K ops/s 🟢▲ +0.37% 1.60%
fork_join_lockless/1 iocp ops_per_sec 112.9K ops/s 112.6K ops/s 🔴▼ -0.04% 0.42%
fork_join_lockless/16 iocp ops_per_sec 7.17K ops/s 7.20K ops/s 🟢▲ +0.35% 0.28%
fork_join_lockless/4 iocp ops_per_sec 29.22K ops/s 29.24K ops/s 🟢▲ +0.13% 0.25%
fork_join_lockless/64 iocp ops_per_sec 1.73K ops/s 1.73K ops/s 🟢▲ +0.02% 0.84%
nested/16 iocp ops_per_sec 1.70K ops/s 1.70K ops/s 🔴▼ -0.03% 0.77%
nested/4 iocp ops_per_sec 7.02K ops/s 7.04K ops/s 🟢▲ +0.50% 0.36%
nested_lockless/16 iocp ops_per_sec 1.70K ops/s 1.70K ops/s 🟢▲ +0.25% 0.81%
nested_lockless/4 iocp ops_per_sec 7.03K ops/s 7.02K ops/s 🔴▼ -0.08% 0.16%

http_server

Benchmark Backend Metric Base Head Δ Noise
concurrent/1 iocp ops_per_sec 112.0K ops/s 111.5K ops/s 🔴▼ -0.30% 0.29%
concurrent/16 iocp ops_per_sec 108.7K ops/s 108.0K ops/s 🔴▼ -0.60% 0.26%
concurrent/32 iocp ops_per_sec 108.0K ops/s 107.7K ops/s 🔴▼ -0.37% 0.27%
concurrent/4 iocp ops_per_sec 109.4K ops/s 109.2K ops/s 🔴▼ -0.52% 0.33%
multithread/1 iocp ops_per_sec 107.9K ops/s 107.7K ops/s 🔴▼ -0.19% 0.32%
multithread/16 iocp ops_per_sec 271.0K ops/s 271.3K ops/s 🟢▲ +0.09% 0.66%
multithread/2 iocp ops_per_sec 182.2K ops/s 182.2K ops/s 🟢▲ +0.10% 0.68%
multithread/4 iocp ops_per_sec 227.5K ops/s 228.9K ops/s 🟢▲ +0.66% 0.52%
multithread/8 iocp ops_per_sec 269.9K ops/s 272.5K ops/s 🟢▲ +0.90% 0.43%
single_conn iocp ops_per_sec 111.7K ops/s 111.2K ops/s 🔴▼ -0.28% 0.34%
single_conn_lockless iocp ops_per_sec 111.7K ops/s 111.4K ops/s 🔴▼ -0.21% 0.37%

socket_latency

Benchmark Backend Metric Base Head Δ Noise
concurrent/1 iocp latency_mean_ns 7.61 µs 7.60 µs 🟢▲ +0.28% 0.44%
concurrent/16 iocp latency_mean_ns 133.24 µs 126.02 µs 🟢▲ +5.51% 5.09%
concurrent/4 iocp latency_mean_ns 30.62 µs 30.54 µs 🟢▲ +0.32% 0.29%
concurrent_lockless/1 iocp latency_mean_ns 7.61 µs 7.60 µs 🟢▲ +0.11% 0.44%
concurrent_lockless/16 iocp latency_mean_ns 121.92 µs 122.20 µs 🟢▲ +0.14% 1.02%
concurrent_lockless/4 iocp latency_mean_ns 30.60 µs 30.44 µs 🟢▲ +0.33% 0.38%
pingpong/1 iocp latency_mean_ns 7.59 µs 7.60 µs ⚪ 0.00% 0.61%
pingpong/1024 iocp latency_mean_ns 7.70 µs 7.69 µs 🟢▲ +0.02% 0.40%
pingpong/64 iocp latency_mean_ns 7.62 µs 7.63 µs 🔴▼ -0.12% 0.38%
pingpong_lockless/1 iocp latency_mean_ns 7.59 µs 7.59 µs 🟢▲ +0.09% 0.58%
pingpong_lockless/1024 iocp latency_mean_ns 7.71 µs 7.69 µs 🟢▲ +0.46% 0.48%
pingpong_lockless/64 iocp latency_mean_ns 7.60 µs 7.62 µs 🔴▼ -0.22% 0.31%

socket_throughput

Benchmark Backend Metric Base Head Δ Noise
bidirectional/1024 iocp bytes_per_sec 279.3 MB/s 280.4 MB/s 🟢▲ +0.40% 0.59%
bidirectional/1048576 iocp bytes_per_sec 4.42 GB/s 4.40 GB/s 🔴▼ -0.90% 7.39%
bidirectional/16384 iocp bytes_per_sec 3.71 GB/s 3.70 GB/s 🔴▼ -0.81% 1.15%
bidirectional/262144 iocp bytes_per_sec 9.50 GB/s 9.34 GB/s 🔴▼ -1.17% 3.37%
bidirectional/4096 iocp bytes_per_sec 1.07 GB/s 1.08 GB/s 🟢▲ +0.55% 0.33%
bidirectional/65536 iocp bytes_per_sec 8.89 GB/s 8.75 GB/s 🔴▼ -1.48% 4.13%
bidirectional_lockless/1024 iocp bytes_per_sec 279.2 MB/s 280.8 MB/s 🟢▲ +0.70% 0.51%
bidirectional_lockless/1048576 iocp bytes_per_sec 4.57 GB/s 4.60 GB/s 🔴▼ -1.24% 7.18%
bidirectional_lockless/16384 iocp bytes_per_sec 3.71 GB/s 3.75 GB/s 🟢▲ +1.09% 0.76%
bidirectional_lockless/262144 iocp bytes_per_sec 9.68 GB/s 9.65 GB/s 🟢▲ +0.57% 1.10%
bidirectional_lockless/4096 iocp bytes_per_sec 1.07 GB/s 1.08 GB/s 🟢▲ +0.11% 0.49%
bidirectional_lockless/65536 iocp bytes_per_sec 8.98 GB/s 9.04 GB/s 🟢▲ +1.83% 1.98%
multithread/2 iocp bytes_per_sec 12.71 GB/s 12.77 GB/s 🟢▲ +0.42% 0.73%
multithread/4 iocp bytes_per_sec 17.03 GB/s 17.11 GB/s 🟢▲ +0.09% 1.44%
multithread/8 iocp bytes_per_sec 18.30 GB/s 18.47 GB/s 🟢▲ +0.99% 0.85%
unidirectional/1024 iocp bytes_per_sec 283.0 MB/s 284.6 MB/s 🟢▲ +0.40% 0.51%
unidirectional/1048576 iocp bytes_per_sec 3.88 GB/s 3.92 GB/s 🟢▲ +2.42% 3.75%
unidirectional/16384 iocp bytes_per_sec 3.78 GB/s 3.81 GB/s 🟢▲ +0.98% 0.46%
unidirectional/262144 iocp bytes_per_sec 8.75 GB/s 9.15 GB/s 🟢▲ +3.21% 4.74%
unidirectional/4096 iocp bytes_per_sec 1.09 GB/s 1.09 GB/s 🟢▲ +0.07% 0.39%
unidirectional/65536 iocp bytes_per_sec 9.15 GB/s 9.32 GB/s 🟢▲ +2.15% 1.42%
unidirectional_lockless/1024 iocp bytes_per_sec 283.7 MB/s 283.5 MB/s 🟢▲ +0.09% 0.41%
unidirectional_lockless/1048576 iocp bytes_per_sec 4.21 GB/s 4.21 GB/s 🟢▲ +0.89% 3.70%
unidirectional_lockless/16384 iocp bytes_per_sec 3.76 GB/s 3.81 GB/s 🟢▲ +1.53% 0.71%
unidirectional_lockless/262144 iocp bytes_per_sec 9.63 GB/s 9.68 GB/s 🟢▲ +0.93% 1.22%
unidirectional_lockless/4096 iocp bytes_per_sec 1.09 GB/s 1.09 GB/s 🟢▲ +0.23% 1.24%
unidirectional_lockless/65536 iocp bytes_per_sec 9.20 GB/s 9.40 GB/s 🟢▲ +2.61% 1.64%
macos — full results (113 benchmarks, 5 iterations, 1.5s each)

accept_churn

Benchmark Backend Metric Base Head Δ Noise
burst/10 kqueue ops_per_sec 4.25K ops/s 4.25K ops/s 🟢▲ +0.12% 0.28%
burst/100 kqueue ops_per_sec 430.7 ops/s 431.3 ops/s 🟢▲ +0.18% 0.81%
burst_lockless/10 kqueue ops_per_sec 4.26K ops/s 4.28K ops/s 🟢▲ +0.32% 0.93%
burst_lockless/100 kqueue ops_per_sec 432.2 ops/s 434.4 ops/s 🟢▲ +0.66% 0.54%
concurrent/1 kqueue ops_per_sec 19.59K ops/s 19.41K ops/s 🔴▼ -0.66% 0.76%
concurrent/16 kqueue ops_per_sec 35.24K ops/s 35.35K ops/s 🟢▲ +0.30% 0.87%
concurrent/4 kqueue ops_per_sec 29.14K ops/s 29.48K ops/s 🟢▲ +0.79% 1.27%
sequential kqueue ops_per_sec 19.91K ops/s 19.86K ops/s 🔴▼ -0.17% 0.71%
sequential_lockless kqueue ops_per_sec 19.62K ops/s 19.90K ops/s 🟢▲ +1.47% 0.67%

fan_out

Benchmark Backend Metric Base Head Δ Noise
concurrent_parents/1 kqueue ops_per_sec 7.72K ops/s 7.73K ops/s 🟢▲ +0.01% 1.25%
concurrent_parents/16 kqueue ops_per_sec 6.99K ops/s 7.10K ops/s 🟢▲ +2.42% 19.50%
concurrent_parents/4 kqueue ops_per_sec 8.18K ops/s 8.18K ops/s 🔴▼ -0.12% 2.08%
concurrent_parents_lockless/1 kqueue ops_per_sec 7.77K ops/s 7.77K ops/s 🟢▲ +0.27% 1.19%
concurrent_parents_lockless/16 kqueue ops_per_sec 7.03K ops/s 7.05K ops/s 🔴▼ -0.38% 20.69%
concurrent_parents_lockless/4 kqueue ops_per_sec 8.27K ops/s 8.25K ops/s 🔴▼ -0.19% 1.71%
fork_join/1 kqueue ops_per_sec 46.06K ops/s 45.88K ops/s 🔴▼ -0.18% 0.75%
fork_join/16 kqueue ops_per_sec 7.69K ops/s 7.71K ops/s 🟢▲ +0.19% 0.77%
fork_join/4 kqueue ops_per_sec 21.70K ops/s 21.61K ops/s 🟢▲ +0.16% 0.75%
fork_join/64 kqueue ops_per_sec 2.04K ops/s 2.06K ops/s 🟢▲ +0.39% 2.05%
fork_join_lockless/1 kqueue ops_per_sec 46.94K ops/s 47.05K ops/s 🟢▲ +0.31% 0.18%
fork_join_lockless/16 kqueue ops_per_sec 7.77K ops/s 7.80K ops/s 🟢▲ +0.41% 0.88%
fork_join_lockless/4 kqueue ops_per_sec 21.81K ops/s 21.78K ops/s 🔴▼ -0.18% 0.82%
fork_join_lockless/64 kqueue ops_per_sec 2.06K ops/s 2.07K ops/s 🟢▲ +0.17% 2.02%
nested/16 kqueue ops_per_sec 2.01K ops/s 2.03K ops/s 🟢▲ +0.37% 1.77%
nested/4 kqueue ops_per_sec 7.06K ops/s 7.08K ops/s 🟢▲ +0.27% 0.90%
nested_lockless/16 kqueue ops_per_sec 2.03K ops/s 2.05K ops/s 🟢▲ +0.47% 1.96%
nested_lockless/4 kqueue ops_per_sec 7.19K ops/s 7.20K ops/s 🟢▲ +0.32% 1.15%

http_server

Benchmark Backend Metric Base Head Δ Noise
concurrent/1 kqueue ops_per_sec 49.62K ops/s 49.67K ops/s 🟢▲ +0.08% 0.22%
concurrent/16 kqueue ops_per_sec 128.3K ops/s 127.6K ops/s 🔴▼ -0.40% 0.75%
concurrent/32 kqueue ops_per_sec 128.0K ops/s 128.3K ops/s 🟢▲ +0.91% 1.54%
concurrent/4 kqueue ops_per_sec 88.40K ops/s 87.88K ops/s 🔴▼ -0.23% 0.85%
multithread/1 kqueue ops_per_sec 127.9K ops/s 128.1K ops/s 🟢▲ +0.75% 1.34%
multithread/16 kqueue ops_per_sec 127.5K ops/s 128.1K ops/s 🟢▲ +0.54% 1.28%
multithread/2 kqueue ops_per_sec 177.6K ops/s 173.8K ops/s 🔴▼ -3.04% 1.72%
multithread/4 kqueue ops_per_sec 161.2K ops/s 161.2K ops/s 🔴▼ -0.41% 1.18%
multithread/8 kqueue ops_per_sec 132.2K ops/s 133.0K ops/s 🔴▼ -0.20% 1.28%
single_conn kqueue ops_per_sec 49.62K ops/s 49.61K ops/s 🟢▲ +0.07% 0.40%
single_conn_lockless kqueue ops_per_sec 49.78K ops/s 49.84K ops/s 🟢▲ +0.05% 0.34%

local_socket_latency

Benchmark Backend Metric Base Head Δ Noise
concurrent/1 kqueue latency_mean_ns 2.07 µs 2.06 µs 🟢▲ +0.26% 0.35%
concurrent/16 kqueue latency_mean_ns 30.68 µs 30.54 µs 🟢▲ +0.21% 0.57%
concurrent/4 kqueue latency_mean_ns 7.68 µs 7.66 µs 🟢▲ +0.09% 0.58%
concurrent_lockless/1 kqueue latency_mean_ns 2.04 µs 2.03 µs 🟢▲ +0.18% 0.67%
concurrent_lockless/16 kqueue latency_mean_ns 30.22 µs 30.15 µs 🟢▲ +0.24% 0.24%
concurrent_lockless/4 kqueue latency_mean_ns 7.56 µs 7.53 µs 🟢▲ +0.37% 0.45%
pingpong/1 kqueue latency_mean_ns 2.06 µs 2.06 µs 🟢▲ +0.35% 0.73%
pingpong/1024 kqueue latency_mean_ns 2.16 µs 2.16 µs 🟢▲ +0.02% 0.57%
pingpong/64 kqueue latency_mean_ns 2.06 µs 2.06 µs 🟢▲ +0.14% 0.77%
pingpong_lockless/1 kqueue latency_mean_ns 2.03 µs 2.02 µs 🟢▲ +0.25% 0.78%
pingpong_lockless/1024 kqueue latency_mean_ns 2.14 µs 2.13 µs 🟢▲ +0.13% 0.47%
pingpong_lockless/64 kqueue latency_mean_ns 2.03 µs 2.03 µs 🟢▲ +0.31% 0.58%

local_socket_throughput

Benchmark Backend Metric Base Head Δ Noise
bidirectional/1024 kqueue bytes_per_sec 1.22 GB/s 1.22 GB/s 🔴▼ -0.11% 0.78%
bidirectional/1048576 kqueue bytes_per_sec 3.84 GB/s 3.83 GB/s 🔴▼ -0.33% 0.55%
bidirectional/16384 kqueue bytes_per_sec 3.85 GB/s 3.83 GB/s 🔴▼ -0.26% 0.72%
bidirectional/262144 kqueue bytes_per_sec 3.84 GB/s 3.81 GB/s 🔴▼ -1.09% 0.72%
bidirectional/4096 kqueue bytes_per_sec 2.90 GB/s 2.90 GB/s 🟢▲ +0.26% 0.75%
bidirectional/65536 kqueue bytes_per_sec 3.85 GB/s 3.83 GB/s 🔴▼ -0.71% 0.67%
bidirectional_lockless/1024 kqueue bytes_per_sec 1.23 GB/s 1.23 GB/s 🟢▲ +0.10% 1.14%
bidirectional_lockless/1048576 kqueue bytes_per_sec 3.90 GB/s 3.92 GB/s 🟢▲ +0.58% 0.69%
bidirectional_lockless/16384 kqueue bytes_per_sec 3.90 GB/s 3.91 GB/s 🟢▲ +0.14% 0.85%
bidirectional_lockless/262144 kqueue bytes_per_sec 3.92 GB/s 3.91 GB/s 🔴▼ -0.25% 0.68%
bidirectional_lockless/4096 kqueue bytes_per_sec 2.95 GB/s 2.94 GB/s 🔴▼ -0.21% 0.51%
bidirectional_lockless/65536 kqueue bytes_per_sec 3.89 GB/s 3.90 GB/s 🟢▲ +0.44% 0.79%
unidirectional/1024 kqueue bytes_per_sec 1.19 GB/s 1.18 GB/s 🔴▼ -0.47% 1.20%
unidirectional/1048576 kqueue bytes_per_sec 3.47 GB/s 3.45 GB/s 🔴▼ -0.40% 0.52%
unidirectional/16384 kqueue bytes_per_sec 3.46 GB/s 3.46 GB/s 🟢▲ +0.03% 0.63%
unidirectional/262144 kqueue bytes_per_sec 3.46 GB/s 3.46 GB/s 🟢▲ +0.15% 0.47%
unidirectional/4096 kqueue bytes_per_sec 2.69 GB/s 2.67 GB/s 🔴▼ -0.67% 0.85%
unidirectional/65536 kqueue bytes_per_sec 3.46 GB/s 3.46 GB/s 🔴▼ -0.09% 0.58%
unidirectional_lockless/1024 kqueue bytes_per_sec 1.20 GB/s 1.19 GB/s 🔴▼ -0.33% 1.06%
unidirectional_lockless/1048576 kqueue bytes_per_sec 3.52 GB/s 3.52 GB/s 🟢▲ +0.04% 0.58%
unidirectional_lockless/16384 kqueue bytes_per_sec 3.51 GB/s 3.51 GB/s 🟢▲ +0.16% 0.64%
unidirectional_lockless/262144 kqueue bytes_per_sec 3.52 GB/s 3.50 GB/s 🔴▼ -0.39% 0.57%
unidirectional_lockless/4096 kqueue bytes_per_sec 2.72 GB/s 2.71 GB/s 🟢▲ +0.10% 0.87%
unidirectional_lockless/65536 kqueue bytes_per_sec 3.52 GB/s 3.51 GB/s 🔴▼ -0.25% 0.61%

socket_latency

Benchmark Backend Metric Base Head Δ Noise
concurrent/1 kqueue latency_mean_ns 18.81 µs 18.70 µs 🟢▲ +0.50% 0.72%
concurrent/16 kqueue latency_mean_ns 121.00 µs 120.41 µs 🟢▲ +0.37% 0.44%
concurrent/4 kqueue latency_mean_ns 44.09 µs 43.76 µs 🟢▲ +0.42% 0.99%
concurrent_lockless/1 kqueue latency_mean_ns 18.76 µs 18.66 µs 🟢▲ +0.58% 0.35%
concurrent_lockless/16 kqueue latency_mean_ns 120.38 µs 120.74 µs 🔴▼ -0.38% 0.16%
concurrent_lockless/4 kqueue latency_mean_ns 43.44 µs 42.97 µs 🟢▲ +0.70% 1.24%
pingpong/1 kqueue latency_mean_ns 15.45 µs 15.42 µs 🟢▲ +0.25% 0.30%
pingpong/1024 kqueue latency_mean_ns 18.90 µs 18.91 µs 🔴▼ -0.13% 0.33%
pingpong/64 kqueue latency_mean_ns 18.74 µs 18.69 µs 🟢▲ +0.27% 0.40%
pingpong_lockless/1 kqueue latency_mean_ns 15.41 µs 15.41 µs 🟢▲ +0.15% 0.54%
pingpong_lockless/1024 kqueue latency_mean_ns 18.95 µs 18.89 µs 🟢▲ +0.26% 0.19%
pingpong_lockless/64 kqueue latency_mean_ns 18.70 µs 18.65 µs 🟢▲ +0.26% 0.23%

socket_throughput

Benchmark Backend Metric Base Head Δ Noise
bidirectional/1024 kqueue bytes_per_sec 410.0 MB/s 406.1 MB/s 🔴▼ -0.55% 1.09%
bidirectional/1048576 kqueue bytes_per_sec 7.80 GB/s 7.63 GB/s 🔴▼ -1.10% 4.28%
bidirectional/16384 kqueue bytes_per_sec 4.00 GB/s 3.96 GB/s 🔴▼ -1.33% 0.80%
bidirectional/262144 kqueue bytes_per_sec 8.89 GB/s 8.79 GB/s 🔴▼ -1.77% 3.18%
bidirectional/4096 kqueue bytes_per_sec 1.50 GB/s 1.49 GB/s 🔴▼ -0.47% 0.68%
bidirectional/65536 kqueue bytes_per_sec 7.68 GB/s 7.57 GB/s 🔴▼ -2.08% 2.60%
bidirectional_lockless/1024 kqueue bytes_per_sec 411.4 MB/s 412.4 MB/s 🟢▲ +0.15% 1.42%
bidirectional_lockless/1048576 kqueue bytes_per_sec 7.76 GB/s 7.69 GB/s 🟢▲ +0.86% 5.63%
bidirectional_lockless/16384 kqueue bytes_per_sec 4.04 GB/s 3.98 GB/s 🔴▼ -1.99% 1.18%
bidirectional_lockless/262144 kqueue bytes_per_sec 9.01 GB/s 8.70 GB/s 🔴▼ -3.42% 6.32%
bidirectional_lockless/4096 kqueue bytes_per_sec 1.50 GB/s 1.49 GB/s 🔴▼ -0.39% 0.95%
bidirectional_lockless/65536 kqueue bytes_per_sec 7.61 GB/s 7.62 GB/s 🟢▲ +0.43% 1.30%
multithread/2 kqueue bytes_per_sec 7.60 GB/s 7.45 GB/s 🟢▲ +0.41% 3.68%
multithread/4 kqueue bytes_per_sec 3.66 GB/s 3.59 GB/s 🔴▼ -2.29% 3.95%
multithread/8 kqueue bytes_per_sec 3.42 GB/s 3.28 GB/s 🔴▼ -5.30% 4.56%
unidirectional/1024 kqueue bytes_per_sec 319.2 MB/s 318.4 MB/s 🔴▼ -0.64% 1.06%
unidirectional/1048576 kqueue bytes_per_sec 9.01 GB/s 8.75 GB/s 🔴▼ -0.92% 5.37%
unidirectional/16384 kqueue bytes_per_sec 3.73 GB/s 3.73 GB/s 🟢▲ +0.04% 0.94%
unidirectional/262144 kqueue bytes_per_sec 8.99 GB/s 8.93 GB/s 🟢▲ +0.18% 2.67%
unidirectional/4096 kqueue bytes_per_sec 1.23 GB/s 1.24 GB/s 🔴▼ -0.24% 2.08%
unidirectional/65536 kqueue bytes_per_sec 7.28 GB/s 7.29 GB/s 🟢▲ +0.81% 1.39%
unidirectional_lockless/1024 kqueue bytes_per_sec 328.0 MB/s 327.4 MB/s 🔴▼ -0.20% 0.36%
unidirectional_lockless/1048576 kqueue bytes_per_sec 9.04 GB/s 8.80 GB/s 🔴▼ -0.66% 4.25%
unidirectional_lockless/16384 kqueue bytes_per_sec 3.74 GB/s 3.76 GB/s 🔴▼ -0.18% 1.34%
unidirectional_lockless/262144 kqueue bytes_per_sec 8.97 GB/s 8.97 GB/s 🟢▲ +0.26% 2.46%
unidirectional_lockless/4096 kqueue bytes_per_sec 1.25 GB/s 1.25 GB/s 🟢▲ +0.09% 0.35%
unidirectional_lockless/65536 kqueue bytes_per_sec 7.34 GB/s 7.29 GB/s 🔴▼ -0.63% 1.20%

Run & raw JSON artifacts · flag rule: |median Δ| > max(2%, 3×CV of the noisier side) · advisory only

@sgerbino
sgerbino force-pushed the pr/benchmark-report branch 3 times, most recently from 1137e95 to 60bf84c Compare October 2, 2026 16:46
@sgerbino
sgerbino marked this pull request as ready for review October 2, 2026 19:20
@sgerbino
bench: skip local socket suites on Windows
a1a2ba8 
local::connect_pair relies on socketpair(), which has no Windows
implementation and throws operation_not_supported at runtime, taking
the whole comparison run down with it. The corosio local socket suites
are already POSIX-only, so Windows loses no comparison coverage.
@sgerbino
bench: prevent TIME_WAIT pileup in the churn benchmarks
f9f3f29 
Only the connecting side carried linger(0), so on loopback a server
FIN racing the client RST could still park connections in TIME_WAIT.
At production durations the leak overflowed the macOS PCB table
mid-benchmark (70k against a 12k drain threshold), socket creation
started failing, and the corosio loop ignored the open() error before
set_option threw and took down the whole suite. The listener now
carries the churn socket options -- accepted sockets inherit them on
every supported platform -- so both sides RST at close without adding
per-connection syscalls to the measured loop, and a failed open() now
ends the benchmark instead of aborting the process.
@sgerbino
bench: publish a generated, reproducible benchmark report
99008d2 
Replaces the hand-written February benchmark page with a pipeline that
regenerates it from fresh runs on the three dedicated machines
(closes #352):

- benchmark-report.yml: dispatch-only workflow running the comparison
  suite (corosio, asio coroutines, asio callbacks) on every platform,
  uploading raw per-iteration JSON and an environment capture. Linux
  builds asio against both epoll and io_uring so each corosio
  configuration is measured against a same-reactor asio.
- report_page.py: stdlib-only generator producing the four Antora
  pages and theme-aware inline SVG charts; medians across iterations,
  CV-based noise classification, diverging bars against the
  asio-coroutines baseline.
- Scenario descriptions in the bench harness (describe and
  category_description), rendered as category intros and chart hover
  text.
- bench/README.md: how to run the suite, dispatch the workflow, and
  regenerate the pages.
- Generated pages and charts from a full 7x2 production run; the old
  benchmark-report.adoc URL survives via a page alias, nav gains the
  per-platform entries, and the lint baseline follows the page swap.
@sgerbino
sgerbino force-pushed the pr/benchmark-report branch from 60bf84c to 99008d2 Compare October 2, 2026 21:55
@cppalliance-bot

cppalliance-bot commented Oct 2, 2026

Copy link
Copy Markdown

An automated preview of the documentation is available at https://377.corosio.prtest3.cppalliance.org/index.html

If more commits are pushed to the pull request, the docs will rebuild at the same URL.

2026-10-02 22:00:20 UTC

@sgerbino
sgerbino merged commit 1533914 into develop Oct 6, 2026
90 checks passed
@sgerbino
sgerbino deleted the pr/benchmark-report branch October 6, 2026 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mvandeberg mvandeberg mvandeberg approved these changes

Assignees

No one assigned

Labels

None yet

Projects

Beast2
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

perf: make the published benchmark report reproducible and current

3 participants

@sgerbino @cppalliance-bot @mvandeberg