Skip to content

docs: re-run the fork-vs-upstream benchmark on 633fc82c against upstream 986de95c - #527

Merged
bompus merged 1 commit into
fork/consolidatedfrom
docs/readme-bench-20261018
Oct 18, 2026
Merged

bompus merged 1 commit into
fork/consolidatedfrom
docs/readme-bench-20261018

Conversation

@bompus

@bompus bompus commented Oct 18, 2026 •

Copy link
Copy Markdown
Owner

Against upstream's current main, the fork indexes n8n 1.3 times faster (66.3 s against 84.5 s on Node), syncs one edited file 3.0 times faster and is better in 65 of 74 comparisons, about the same in 4 and worse in 5.

The README's "Measured results" quoted the 2026-10-14 run (fork ba972275, upstream 3f714bcc). Since then the fork gained the mid-size resolver pool (#506), the Kotlin, regex-cache and WAL-checkpoint changes (#515 to #519), imported-constant member links (#523) and the markup arrow-parameter guard (#526). This repeats the same run with the same scripts, corpora, arms and quiet-host rule on fork 633fc82c and upstream 986de95c (Node 24.22.0, Bun 1.4.3, 16-vCPU WSL2 host, mirrored arm order, index median of 2, sync median of 4). The fork lacks one upstream commit, 986de95c, a Vue import fix, so the arms differ in it. Protocol written before the first timed run; no attempt was rejected; all steps returned 0 apart from upstream's supabase syncs, which fail with FOREIGN KEY constraint failed as before (colbymchenry#2565) and stay out of the 74.

What moved in the README, Node arms (the full tables are in the new doc):

Measure 2026-10-14 2026-10-18 Note
n8n index, upstream → fork 88.3 s → 71.2 s (1.2×) 84.5 s → 66.3 s (1.3×)
CPython index, upstream → fork 47.1 s → 41.1 s 40.2 s → 33.3 s (17% faster)
pretix index, upstream → fork 9.85 s → 11.3 s (+15%) 8.36 s → 9.40 s (+13%) still slower
Alamofire index peak memory 809 MiB → 668 MiB 830 MiB → 953 MiB (+15%) now includes the pool change (#506); the old row predated it
n8n one-file sync 14.0 s → 4.72 s (3.0×) 13.1 s → 4.37 s (3.0×)
n8n warm explore 700 ms → 203 ms 600 ms → 177 ms
Tally 61 better, 6 same, 7 worse 65 better, 4 same, 5 worse

The worse five are pretix indexing (+13% Node, +20% Bun) and index memory on Alamofire (+15% Node) and pretix (+11% Node, +6% Bun). Each percentage is a median of two runs, so a difference within 3% is called no clear change, and the host was faster than on 10-14, so compare ratios between arms and not seconds across runs. Not measured: Windows and macOS, a second Bun or Node version.

Changes: README.md (Measured results, the four tables, the Node-or-Bun paragraph, the "Slower to index" bullet), assets/fork-vs-upstream-performance.svg (version tag bumped to ?v=4), the new docs/benchmarks/fork-vs-upstream-node-bun-2026-10-18.md and its heatmap. The 10-14 doc stays as a dated record. Every figure comes from render-charts.mjs and a script over the raw rows; the README tallies come from the chart renderer.

README rows checked: the "Measured results" intro, chart alt text, the full-index, sync, MCP and CLI tables, the Alamofire note and the Python-regression paragraph, the Node-or-Bun paragraph, and the "What it costs" bullets. Unchanged on purpose: the mid-size paragraph (its own 10-15 and 10-17 measurements), database size and edge-count bullets (the percentages round to the same values). readme-sync and check:agent-docs pass.


Summary by cubic

Re-runs the fork-vs-upstream benchmark on the current commits (fork 633fc82c against upstream 986de95c) and updates the README's Measured results with the new numbers, chart, and benchmark doc. The fork now indexes n8n 1.3× faster, syncs an edited file 3.0× faster, and is better in 65 of 74 comparisons (up from 61). The run includes the recent resolver pool, Kotlin, regex-cache, WAL-checkpoint, and import-link changes, so Alamofire's index memory is now higher (+15% on Node). The 2026-10-14 doc stays as a dated record.

Written for commit 0a96fc1. Summary will update on new commits.

View guided diff Turn on auto-fix

@coderabbitai

coderabbitai Bot commented Oct 18, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: Repository YAML (base), Organization UI (inherited)
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 4eed92b2-0767-456b-84e5-996b7b8a0038
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Low impact] The documentation changes appear safe to merge.

Summary

Refreshes the README and benchmark charts with the 2026-10-18 comparison of fork 633fc82c against upstream 986de95c.

  • The README now shows the October 18 benchmark results.

Reviews (1) · Last reviewed commit: "docs: re-run the fork-vs-upstream benchm..." · Reviewed by Greptile

@bompus
bompus merged commit 3581ae6 into fork/consolidated Oct 18, 2026
3 checks passed
@bompus
bompus deleted the docs/readme-bench-20261018 branch October 18, 2026 18:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant