Repository navigation
perf(browser): query six peers per lookup round - #211
Merged
Merged
Conversation
mickvandijke
force-pushed
the
fix/browser-media-streaming
branch
from
September 29, 2026 13:14
e85cc57 to
f86e56b
Compare
A third to a half of the WebRTC Direct endpoints that mainnet lookups dial time out after ten seconds. A lookup round takes its first answer and then waits up to the five-second grace period for the rest, so rounds of the Kademlia default of three usually paid most of that grace, and cold record reads took tens of seconds. Browser lookups now query six peers per round. On mainnet, cold record reads of a 612 MB video fell from a mean of 11.5 s (p90 29.2 s) to 6.1 s (p90 17.2 s). Ten was faster for isolated reads, but stalled playback after seeks, because read-ahead runs several lookups against the 64-connection pool. A shorter grace period did not help. The benchmark scripts and raw results are in docs/investigations/2026-09-29-browser-lookup-alpha; ADR-0004 records the decision. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Browser uploads use the same lookups to find and cross-check each record's close group, so the lookup alpha applies to them too. On mainnet, 8 MB uploads of four records reached the payment request in a mean of 120 s at alpha 6 against 155 s at alpha 3. No upload failed at either setting. The bench stops at the payment request, so nothing was paid for or stored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first runs used a 32-core workstation. The same builds, pages and runners on a 2-vCPU DigitalOcean droplet agree: at alpha 6 cold reads averaged 6.2 s instead of 8.0 s, and upload quoting 105 s instead of 161 s, with no failures. The extra lookup concurrency costs no CPU headroom on the small machine. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A round of six is more likely than a round of three to include an unreachable peer, so it waits out the grace period at least as often. The gain is more answers per round, and therefore fewer rounds, not less waiting; the ADR, the README and the constant's comment now say so. The write-up also overstated a few results. Six halved the cold-read mean only on the M3 Ultra; on the droplet the mean fell by a fifth and the p90 rose slightly. One alpha 3 read timed out on the M3 Ultra, so not every read succeeded. The runs used #209's read-ahead from before its bounds, so the six-versus-ten playback comparison should be repeated on the merged read-ahead. The README now maps each pkg-vN build to its alpha and grace, names the run each row comes from, and notes that the first run's dial records are unusable. The eleven playback logs behind streaming-summary.txt are committed next to the one that was, and stream-seek.mjs finds vite relative to its working directory, so SDK_DIR may be any path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mickvandijke
force-pushed
the
perf/browser-lookup-latency
branch
from
October 1, 2026 14:46
ea02182 to
a945e1d
Compare
The playback runs do not separate alpha 6 from 10. stream-a6-3 ended mid-stall 26 s after resuming, and the runs were watched for only 14 to 46 s after their seeks. The README and ADR claimed six never waited more than two seconds. They now say six is the conservative choice, keeping most of ten's cold-read gain with fewer connections, and that seek resume did not improve at any width. stream-summary.mjs took the first `video playing` anywhere in the log, so stream-a3-2's post-seek resume was reported as a 62.3 s first frame. It now reads the first frame only from before the seek, flags a wait still open when the log ends, reports how long playback was watched after resuming, and reads the gzipped logs directly. The upload table counted dial attempts the failed-endpoint cache refused at 0 ms as failed dials, and showed the upper middle sample as the median. The cold-read p50 and p75 used the same off-by-one index. The tables now show true medians and nearest-rank p75, count refusals apart, and the reproduce steps give each run's variants, rounds, uploads per page and seek target. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linear issue
Closes V2-1357
Summary
Browser DHT lookups were the remaining bottleneck for cold reads, media startup and seek resume:
Browser lookups now query six peers per round (
BROWSER_LOOKUP_ALPHA). The grace period, the dial timeout, the lookup termination rules and native lookups are unchanged. The ADR-0004 amendment records the decision.docs/investigations/2026-09-29-browser-lookup-alphahas the method, raw results and scripts.Risk tier
Compatibility
Semver impact
Test evidence
Mainnet, headless Chromium. Each run opened the 612 MB video
134e4537…a889in a fresh page, then read 12 cold records spread across it. Variants were interleaved across rounds.Why not ten: ten opens more connections, and during media playback with #209's read-ahead it stalled for 12 to 21 s after four of six seeks, because concurrent lookups contend for the 64-connection pool. Six waited at most 2 s after its three seeks, but those runs were watched for only 26 to 46 s after resuming and one ended mid-stall, so the playback runs don't separate six from ten. Six is the conservative choice: it keeps most of ten's cold-read gain with fewer connections. Its first frames (14 to 19 s) were the most consistent; at three, one run had not started playing by its seek. Seek resume did not improve at any width (mean 14.6 s at three, 15.1 s at six, 13.3 s at ten). These playback runs used #209's read-ahead from before its bounds (up to eight fetches in flight); the merged read-ahead keeps at most five, so the six-versus-ten playback comparison should be repeated on it.
Grace period: a 2 s grace did not help (mean 12.9 s at three peers, 8.2 s at six), so it is unchanged.
Upload quoting uses the same lookups to find and cross-check each record's close group. Each run uploaded 8 MB (four records) on mainnet in single-node payment mode and stopped at the payment request, so nothing was paid for or stored. Every upload reached the payment request with quotes for all four records.
About 23% faster, at the cost of about 40% more connections. Quoting is slow at either setting and needs its own investigation.
Second machine: the runs above used a 32-core Apple M3 Ultra. The same builds (identical
.wasmSHA-256) on a 2-vCPU DigitalOcean droplet (s-2vcpu-4gb-amd, ams3) agree, with no failures:The extra concurrency costs no CPU headroom on the small machine.
Other checks:
cargo fmt --all -- --checkand WASMcargo clippy -D warningspass.mainafter fix(browser): keep media playback streaming while records are discovered #209 merged: WASM bindings withbrowser-wasm,test-utils, 195 tests pass.New dependency
none
ADR
https://github.com/WithAutonomi/ant-client/blob/main/docs/adr/ADR-0004-direct-browser-read-client.md#browser-lookup-parallelism-2026-09-29
Mitigation / rollback
Revert this commit, or set
BROWSER_LOOKUP_ALPHAback toDEFAULT_ALPHA_VALUE(3).🤖 Generated with Claude Code