feat(storage): answer get-or-closer requests (ADR-0020)

[mickvandijke]

ADR-0020: a chunk GET returns the chunk, or closer peers, in one request, as Kademlia's FIND_VALUE does. This PR adds the ADR and the node side.

Why. On production, a native chunk read spends 82% of its time finding a holder, because the closest-peers lookup runs to convergence before the GETs start. Asking each queried peer for the chunk as well as for closer peers cuts native p50 from 5.95 s to about 1 s; see the investigation.

Node.

• A node that holds the chunk answers GetOrCloserRequest with Found.
• Otherwise it answers Closer, with saorsa-core's encoding of the FIND_NODE answer it would give the authenticated requester.
• A node with storage enabled announces get-or-closer/1 in its user agent. One with storage off never answers chunk requests, so it does not announce it.
• Browsers (new). Over WebRTC Direct, HELLO lists get_or_closer when storage is enabled, and a browser sends the same request inside chunk_protocol:
  • A held chunk comes back Found, as over QUIC.
  • A missing one gets the node's browser find_node answer. Its peers carry the WebRTC Direct endpoints and owner-signed records a browser dials. Native Closer never crosses the browser boundary: it holds QUIC addresses a browser cannot dial.
  • Only a node that holds the chunk schedules and bounds the request as a GET. A step to any other node is answered as find_node is, so a browser's lookup step never waits behind a transfer to the same browser. A node gives each source address one bulk slot and 16 MiB in flight.
  • No saorsa-transport change: the request travels in chunk_protocol, and both answers are existing browser response bodies.
• Get-or-closer gets its own rows in the chunk traffic summary (group 4), and each answer is logged as get_or_closer_rpc.

Compatibility by design.

• Old client, new node: the request is never sent; nothing changes.
• New client, old node: the client sees no token, so it sends FIND_NODE and a GET in the same step instead. That works with every deployed node at the same speed.
• Lookup finds no holder: the client falls back to today's read path, so the absence rule is unchanged.
• Browsers: an older node's HELLO does not list get_or_closer, so a new browser never sends it the request, which it could not decode. An old browser never sends it.

Stack, merge in this order:

1. feat(dht): expose lookup steps for caller-driven lookups saorsa-core#173
2. feat(chunk): add a get-or-closer request ant-protocol#44
3. this PR
4. feat(read): read chunks through a get-or-closer lookup (ADR-0020) ant-client#221

This PR pins saorsa-core and ant-protocol at those branches by git rev.

Linear issue

Closes V2-1358

Risk tier

• T0 — docs / tooling / CI / pure UX-output. Repo CI only.
• T1 — client-only, no network-facing behavior change. CI + prod compat smoke.
• T2 — node/client logic with behavioral surface, no protocol/format/economics change. Dev testnet + ADR.
• T3 — protocol / storage format / payments / routing. T2 evidence + adversarial testing.

Compatibility

• Wire: the node answers the new GetOrCloserRequest, and its user agent gains the token get-or-closer/1. The node/ prefix and the migration token are kept. Every existing message is unchanged.
• Browser wire: HELLO gains the get_or_closer capability, and chunk_protocol admits GetOrCloserRequest. Its answers are existing response bodies.
• Storage: none.
• API: none outside the crate. ChunkRequestContext::with_requester and migration_signal::user_agent(signal, serves_chunks) are crate-private.

Semver impact

• breaking
• feature
• fix

Test evidence

• cargo test --lib --features test-utils: 1,269 passed. New tests:

  • Found for a held chunk
  • an error rather than an empty answer without a routing table
  • the user agent announces the token only when serving chunks, and keeps the node/ prefix
  • HELLO lists get_or_closer only with storage
  • the browser request is scheduled as a GET only where the chunk is held
  • only Found crosses to a browser; Closer is refused, and errors are redacted

• cargo test --all-features --test webrtc_direct_devnet: 7 passed, 1 ignored as before. New: get_or_closer_returns_the_chunk_or_the_find_node_answer. Over a real WebRTC Direct session, HELLO lists the capability, a held chunk comes back Found, and a missing one gets nodes with a signed record for every peer.

• Real browser. ant-client's Chromium suite (ant-core/browser-tests), run against seven nodes built from this branch: 2 of 2 passed. With a temporary trace on the download, the first record took the ordinary read, as a cold client does. The other three records came through get-or-closer: 3 Found, 2 nodes answers, no plain GET.

• cargo test --test e2e --features test-utils test_get_or_closer_returns_chunk_or_closer_peers: passed. On a local test network, the holder answers Found, and a non-holder answers Closer whose peers the client decodes as a FIND_NODE answer, without the requester.

• cargo clippy --all-features -- -D clippy::panic -D clippy::unwrap_used -D clippy::expect_used -D warnings: clean.

• cargo +1.99.0 clippy --all-targets --all-features -- -D warnings, as CI runs it: clean.

• Live. A client from the ant-client PR downloaded a 612 MB production file in 56 s, against 242 s for the current client, with identical bytes. All 150 chunks came through the new lookup, on the route for nodes without the token.

• Browser reads on the dev testnet (2026-10-12): the kad fleet's nodes were swapped in place to this branch, keeping their data, and browser downloads were A/B'd. The new browser read did not help: fra1 is about a fifth slower at p50, and syd1 is unchanged. See ant-client#221 for the table.

• Dev testnet comparison against main, matched (2026-10-09, native clients). Two 60-node fleets on the same layout:

  • 30% symmetric NAT across 9 regions.
  • 4 uploaders (50 MiB) and 2 verified downloaders per fleet, over a 60-minute window.
  • Both fleets on the rc-2026.10.1 release baseline. The clients differ only by the ADR-0020 commits.
  • Downloads paced to one every 40 s per downloader, so both fleets serve the same reads.

  	main	this stack	delta
  download p50 / p95 / max	20.5 / 28.6 / 35.5 s	9.7 / 15.5 / 20.4 s	−53% / −46% / −43%
  downloads ok / failed	176 / 0	176 / 0	same load
  read bytes nodes sent per byte downloaded	1.35	1.73	duplicate copies
  no-merkle uploads ok, p50 / p95	211, 32.9 / 45.6 s	211, 33.2 / 45.1 s	identical
  merkle uploads ok, p50 / p95	130, 54.1 / 66.2 s	112, 62.9 / 76.1 s	−14%, +16% / +15%
  node CPU mean	27.2%	26.3%
  node restarts	0	0

  • The routing works on a real network: since the redeploy, this stack's nodes served only get-or-closer requests (127 before T0), and main's only plain GETs.
  • The merkle upload gap is not attributed to this change:
    • It does not touch the merkle payment, quote or PUT path.
    • Non-merkle uploads, which run the same lookups and PUTs, are identical.
    • The kad fleet's chain (Anvil) droplet showed 2.8× the CPU steal of main's, and merkle uploads lean on it hardest.
    • A run with the code swapped between the two fleets' droplets would settle it.
  • An earlier, unmatched run is superseded. Its clients linked different saorsa-transport versions, and its downloaders ran closed-loop, so the faster fleet carried 1.85× the reads.
  • Full report: ant-testnet/state/comparisons/v2-1358-kad-vs-main-matched-60m-1009/README.md.

New dependency

none

ADR

ADR-0020: Kademlia-style chunk GET: return the chunk or closer peers, added in this PR.

Mitigation / rollback

The node is passive until clients send the request. To stop clients using it, drop the user-agent token; they then fall back to FIND_NODE plus GET. Otherwise revert.

🤖 Generated with Claude Code

[dirvine]

Code approval at the reviewed head

No introduced blocking runtime defect found in this PR. Production responses use the size-capped encoder; authenticated requester identity is passed into the handler. Decoder use here is test-only.

Stack coordination remains required: saorsa-core#173 has a P2 decoder-size blocker. Refresh the core patch pin at Cargo.toml:240 after that fix and rerun integration tests; propagate the resulting node head into the client. This code approval is not approval to ship the current linked dependency set or a waiver of merge/release gates.

Reviewed commit: 79385bfddc04a26b5c0747b3355364c12b9a127b.

Review method: six substantive seats across the linked set, including a completed z-ai/glm-5.2 API review recovered from a worker that timed out after saving its response; coordinator source checks and local execution. Initial functional/security/test seats recommended approval. The coordinator probe, independent adjudicator, API seat and replacement concurrency/release seats corroborate the missing decoder bound. This is not unanimous approval. The API seat recommended client approval with concerns; the coordinator withholds it because the exact client pin still exposes the defect.

Local verification (macOS, rustc 1.96.0): core closer-peers 3 passed and DHT-manager regression 143 passed; protocol all-features 108 passed; node get-or-closer unit tests 3 passed and real-transport E2E 1 passed; client kad_read unit tests 10 passed and real-transport E2E 1 passed. Formatting passed in all four repositories. No production/testnet deployment or merge performed. Draft git pins were verified with cargo metadata --locked; this is not published dependency closure. ADR-0020 remains Proposed.

[mickvandijke]

Thanks.

• ad88647 moves the saorsa-core pin to fd37775, the bounded closer-peers decoder. The 3 get-or-closer unit tests and the real-transport e2e pass, and CI was green (21/21).
• 900ff66 updates ADR-0020:
  • the 64 KiB bound on closer peers;
  • the client dropping a peer whose content fails verification (ant-client#221, 7d9a248);
  • that every queried peer can now send content, listed as a trade-off;
  • the validation list.