Repository navigation
fix(esplora): stop paging when the server does not advance the cursor - #2361
Merged
evanlinjin merged 1 commit intoOct 18, 2026
Merged
evanlinjin merged 1 commit into
evanlinjin merged 1 commit into
Conversation
3 of 6 tasks
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #2361 +/- ##
==========================================
+ Coverage 79.53% 79.67% +0.14%
==========================================
Files 31 31
Lines 6141 6149 +8
Branches 290 292 +2
==========================================
+ Hits 4884 4899 +15
+ Misses 1182 1178 -4
+ Partials 75 72 -3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
evanlinjin
requested changes
Oct 18, 2026
Comment on lines
+348
to
+351
| if next_last_seen == last_seen { | ||
| return Err(Box::new(esplora_client::Error::InvalidResponse)); | ||
| } | ||
| last_seen = next_last_seen; |
Member
There was a problem hiding this comment.
This only catches a cursor that repeats straight away. A server that cycles (A → B → A, e.g. one that maps the cursor to a wrong page) still loops forever. Tracking every cursor seen catches any finite cycle for about the same amount of code:
let mut seen = HashSet::new();
// ...
let cursor = txs.last().map(|tx| tx.txid);
// ...
if !seen.insert(cursor) { return Err(...) }
last_seen = cursor;Let's do this instead as it's more thorough.
The per-script-pubkey pagination loop paged forward by asking for the transactions following the last txid of the previous page, but never checked that the cursor moved. A server that answers every request with the same full page of 25 transactions kept the cursor in place, so the loop re-requested that page forever while appending to an unbounded `Vec`, and `sync`/`full_scan` never returned. Track every cursor paged from, and return `InvalidResponse` if a full page ends on one we have already seen. This catches a cursor that repeats straight away as well as one that cycles (`A -> B -> A`). Erroring rather than breaking avoids silently handing the caller a truncated transaction history, which is how `chain_update` already treats a server it cannot reconcile with. Closes bitcoindevkit#2339
evanlinjin
force-pushed
the
fix/esplora-pagination-non-advancing
branch
from
October 18, 2026 09:47
07c0115 to
19ba4e3
Compare
evanlinjin
approved these changes
Oct 18, 2026
evanlinjin
left a comment
Member
There was a problem hiding this comment.
ACK 19ba4e3
I had to update the PR description as it was too wordy.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #2339.
bdk_esplorapages each script pubkey's history by passing the previous page's last txid as the cursor, and stops only on a page of fewer than 25 txs. A server that ignores the cursor returns the same full page forever, sosync/full_scannever returns and memory grows without bound.The loop now records every cursor it pages from and returns
InvalidResponsewhen a full page ends on one it has already seen. This catches both a repeated page and a cycle (A -> B -> A). Applied to bothasync_ext.rsandblocking_ext.rs.Notes to the reviewers
breaking, so thatsyncnever reports a truncated history as success.TcpListenerstub intests/commonthat always returns the same page, sinceTestEnvcan't fake a misbehaving server.tokio'stimedev-dep feature is for the async test's timeout.Changelog notice
bdk_esplora's pagination loop hanging when a chain source returns a page that does not advance thelast_seencursor.Checklists
All Submissions:
cargo fmtandcargo clippybefore committingBugfixes: