Repository navigation
Make the searched GitHub owners configurable and stop flooding the API - #64
Draft
Philipp0205 wants to merge 2 commits into
Draft
Philipp0205 wants to merge 2 commits into
Philipp0205 wants to merge 2 commits into
Conversation
Listing pull requests across every accessible repository searches one owner scope at a time, but a single rejected scope aborted the whole listing. GitHub answers GET /search/issues with HTTP 422 "The listed users and repositories cannot be searched" as soon as one qualifier names something the token may not search, which happens for organizations the token was never SSO-authorized for, for private repositories when the token lacks the 'repo' scope, and for repositories renamed or deleted after they were listed. The connection test then reported "FAILED Read pull requests" even though every other scope would have returned results. Run the per-scope searches through GitHubPullRequestSearch, which skips a scope GitHub refuses and only fails when no scope is searchable at all, in which case the message names the scopes and how to fix the token. Failures that are not scope rejections still propagate, so a server or network error is never mistaken for partial results. Organizations and collaborator repositories that cannot be listed no longer hide the user's own pull requests either, and the connection test reports the skipped scopes as a warning. Co-authored-by: Kurrle, Philipp <philipp.kurrle+ADVNTST@advantest.com>
Two problems showed up once a token could reach many repositories. The listing searched every reachable owner scope and then read every pull request any of those searches returned, one request each, before sorting and cutting the result down to the requested page. A token with sixty repositories therefore spent sixty search requests and six hundred detail requests on a single refresh and ran into "API rate limit exceeded". The search response already says when each pull request was last updated, so order and page the results first and read only that page in full: the same case now costs three searches and a hundred detail requests. Owner scopes are also batched into shared queries, since GitHub combines repeated qualifiers of the same kind with OR. A batch GitHub refuses is retried scope by scope so that only the truly unsearchable scope is dropped. Which pull requests appear was also decided entirely by whichever account the token belongs to, with no way to narrow it down. Add a "Search only" preference listing the owners and repositories to search; when it is set the plugin searches exactly those and skips discovering organizations and collaborator repositories altogether. A bare name covers everything a user or organization owns, and owner/name covers a single repository. Rate limit responses are now recognized as such: they abort the listing with an explanation and a pointer to the preference, instead of the raw JSON error, and the remaining scopes are not retried into the same wall. Co-authored-by: Kurrle, Philipp <philipp.kurrle+ADVNTST@advantest.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problems
Three failures in the connection test, all from the same place: listing pull requests across every repository a token can reach.
A scope the token may not search aborted the whole listing.
GitHub answers
GET /search/issueswith HTTP 422 as soon as one qualifier names something the token may not search, which happens for organizations the token was never SSO-authorized for, private repositories when the token lacks thereposcope, and repositories renamed since they were listed. One rejected scope threw out of the loop and killed the listing.The API rate limit ran out.
The listing sent one search per owner scope and then read every pull request any of those searches returned, one request each, before sorting and cutting the result down to the requested page. A token reaching 60 repositories spent 660 requests on a single refresh.
No control over which account's pull requests appear. Which pull requests show up is decided entirely by the account the token belongs to, and there was no way to narrow that down. A token issued by the
philipp-kurrle_ADVNTSTaccount lists that account's world, notPhilipp0205's.Changes
Search onlypreference. Lists the owners and repositories to search. When set, the plugin searches exactly those and skips discovering organizations and collaborator repositories entirely. A bare name covers everything a user or organization owns;owner/namecovers a single repository. Verified against the live API thatuser:matches organizations too, so one entry form works for both.updated_at, so results from all scopes are merged, ordered and cut to the requested page first, and only that page is read in full.GitHubPullRequestSearchrecords them, and the connection test reports them as a warning naming the scope. The listing fails only when no scope is searchable, and then the message names the scopes and the token requirement./user/orgsand/user/reposfailures degrade to an empty scope list with a log warning, so a token withoutread:orgno longer hides the user's own pull requests.Testing
mvn clean verify— 165 tests pass, 34 of them new.GitHubConnectionTestdrives the realGitHubClientagainst a local stub API that reproduces the reported 422 and 403 bodies and counts every request the client sends.Requests to list 100 pull requests for a token reaching 60 repositories, each search matching 600 pull requests, measured against the stub:
With
Search onlyset to a single owner, the client sends exactly one search request and never calls/user,/user/orgsor/user/repos.Connection test report for a refused scope, before and after:
And for the rate limit: