Skip to content

Make the searched GitHub owners configurable and stop flooding the API - #64

Draft
Philipp0205 wants to merge 2 commits into
mainfrom
cursor/skip-unsearchable-github-scopes-924b
Draft

Philipp0205 wants to merge 2 commits into
mainfrom
cursor/skip-unsearchable-github-scopes-924b

Conversation

@Philipp0205

Copy link
Copy Markdown
Owner

Problems

Three failures in the connection test, all from the same place: listing pull requests across every repository a token can reach.

A scope the token may not search aborted the whole listing.

OK       Authentication
         Authenticated as Philipp0205
FAILED   Read pull requests
         GitHub API request failed: HTTP 422 - {"message":"Validation Failed","errors":[{"message":"The listed users and repositories cannot be searched either because the resources do not exist or you do not have permission to view them.",...}],"status":"422"}

GitHub answers GET /search/issues with HTTP 422 as soon as one qualifier names something the token may not search, which happens for organizations the token was never SSO-authorized for, private repositories when the token lacks the repo scope, and repositories renamed since they were listed. One rejected scope threw out of the loop and killed the listing.

The API rate limit ran out.

OK       Authentication
         Authenticated as philipp-kurrle_ADVNTST
FAILED   Read pull requests
         GitHub API request failed: HTTP 403 - {"message":"API rate limit exceeded for user ID 201185819. ...","status":"403"}

The listing sent one search per owner scope and then read every pull request any of those searches returned, one request each, before sorting and cutting the result down to the requested page. A token reaching 60 repositories spent 660 requests on a single refresh.

No control over which account's pull requests appear. Which pull requests show up is decided entirely by the account the token belongs to, and there was no way to narrow that down. A token issued by the philipp-kurrle_ADVNTST account lists that account's world, not Philipp0205's.

Changes

  • Search only preference. Lists the owners and repositories to search. When set, the plugin searches exactly those and skips discovering organizations and collaborator repositories entirely. A bare name covers everything a user or organization owns; owner/name covers a single repository. Verified against the live API that user: matches organizations too, so one entry form works for both.
  • Page the search results before reading them. The search response already reports each pull request's updated_at, so results from all scopes are merged, ordered and cut to the requested page first, and only that page is read in full.
  • Batch owner scopes into shared queries. GitHub combines repeated qualifiers of the same kind with OR, so up to 20 scopes go into one request. A batch GitHub refuses is retried scope by scope, so only the genuinely unsearchable scope is dropped.
  • Skip refused scopes instead of failing. GitHubPullRequestSearch records them, and the connection test reports them as a warning naming the scope. The listing fails only when no scope is searchable, and then the message names the scopes and the token requirement.
  • Recognize rate limiting. A throttled response aborts the listing with an explanation and a pointer to the new preference rather than the raw JSON, and the remaining scopes are not retried into the same wall. Failures that are neither refusals nor throttling (5xx, network errors) still propagate immediately, so an outage is never silently turned into a partial list.
  • /user/orgs and /user/repos failures degrade to an empty scope list with a log warning, so a token without read:org no longer hides the user's own pull requests.

Testing

mvn clean verify — 165 tests pass, 34 of them new. GitHubConnectionTest drives the real GitHubClient against a local stub API that reproduces the reported 422 and 403 bodies and counts every request the client sends.

Requests to list 100 pull requests for a token reaching 60 repositories, each search matching 600 pull requests, measured against the stub:

searches detail reads total
Before 60 600 660
After 3 100 103

With Search only set to a single owner, the client sends exactly one search request and never calls /user, /user/orgs or /user/repos.

Connection test report for a refused scope, before and after:

OK       Authentication
         Authenticated as Philipp0205
FAILED   Read pull requests
         GitHub API request failed: HTTP 422 - {"message":"Validation Failed","errors":[{"message":"The listed users and repositories cannot be searched either because the resources do not exist or you do not have permission to view them.","resource":"Search","field":"q","code":"invalid"}],"status":"422"}
OK       Authentication
         Authenticated as philipp-kurrle_ADVNTST
OK       Read pull requests
         Returned 1 pull request(s)
WARNING  Search scopes
         GitHub refused to search org:acme.
         Use a classic personal access token with the 'repo' and 'read:org' scopes, and authorize it for every organization that enforces SAML single sign-on. Fine-grained tokens can only search the repositories they were granted.

And for the rate limit:

OK       Authentication
         Authenticated as philipp-kurrle_ADVNTST
FAILED   Read pull requests
         The GitHub API rate limit for this access token is exhausted. Wait for the limit to reset, or search fewer repositories by listing the wanted owners on the pull request preference page. GitHub API request failed: HTTP 403 - {"message":"API rate limit exceeded for user ID 201185819.","status":"403"}
Open in Web Open in Cursor 

Philipp0205 and others added 2 commits September 7, 2026 12:06
Listing pull requests across every accessible repository searches one
owner scope at a time, but a single rejected scope aborted the whole
listing. GitHub answers GET /search/issues with HTTP 422 "The listed
users and repositories cannot be searched" as soon as one qualifier
names something the token may not search, which happens for
organizations the token was never SSO-authorized for, for private
repositories when the token lacks the 'repo' scope, and for
repositories renamed or deleted after they were listed. The connection
test then reported "FAILED Read pull requests" even though every other
scope would have returned results.

Run the per-scope searches through GitHubPullRequestSearch, which skips
a scope GitHub refuses and only fails when no scope is searchable at
all, in which case the message names the scopes and how to fix the
token. Failures that are not scope rejections still propagate, so a
server or network error is never mistaken for partial results.
Organizations and collaborator repositories that cannot be listed no
longer hide the user's own pull requests either, and the connection
test reports the skipped scopes as a warning.

Co-authored-by: Kurrle, Philipp <philipp.kurrle+ADVNTST@advantest.com>
Two problems showed up once a token could reach many repositories.

The listing searched every reachable owner scope and then read every
pull request any of those searches returned, one request each, before
sorting and cutting the result down to the requested page. A token with
sixty repositories therefore spent sixty search requests and six hundred
detail requests on a single refresh and ran into "API rate limit
exceeded". The search response already says when each pull request was
last updated, so order and page the results first and read only that
page in full: the same case now costs three searches and a hundred
detail requests. Owner scopes are also batched into shared queries,
since GitHub combines repeated qualifiers of the same kind with OR. A
batch GitHub refuses is retried scope by scope so that only the truly
unsearchable scope is dropped.

Which pull requests appear was also decided entirely by whichever
account the token belongs to, with no way to narrow it down. Add a
"Search only" preference listing the owners and repositories to search;
when it is set the plugin searches exactly those and skips discovering
organizations and collaborator repositories altogether. A bare name
covers everything a user or organization owns, and owner/name covers a
single repository.

Rate limit responses are now recognized as such: they abort the listing
with an explanation and a pointer to the preference, instead of the raw
JSON error, and the remaining scopes are not retried into the same wall.

Co-authored-by: Kurrle, Philipp <philipp.kurrle+ADVNTST@advantest.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant