v0.8.42: knowledgebase queueing improvemnets, file search time fixes, outbox fixes - #7945
Merged
Merged
Conversation
waleedlatif1
commented
Sep 17, 2026
waleedlatif1
commented
Collaborator
- feat(library): AI Agent Marketplace vs Building From Scratch: Which Should You Choose? (feat(library): AI Agent Marketplace vs Building From Scratch: Which Should You Choose? #7929)
- docs(library): update n8n-alternatives (docs(library): update n8n-alternatives #7930)
- fix(coda): enable inline token setup and clarify search guide (fix(coda): enable inline token setup and clarify search guide #7932)
- feat(library): How do AI-native workflow automation platforms compare to traditional automation tools like Zapier? (feat(library): How do AI-native workflow automation platforms compare to traditional automation tools like Zapier? #7928)
- fix(ci): stop the CodeQL cron cancelling the main merge scan (fix(ci): stop the CodeQL cron cancelling the main merge scan #7933)
- feat(library): AI-Native Workflow Automation vs Traditional Automation Platforms: Sim, Zapier, Make, and n8n (feat(library): AI-Native Workflow Automation vs Traditional Automation Platforms: Sim, Zapier, Make, and n8n #7936)
- fix(knowledge): give document processing per-tenant queue lanes (fix(knowledge): give document processing per-tenant queue lanes #7937)
- improvement(provenance): enforce tracked durable reads (improvement(provenance): enforce tracked durable reads #7938)
- fix(search): resume Google crawls around unavailable accounts (fix(search): resume Google crawls around unavailable accounts #7934)
- feat(atlassian): resolve dynamic scopes and index Confluence attachments (feat(atlassian): resolve dynamic scopes and index Confluence attachments #7935)
- fix(oauth): refresh access tokens before expiry (fix(oauth): refresh access tokens before expiry #7939)
- fix(file-search): bound dispatcher claims and skip locked rows (fix(file-search): bound dispatcher claims and skip locked rows #7940)
- fix(outbox): avoid backlog scans during event discovery (fix(outbox): avoid backlog scans during event discovery #7942)
- fix(file-search): store workspace file revisions at millisecond precision (fix(file-search): store workspace file revisions at millisecond precision #7941)
- fix(knowledge): write external group membership as a diff (fix(knowledge): write external group membership as a diff #7943)
…hould You Choose? (#7929) Co-authored-by: Sim Pi Agent <pi@sim.ai>
* fix(coda): enable inline token setup and clarify search guide * docs(coda): clarify workspace access scope
… to traditional automation tools like Zapier? (#7928) Co-authored-by: Sim Pi Agent <pi@sim.ai>
The workflow-level `cancel-in-progress: true` applied to every trigger, but push and schedule both resolve to `refs/heads/main` and therefore share the `codeql-refs/heads/main` concurrency group. A merge landing shortly before the daily cron had its scan cancelled mid-extraction, leaving main with a red status rollup for a commit the cron then scanned clean. Scoping the cancel to `pull_request` keeps the superseding behavior where it belongs -- each PR is its own group via `refs/pull/N/merge` -- and lets non-PR events queue behind an in-progress run instead of killing it.
Document processing ran through one Trigger.dev queue with a global concurrency limit and no concurrency key, so a single connector backfill could hold every slot and leave every other tenant's uploads queued behind it for hours. Dispatch now names a lane — interactive for work a person is waiting on, backfill for connector-driven ingestion — and keys each run by the entity that owns the knowledge base, so the limit applies per tenant copy of the queue rather than fleet-wide. - Key on the owning entity, not the workspace: an organization-scoped knowledge base carries a null workspace id and would otherwise share one bucket with every other such tenant. Owned scopes reuse resourceScopeKey so tenant identity keeps one spelling. - Stamp the lane on the payload so quota and capacity continuations resume in the lane they were admitted against instead of promoting themselves out of the backfill ceiling. - Parse an absent or unrecognized lane as backfill rather than throwing, so payloads written before the lanes existed and payloads stamped by a newer version mid-rollout cannot burn a run's retry budget. - Keep interactive work on the pre-existing queue name; a queue the running worker has not registered parks its runs in PENDING_VERSION, and the app deploys separately from the worker. - Stop absorbing a rejected backfill chunk in-process: those documents are reported failed and reclaimed by the stuck-document sweep instead of outlasting the connector lease they run under.
* improvement(provenance): enforce tracked durable reads * fix(provenance): mark legacy attachment replay fixtures
* fix(search): resume Google crawls around unavailable accounts * fix(search): harden crawl completion and recovery * improvement(search): generalize connector partition progress * fix(search): clear unverifiable placeholder permissions
…nts (#7935) * feat(atlassian): resolve dynamic scopes and index Confluence attachments * fix(atlassian): reject mixed All scopes and run permission tests in CI * fix(ci): isolate Confluence migration test database
…sion (#7941) * fix(file-search): store workspace file revisions at millisecond precision The content revision round-trips through JavaScript Date and JSON, both of which truncate to milliseconds, while now() stores microseconds. A sub-millisecond revision stops comparing equal to its own round-trip, so every SQL equality keyed on it matches zero rows and the file is never claimed, indexed, or cleaned up. Truncate at the column default and enforce it with a BEFORE trigger for the writers a default cannot reach, then retire the rows minted before the invariant existed in a batched script migration. * fix(file-search): normalize revisions promoted by metadata-only writes A chat upload is materialized into a workspace file by setting context alone, so the revision is never written and the search-index trigger would key the promoted file to a value nothing can claim. Fire the normalizer on every update; `UPDATE OF` is evaluated against the statement's target columns, so this does not demote provenance on a metadata write. Accept the shared integration database in the repair suite the way the sibling suites do, and record the new script migration in the registry order assertions. * chore(db): drop the 0357 migration ahead of renumbering * chore(db): renumber the content revision migration to 0358 Staging took 0357 for the outbox autovacuum migration. The SQL is byte for byte the previously reviewed file; only the number and the references to it changed.
* fix(knowledge): write external group membership as a diff Every directory sync deleted a group's entire membership and reinserted it, whether or not anything had changed. That took the membership table to roughly 55M lifetime inserts and 55M deletes against ~127k live rows — about 430x write amplification — holding it near 91% dead tuples across 2,447 autovacuum cycles, an order of magnitude more churn than any comparable table. The vacuum load that generates competes for the same I/O as every other query on the instance. Membership is now written as a difference: read the group's current tokens, delete only those the enumeration no longer lists, insert only those it newly lists. An unchanged group writes nothing. The group row is locked before that read. The two callers fence on different leases — the directory lease and the connector sync lease — so neither excludes the other, and on the directory path the group upsert commits in a separate transaction, so its row lock is already released. Without the lock, the removal set is computed against a snapshot a concurrent pass may have moved past, and a subject that pass inserted would survive an enumeration that never observed it: membership retained rather than revoked. Locking also closes the narrower window the blind delete already had between its delete and its commit. `created_at` on a member row now means first-observed rather than last-observed. No reader projects it, and the group's `lastSyncedAt` remains the freshness signal and is still written every pass. Also corrects the processing-queue TSDoc, which argued the per-tenant lane limits were sized rather than inherited and cited a concurrency figure derived from an unsound measurement. * test(knowledge): assert the group lock precedes the membership read Presence of the lock was the only thing asserted, so moving it below the read would have kept the test green while reopening the stale-snapshot race the lock exists to close. The assertion is now the relative call order of the lock and the member-table read.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
1053
to
+1056
| if ( | ||
| row?.accessToken && | ||
| row.accessTokenExpiresAt && | ||
| row.accessTokenExpiresAt > new Date() | ||
| !isOAuthAccessTokenExpiring(row.accessTokenExpiresAt, providerId) |
Contributor
There was a problem hiding this comment.
When concurrent callers encounter a token inside the new five-minute refresh window, only one becomes the refresh leader. Other callers stop waiting after three seconds, and this check rejects the stored token even though it remains valid and the leader may still be refreshing it. Those callers therefore receive no credential and fail their provider requests. Extend the non-Slack wait to cover the refresh operation, or allow the valid stored token while a refresh is in progress.
| @@ -267,6 +285,7 @@ export async function getServiceAccountToken( | |||
| hasSubject: Boolean(impersonateEmail), | |||
| scopes: filteredScopes.join(' '), | |||
Contributor
There was a problem hiding this comment.
The service-account token path now writes the impersonated user's email to info-level logs on every ordinary token mint. This unnecessarily exposes customer identities and increases the amount of sensitive data retained in application logs. Keep the existing hasSubject field, but omit the raw subject as the selector-safe branch already does.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.