fix: skip github-deleted records in shadow diff (CM-1473)

[mbani01]

Problem

The daily shadow-diff Temporal workflow compares Nango records against our shadow GitHub connector records and persists missing_in_shadow mismatches. Nango never retracts deletes, so records that were deleted on GitHub after Nango captured them (issue/PR comments, pull requests + their timeline events, review comments, forks) surface as steady false-positive missing_in_shadow noise — around 200-700/day on busy repos (e.g. openclaw/openclaw bot comments, NousResearch/hermes-agent deleted-account PRs and children), all confirmed via GraphQL node(id:) returning null/NOT_FOUND.

An analogous confirm-before-flag filter already exists for force-pushed commits (forcePushedCommits.ts), but commit sourceIds are SHAs, not GraphQL node ids, so it can't cover these record kinds.

Fix

Adds services/apps/connectors_worker/src/deletedRecords.ts, mirroring forcePushedCommits.ts's structure:

• hasDeletedRecordCandidates / dropConfirmedDeletedRecords apply to any sync other than pull-request-commits, since every other record kind's sourceId is a GitHub GraphQL node id (issue/PR comments, pull requests, reviews, review comments, forks, discussions). Synthetic composite ids generated for PR/issue timeline events (gen-..., not real node ids) are excluded from lookup.
• Candidates are looked up in batches of up to 100 via a single nodes(ids: [...]) GraphQL query, instead of per-record calls.
• A response with top-level data and per-id null entries (with NOT_FOUND in errors) is treated as a valid confirmation that the node is deleted — dropped from the mismatch list. A response with no top-level data (request error, timeout, forbidden, etc.) is treated as unconfirmed and the record is kept as missing_in_shadow (fail open).
• A 60s time budget (matching the force-push filter) bounds total confirmation work per unit; once exhausted, all remaining unchecked candidates are kept.
• Wired into runShadowDiffForChannel right after the existing force-push filter, reusing the same GitHub confirmation HTTP client/token pool. Logs how many records were confirmed deleted vs kept unconfirmed.

Validation

• New Vitest unit tests in services/apps/connectors_worker/src/deletedRecords.test.ts (written and run locally, not committed per repo convention): confirmed-deleted dropped, alive kept, request-error kept, no-data-with-NOT_FOUND-errors treated as confirmation, commit sync untouched, synthetic gen- ids skipped, budget-exhaustion keeps the remainder unconfirmed. All 10 pass, plus the existing shadowDiffActivities.test.ts suite (11 tests) still passes.
• pnpm lint:fix + pnpm format run; npx oxfmt --check and npx oxlint --deny-warnings pass clean on the changed files.
• npx tsc -b services/apps/connectors_worker/tsconfig.json (forced full rebuild) passes with no errors. Root pnpm tsc-check still fails on the 2 pre-existing unrelated errors (resolveCdpEmails.test.ts TS2550, backend/src/database/models/index.ts TS2339).

🤖 Generated with Claude Code

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[cursor]

PR Summary

Medium Risk
Changes which shadow-diff mismatches are persisted and adds GitHub GraphQL calls in the daily workflow; incorrect confirmation logic could hide real drift, though unconfirmed cases are kept as mismatches.

Overview
Adds a GitHub deletion confirmation step to the shadow-diff pipeline so missing_in_shadow mismatches caused by records deleted on GitHub (but still present in Nango) are dropped instead of persisted as noise.

New deletedRecords helpers mirror the existing force-push filter: eligible missing_in_shadow rows (real GraphQL node sourceIds, not gen- synthetics or pull-request-commits) are checked in batches via nodes(ids: [...]). Only null nodes with per-id NOT_FOUND are removed; API failures, ambiguous nulls, and budget/timeouts fail open and keep the mismatch.

runShadowDiffForChannel now shares the same GitHub confirmation HTTP client for force-push and deleted-record checks, with logging for confirmed-deleted vs kept-unconfirmed counts. Vitest coverage exercises batching, error handling, sync/id exclusions, and the 60s budget.

^{Reviewed by Cursor Bugbot for commit 18a850c. Bugbot is set up for automated code reviews on this repo. Configure here.}

[Copilot]

Copilot review overview

🟡 Changes recommended

Partial GraphQL errors can suppress valid mismatches, synthetic timeline records remain unresolved, and tests are uncommitted.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 · 1 · 1

Open (3)

• Preserve mismatches for partial GraphQL errors beyond NOT_FOUND · New
• Confirm deleted parents before dropping generated child records · New
• Add automated coverage for deletion-confirmation filtering · New

What changed in this PR

Adds GitHub deletion confirmation to reduce false-positive shadow-diff mismatches.

Changes:

• Batch-checks missing records through GitHub GraphQL.
• Integrates deletion filtering with existing force-push confirmation.

File	Description
deletedRecords.ts	Implements deletion confirmation and filtering.
shadowDiffActivities.ts	Applies filtering during shadow diff processing.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 3b493a7. Configure here.}

[Copilot]

Copilot review overview

🔵 Needs a closer look

The described deletion-filter regression tests must be committed before approval.

Review effort: Balanced
Findings: 1

Open (1)

• Add automated coverage for deletion-confirmation filtering

Resolved since last review (2)

• Preserve mismatches for partial GraphQL errors beyond NOT_FOUND
• Confirm deleted parents before dropping generated child records

[Copilot]

Copilot review overview

🟢 Approval recommended

The implementation is scoped, fail-open, and adequately tested, with previous substantive feedback addressed.

Review effort: Balanced
Findings: 1

Open (1)

• Add automated coverage for deletion-confirmation filtering

[Copilot]

Copilot review overview

🟡 Changes recommended

Per-unit 60-second budgets can collectively exceed the activity’s five-minute timeout before results are persisted.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 · 1

Open (2)

• Use a shared channel-level deadline for confirmation checks · New
• Add automated coverage for deletion-confirmation filtering