fix(db): retry migration lock timeouts within a time budget instead of eight attempts

[waleedlatif1]

Summary

• migrate.ts gave up after 8 lock timeouts (~2.5 min of 5 s waits). On a table that long-running write transactions hold all the time, an ACCESS EXCLUSIVE DDL lock only becomes available in short gaps, and eight tries often miss every gap
• Lock timeouts (55P03) are now retried for up to 20 min, measured from the first attempt, with the same 2–30 s jittered backoff. Each attempt still uses the same 5 s lock_timeout, so a queued DDL lock never blocks traffic on the table for longer than 5 s
• 20 min is less than the 30 min advisory-lock acquire deadline, so a second runner waiting on the lock sees this one finish, pass or fail, before its own deadline expires
• Any error other than 55P03 still fails right away
• The retry loop is moved into scripts/lock-timeout-retry.ts so it can be tested, and the stranded runMigrationsWithRetry TSDoc is moved back onto its function

Type of Change

• Bug fix

Testing

• New lock-timeout-retry.test.ts (5 tests, fake clock): retries past 8 attempts within budget, stops before the budget ends, finds 55P03 in a cause chain, fails fast on other errors
• Mutation-checked: an 8-attempt cap, a budget check that ignores the delay, retrying every error, and ignoring the cause chain each fail at least one test
• End to end on a local DB: rolled back two ADD COLUMN migrations on a table and held ACCESS SHARE on it for 40 s from another session. The old script would have given up after 8 attempts in a longer outage. The new one logged 4 lock-timeout retries and applied both migrations at 52 s
• bun run type-check (packages/db), biome, and bun run check:audits (47 audits) pass

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
docs	Ready	Preview	Sep 23, 2026 5:39am UTC

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge; the previous adjustable-clock concern is fully fixed and no new actionable defects remain.

Summary

This PR replaces the migration runner’s fixed eight-attempt lock-timeout retry limit with a reusable, time-budgeted retry loop.

• Retries PostgreSQL 55P03 failures for up to 20 minutes using jittered backoff.
• Uses a monotonic production clock and prevents attempts from starting after the budget expires.
• Preserves immediate failure for non-lock-timeout errors and recognizes wrapped PostgreSQL errors.
• Adds focused tests covering extended retries, budget enforcement, delayed timers, wrapped errors, and attempt numbering.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Start migration attempt] --> B{Migration succeeds?}
    B -- Yes --> C[Finish]
    B -- No --> D{Error is PostgreSQL 55P03?}
    D -- No --> E[Fail immediately]
    D -- Yes --> F[Calculate jittered backoff]
    F --> G{Delay fits retry budget?}
    G -- No --> H[Throw last lock-timeout error]
    G -- Yes --> I[Sleep]
    I --> J{Budget still available?}
    J -- No --> H
    J -- Yes --> A

Loading

_{Reviews (2) · Last reviewed commit: "fix(db): measure the migration lock budg..."}

[cubic-dev-ai]

All reported issues were addressed across 3 files

<sub>Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic</sub>

[waleedlatif1]

@greptile

[waleedlatif1]

@cubic-dev-ai review this PR

[cubic-dev-ai]

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

[cubic-dev-ai]

No issues found across 3 files

Confidence score: 5/5

• Automated review surfaced no issues in the provided summaries.
• No files require special attention.

_{Re-trigger cubic}