Skip to content

Restore one-minute retries for healthcheck delivery - #798

Merged
f-f merged 4 commits into
masterfrom
fix-healthcheck-delivery
Oct 10, 2026
Merged

f-f merged 4 commits into
masterfrom
fix-healthcheck-delivery

Conversation

@thomashoneyman

@thomashoneyman thomashoneyman commented Oct 10, 2026 •

Copy link
Copy Markdown
Member

The healthcheck changes in #795 kept the database and executor checks, but changed failed ping delivery from a one-minute retry to a five-minute retry. That gives a transient delivery failure fewer chances to recover before Healthchecks alerts.

This restores the previous timing: wait five minutes after an accepted report, and retry failed deliveries after one minute. It keeps the operational metadata and ten-second request timeout, and logs the actual delivery error. Registry jobs and compiler-matrix scheduling are unchanged.

I verified recovery with a mock that rejected the first ping with HTTP 503. The previous version missed the recovery deadline; this version recovered within 90 seconds while registry jobs ran. This is already manually deployed.

Keep database and executor reporting, retry failed deliveries after one minute, and log the actual error. Leave registry jobs unchanged.
@thomashoneyman
thomashoneyman requested a review from f-f October 10, 2026 12:30
Include nested Fetch causes, error codes, and connection details in server logs. Keep healthcheck timing and registry job behavior unchanged.
@f-f
f-f merged commit c2d3549 into master Oct 10, 2026
7 checks passed
@f-f
f-f deleted the fix-healthcheck-delivery branch October 10, 2026 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants