Skip to content

fix(release-image): space multi-image promotions so their webhooks cannot race - #68

Merged
cshuttle merged 1 commit into
mainfrom
fix/space-image-tag-webhooks
Sep 17, 2026
Merged

cshuttle merged 1 commit into
mainfrom
fix/space-image-tag-webhooks

Conversation

@cshuttle

Copy link
Copy Markdown
Owner

Summary

A release that promotes two images tags them in the same second. ghcr fires one package webhook per tag, argocd-image-updater handles each in its own clone → commit → push against the same GitOps repo, and the loser is rejected:

! [remote rejected] main -> main (cannot lock ref 'refs/heads/main': is at 10496b0 but expected d220f45)

It is not retried, so that image's pin stays on the previous version until the 30-minute poll repairs it (cshuttle/main#244, and the reasoning in apps/argocd/image-updater/image-updater-serialize-patch.yaml). VirtualWindow has hit it on four of its last five releases; Atlas hit it twice in one day.

  • New tag-spacing-seconds input, default 60: the promote loop waits that long between images (no wait before the first, none for a single-image release).
  • A clone + commit + push takes a couple of seconds, so a minute is ample, and the cost is one wait on a release that already runs for minutes.
  • Repos whose images no bot watches can pass tag-spacing-seconds: 0.

Nothing else changes: images are still promoted by digest, in the same order, with the same verification.

Tests

  • The file parses (yaml.safe_load), and the input default reads back as 60.
  • No caller needs a change; the default applies.

🤖 Generated with Claude Code

…nnot race

ghcr fires one package webhook per tag, and argocd-image-updater handles each
in its own clone -> commit -> push against the same GitOps repo. A release
that promotes two images tags them in the same second, so the second push is
rejected with "cannot lock ref refs/heads/main" and is never retried: that
image's pin stays on the previous version until the 30-minute poll repairs it
(cshuttle/main#244). It has bitten VirtualWindow on four of its last five
releases and Atlas twice in a day.

A release now waits tag-spacing-seconds (default 60) between images. A push
takes a couple of seconds, so a minute is far more room than it needs, and it
costs one wait on a release that already runs for minutes. Repos whose images
no bot watches can set 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@cshuttle
cshuttle merged commit 72b9feb into main Sep 17, 2026
11 checks passed
@cshuttle
cshuttle deleted the fix/space-image-tag-webhooks branch September 17, 2026 20:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant