Skip to content

Auto-update melange image if older than 30 days - #30

Merged
xnox merged 3 commits into
mainfrom
auto-update-melange-image-clean
Oct 2, 2026
Merged

xnox merged 3 commits into
mainfrom
auto-update-melange-image-clean

Conversation

@AmberArcadia

@AmberArcadia AmberArcadia commented Mar 26, 2026 •

Copy link
Copy Markdown
Member

What

shellcheck-run-steps compiles each package with cgr.dev/chainguard/melange:latest. docker run never refreshes a local copy of that tag, so new config fields (for example test-resources) fail to compile until someone pulls by hand. This PR refreshes the image when the local copy is missing or older than a threshold.

How

  • docker_image_outdated(image, max_age) reads docker image inspect --format {{.Created}} and returns True when the image is absent or older than max_age.
  • When it is, the first docker run ... compile of the invocation carries --pull=always; later files in the same invocation go back to --pull=missing. One pull code path, and a missing image takes the same route as a stale one.
  • Default threshold is 7 days. Consumers override it with args: [--max-image-age-days=N]; 0 refreshes on every run.
  • The check runs after argument parsing, so --help never touches docker.

Review feedback

  • stevebeattie: the check ran before argparse, so --help could trigger a pull. Moved.
  • dannf: 30 days was too long given how fast melange moves. Default is now 7 and configurable.
  • dannf: replace the two docker pull paths with a boolean check plus --pull always. Done.
  • dannf: reuse it for the shellcheck image. Not done on purpose: DefaultShellCheckImage is pinned by digest, so it never goes stale, and docker run already pulls a missing one.

Bug fixed along the way

docker image inspect prints RFC 3339 with up to nanosecond precision; the shellcheck image on my machine reports 2025-07-31T02:42:03.371039353Z. datetime.fromisoformat() on Python 3.9 and 3.10 accepts only 0, 3 or 6 fractional digits and raises Invalid isoformat string, while setup.cfg keeps python_requires = >=3.9. The melange image happened to report 2026-09-23T13:50:55Z with no fraction, which is why the earlier version worked. The fraction is now stripped before parsing.

Verification

  • pre-commit run --all-files passes (ruff, mypy, reorder-python-imports, shellcheck).

  • --help prints the new option and exits without a docker call.

  • --max-image-age-days 0 -- -S error -- os/zfs.yaml in a stereo checkout printed cgr.dev/chainguard/melange:latest is 9 days old; refreshing it, docker reported Status: Downloaded newer image, and the compile and shellcheck ran as before. The default run right after did not pull.

  • parse_docker_timestamp under python:3.10-alpine:

    2025-07-31T02:42:03.371039353Z -> 2025-07-31 02:42:03+00:00
    2026-09-23T13:50:55Z           -> 2026-09-23 13:50:55+00:00
    

Note

pre-commit runs hooks in parallel batches unless require_serial: true, so with many files several processes may each decide to refresh. Docker serialises the layer download, so the cost is a few extra registry checks; not worth making the hook serial.

@AmberArcadia
AmberArcadia requested review from dannf and removed request for dannf March 27, 2026 00:02
@AmberArcadia

AmberArcadia commented Mar 27, 2026 •

Copy link
Copy Markdown
Member Author

Hmmm it won't show the prompt by default it seems, oh well.

● Bash(timeout 25 pre-commit try-repo /home/amber-arcadia/Documents/GitRepos/pre-commit-hooks shellcheck-run-steps --verbose --files os/aws-privateca-issuer.yaml 2>&1 | tail -50)                     
  ⎿  ===============================================================================                                                                                                                   
     Using config:                                                                                                                                                                                     
     ===============================================================================                                                                                                                   
     repos:                                                                                                                                                                                            
     -   repo: ../pre-commit-hooks                                                                                                                                                                     
         rev: be610949abd356e2daa4e4a9e9659b6dd1d845e2                                                                                                                                                 
         hooks:                                                                                                                                                                                        
         -   id: shellcheck-run-steps                                                                                                                                                                  
     ===============================================================================                                                                                                                   
     [INFO] Initializing environment for ../pre-commit-hooks.                                                                                                                                          
     [INFO] Installing environment for ../pre-commit-hooks.                                                                                                                                            
     [INFO] Once installed this environment will be reused.                                                                                                                                            
     [INFO] This may take a few minutes...                                                                                                                                                             
     shellcheck run steps.....................................................Passed                                                                                                                   
     - hook id: shellcheck-run-steps                                                                                                                                                                   
     - duration: 16.38s                                                                                                                                                                                
                                                                                                                                                                                                       
     ⚠️  Melange image is 0 days old (created 2026-03-26)                                                                                                                                              
     ⚠️  Pulling updated melange image: cgr.dev/chainguard/melange:latest                                                                                                                              
     ⚠️  Press Ctrl+C now to abort or wait 15 seconds to continue...                                                                                                                                   
     Pulling cgr.dev/chainguard/melange:latest...                                                                                                                                                      
     latest: Pulling from chainguard/melange                                                                                                                                                           
     Digest: sha256:6c53e6558fc69b516f5c5704fe6aead50d91bfc40ebe9163797a52a18627f1e1                                                                                                                   
     Status: Image is up to date for cgr.dev/chainguard/melange:latest                                                                                                                                 
     cgr.dev/chainguard/melange:latest                                                                                                                                                                 
     ✓ Melange image updated successfully                                                                                                                                                              
     2026/03/27 00:52:42 WARN unable to detect commit for build config file: opening git repository: repository does not exist                                                                         
     2026/03/27 00:52:42 WARN git repository URL for build config not provided                              

@AmberArcadia
AmberArcadia requested a review from dannf March 27, 2026 01:25

@stevebeattie stevebeattie left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beyond one minor nit that doesn't necessarily need to block landing this looks good to me.

Comment thread pre_commit_hooks/shellcheck_run_steps.py Outdated

@dannf dannf left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great idea!

Comment thread pre_commit_hooks/shellcheck_run_steps.py Outdated
Comment thread pre_commit_hooks/shellcheck_run_steps.py Outdated
Automatically checks the age of the local melange Docker image and pulls
an updated version if it's older than 30 days. Gives users a 15-second
window to abort with Ctrl+C before pulling.
…gurable

Addresses the review feedback on #30:

- The age check now runs after argument parsing, so `--help` no longer
  touches docker (stevebeattie).
- The default threshold drops from 30 to 7 days and is configurable with
  `--max-image-age-days DAYS`; 0 refreshes on every run (dannf).
- `docker_image_outdated()` returns a bool and the first `docker run` of
  the invocation carries `--pull=always` when it is true, replacing the
  two separate `docker pull` paths (dannf). A missing image takes the
  same route as a stale one.

Also fixes timestamp parsing. `docker image inspect` prints RFC 3339 with
up to nanosecond precision (the shellcheck image reports
2025-07-31T02:42:03.371039353Z) and datetime.fromisoformat() on Python
3.9 and 3.10 rejects anything but 0, 3 or 6 fractional digits, while
setup.cfg still declares python_requires >= 3.9. The fraction is now
stripped before parsing; verified under python:3.10-alpine.

The shellcheck image is pinned by digest, so it is left alone: a digest
never goes stale and `docker run` already pulls a missing one.
@AmberArcadia
AmberArcadia force-pushed the auto-update-melange-image-clean branch from c10d026 to 2cde8d7 Compare October 2, 2026 17:50
Comment thread pre_commit_hooks/shellcheck_run_steps.py
@AmberArcadia

Copy link
Copy Markdown
Member Author

Pushed 2cde8d7 on top of a rebase onto main. It addresses all three review asks: the image check now runs after argument parsing (so --help is docker-free), the default drops to 7 days with --max-image-age-days to override, and a boolean docker_image_outdated() feeds --pull=always on the first docker run instead of two separate pull paths. It also fixes datetime.fromisoformat() rejecting docker's nanosecond timestamps on Python 3.9 and 3.10. The description is rewritten to match the code. @stevebeattie @dannf would you take another look?

@xnox
xnox merged commit 9312410 into main Oct 2, 2026
6 checks passed
@xnox
xnox deleted the auto-update-melange-image-clean branch October 2, 2026 18:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants