Build the AstroArch image in CI, and ship a PHD2 that starts - #5
Open
teoteo wants to merge 6 commits into
Open
Conversation
build_img.sh relied on `losetup --partscan` and the /dev/loopNp1, /dev/loopNp2 nodes it is supposed to create. Those nodes are created by udev, which is not guaranteed to be running: inside containers they never appear, and on GitHub-hosted runners they are missing often enough to have an open bug report against them. The script then failed at mkfs with "No such file or directory". Attaching one loop device per partition with --offset/--sizelimit needs no udev at all and works the same way everywhere. The root PARTUUID, which used to come from `blkid` on the partition node, is now derived from the MBR disk identifier, which is exactly how the kernel builds it: <disk-id>-<NN>.
The AstroArch image was still assembled by hand: build the rootfs locally, write it to a disk image, boot that image under QEMU and finish the job from inside the running system. Since the build script upstream stopped relying on systemctl, none of that needs a booted system any more, so the whole chain fits in one CI job. astroarch-image.yml runs `make prepare-rpi-img` end to end: Dockerfile.base, then Dockerfile.aarch64, then Dockerfile.astroarch, then build_img.sh. Each image is tagged with exactly the reference the next FROM line uses, so BuildKit takes it from the local image store and no intermediate image has to be pushed to a registry to make the chain work. It defaults to ubuntu-24.04-arm, which is aarch64 natively: no QEMU user-mode emulation, and free on public repositories. On an x86_64 runner it registers the arm64 binfmt handler instead and still works. Two things the runners force: the AstroArch rootfs plus the image on top of it come close to the free space on a runner, so the working files go on whichever filesystem has more room, Docker's storage is moved along with them when that helps, and every intermediate copy is deleted as soon as it is no longer needed; and pacman aborts its whole transaction when a single mirror response is slow, so the image builds are retried. verify_img.sh checks the result the way the QEMU boot used to: it mounts both partitions and asserts that the kernel, the users, the enabled units, the astrometry indexes and the rewritten root=PARTUUID are all in place.
The AstroArch package install cannot resolve at all right now: :: unable to satisfy dependency 'libjpeg' required by libgphoto2 :: unable to satisfy dependency 'libgphoto2' required by libindi :: unable to satisfy dependency 'libindi' required by kstars error: failed to prepare transaction (could not satisfy dependencies) The [astromatto] repo carries libgphoto2 2.5.30-1, built in May 2024, whose dependency list still names the bare `libjpeg` virtual. Current libjpeg-turbo only provides `libjpeg.so=8-64`, so nothing satisfies it. Since the Dockerfile inserts [astromatto] above [core] and [extra], pacman prefers that copy over extra's 2.5.34-1, which depends on libjpeg-turbo and resolves fine, and the whole transaction dies along with libindi, indi-3rdparty-* and kstars. Asking for extra/libgphoto2 explicitly is the smallest fix that unblocks the build. Removing libgphoto2 from [astromatto] would be the better one, and would make this line unnecessary.
Most of what AstroArch installs is a symlink into /home/astronaut/.astroarch, and those targets are absolute. Testing them from the host followed them to the host's own /home and /usr, so the checks reported the SDDM config as missing, sshd as disabled and novnc as dangling on an image where all three were fine. Paths are now walked link by link with absolute targets re-rooted at the mount point, which is how the booted system would resolve them. The default.target check also falls back to systemd's own /usr/lib/systemd/system/default.target, since an image that never overrides it still boots into whatever that points at.
The [astromatto] phd2 package is linked against opencv 4.13, which Arch has replaced with opencv 5. All 57 libopencv_*.so.413 it asks for are gone from every aarch64 repo, so the shipped binary cannot start. Nothing catches this: the package declares no `opencv` dependency, pacman resolves the transaction cleanly, and kstars pulls in opencv 5 anyway, so the image builds green and PHD2 only fails on the user's Pi. PHD2 does not need OpenCV. It references exactly four symbols (cv::VideoCapture, cv::Mat's ctor and dtor, cv::cvtColor), all from src/cam_opencv.cpp, which is entirely inside `#ifdef OPENCV_CAMERA` - the webcam driver, redundant next to the INDI V4L2 driver and the vendor SDKs. Deleting that define and the unconditional find_package(OpenCV) drops the OpenCV link, so the binary no longer tracks opencv sonames at all. PHD2 2.6.14 is therefore compiled in its own stage, which uses the same [astromatto] repo as the runtime stage so it links against the cfitsio and libindi the image actually ships. The result is 21 DT_NEEDED entries instead of 79, all resolvable, with the same camera coverage: the ZWO and QHY SDKs move from a shared library to the static archives upstream ships for armv8, and Player One, SVBONY, ToupTek and OGMA are still installed in /usr/lib/phd2. verify_img.sh gains the check that would have caught this. `check` only proves a file exists, which is why a dead PHD2 passed verification; the new check_links resolves every DT_NEEDED of the astrophotography binaries against the sonames present in the image. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HpwpeqWfJDvjfbWZJ8jKE6
ArchLinuxARM has no /usr/lib64, and find exits non-zero for a missing starting point even with its stderr discarded. Under `set -e -o pipefail` that killed verify_img.sh the moment it reached the new check, throwing away a three-hour image build over a directory that was never expected to exist. The index is now built one directory at a time, skipping the ones that are absent and tolerating find's exit status; the readelf call gets the same treatment, so a failure there reaches the warn branch instead of aborting the run. Checked against a real ArchLinuxARM aarch64 root rather than in CI: the index comes back with 2602 sonames, PHD2's 21 shared libraries all resolve, the non-ELF /usr/bin/phd2 wrapper is skipped with a warning and a missing binary still fails. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HpwpeqWfJDvjfbWZJ8jKE6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds the whole AstroArch chain on a GitHub-hosted runner and produces a bootable Raspberry Pi
.img, and fixes the PHD2 that the image has been shipping broken.Green end-to-end run, 17 minutes on
ubuntu-24.04-arm: https://github.com/teoteo/ArchLinuxARM/actions/runs/34129179533 — artifactastroarch-2.0.8-20260907-aarch64.img.zst, 4073 MiB, 0 failures and 0 warnings from the verifier.PHD2 is currently dead in the image
The
[astromatto]phd2 package is linked against opencv 4.13. Arch has since replaced opencv 4 with opencv 5, so all 57libopencv_*.so.413it asks for exist in no aarch64 repo any more:Nothing catches it. The package declares no
opencvdependency, so pacman resolves the transaction cleanly; kstars pulls in opencv 5 anyway; the build goes green and PHD2 only fails the first time a user launches it on their Pi.PHD2 does not need OpenCV. It references exactly four symbols —
cv::VideoCapture,cv::Mat's ctor and dtor,cv::cvtColor— all fromsrc/cam_opencv.cpp, which is entirely inside#ifdef OPENCV_CAMERA: the webcam driver, redundant next to the INDI V4L2 driver and the vendor SDKs. Deleting that one define and the unconditionalfind_package(OpenCV)drops the OpenCV link completely, so the binary stops tracking opencv sonames for good.PHD2 2.6.14 is therefore compiled in its own stage, which uses the same
[astromatto]repo as the runtime stage so it links against the cfitsio and libindi the image actually ships. 21DT_NEEDEDentries instead of 79, all resolvable, with the same camera coverage: the ZWO and QHY SDKs move from a shared library to the static archives upstream ships for armv8, and Player One, SVBONY, ToupTek and OGMA are still installed in/usr/lib/phd2.This is a workaround, not the real fix. These files are owned by no pacman package, so
pacman -Syuon a device will not update them, and anyone installingphd2by hand still gets the broken binary. Patching the PKGBUILD in[astromatto]would make this stage unnecessary — it is three lines:vtkandhdf5can come out of that package'sdependstoo — they are not inDT_NEEDEDat all,${OpenCV_LIBS}just drags them in.Verification that would have caught it
verify_img.shchecked that/usr/bin/phd2exists, which a dead binary passes. The newcheck_linksresolves everyDT_NEEDEDof the astrophotography binaries against the sonames present in the image:It reads the ELF headers with
readelf, so it needs neither a chroot nor QEMU.The rest of the branch
.github/workflows/astroarch-image.yml— the full chain on one runner, defaulting toubuntu-24.04-armso nothing is emulated. Each image is tagged with exactly the reference the next Dockerfile'sFROMexpects, which is what makes the from-scratch chain work in a single job without pushing to a registry in between. Nothing is published by default; the image is uploaded as an artifact and releases are opt-in.scripts/build_img.sh— maps partitions with per-partition loop devices, sincelosetup --partscandoes not reliably create/dev/loopNp1on GitHub runners.scripts/verify_img.sh— checks a built.imgwithout booting it. Symlinks are resolved against the image rather than the host, because much of what AstroArch installs points into/home/astronaut/.astroarchwith absolute targets.Dockerfile.astroarch—libgphoto2pinned toextra. The[astromatto]copy still depends on the barelibjpegvirtual that libjpeg-turbo no longer provides, and because[astromatto]sits above[extra]it takes libindi, indi-3rdparty-* and kstars down with it.One thing worth a second opinion
The workflow skips compression and upload when verification fails. That is deliberate — no point compressing a bad image — but it means a bug in the checker throws away a complete build, which is exactly what happened on the first run here. Making the upload
if: always()while leaving the job red would keep the image around to inspect. Happy to change it either way.🤖 Generated with Claude Code
https://claude.ai/code/session_01HpwpeqWfJDvjfbWZJ8jKE6