Skip to content

tags: no way to accept a newer platform than declared; arch lists unexported; libc union footgun; silent PyPy no-ABI target #45

Description

@jonyoder

Four independent gaps in tags, all found while designing declared-environment filtering for Posit Package Manager's air-gapped PyPI mirror. Grouped because they touch the same files and want one release.

Context for the severity of each: the consumer decides whether a wheel enters an air-gapped mirror. Rejecting a wheel pip would accept means the mirror silently lacks it and pip install fails with no fallback. Accepting one too many costs bytes.

1. No way to accept a platform tag newer than the declared version

Platform tag walks run downward from the declared version to a floor — manylinuxTags (generate.go:466-501), musllinuxTags (:515-521), macosPlatformTags (:586-612). So a target declared at glibc 2.40 accepts manylinux_2_40 down to manylinux_2_5 and rejects manylinux_2_45, and a macOS 14 target rejects macosx_15_0_arm64.

That is correct for "can this host run this wheel". It is wrong for a durable declaration: the constant is chosen once, and as upstream publishes newer wheels the mirror silently stops collecting them. The consumer's only defences are baking its own constant and adding a drift test that fails when PyPI publishes a new tag — a test that goes red for reasons unrelated to whatever change is under review.

I originally wanted an "unbounded above" matcher. That is not small: Matcher is exact set membership (tags []Tag plus a rank map, target.go:220-256), and "any newer version" is an infinite set with no defined Rank.

Proposed instead, and this is the tractable shape:

// IsCompatibleOrNewer reports whether any of w is compatible, OR is a
// same-family platform tag whose version floor is ABOVE this target's
// declared version -- i.e. a platform this target does not yet know about.
// Rank is undefined for the newer case.
func (m *Matcher) IsCompatibleOrNewer(w []Tag) bool

parseLibcPlatformTag (distro.go:193) already decomposes manylinux/musllinux tags; macOS needs an analogous helper. Roughly 40 lines plus tests, no change to Matcher's existing behavior.

2. The per-OS arch allow-lists are unexported with no accessor

linuxArchs, macosArchs, windowsArchs (target.go:203-205) are package-private. A consumer that validates an admin-supplied os/arch string can only construct a Target and inspect Compile()'s ErrUnsupportedTarget, which works for rejecting but cannot enumerate — so an error message cannot list the valid values, and documentation cannot be generated from the source of truth.

The spellings are also OS-dependent and unintuitive (windows/amd64 not x86_64; macos/arm64 not aarch64), which is exactly when a good error message matters.

// Archs returns the architectures valid for os ("linux", "macos", "windows"),
// or nil for an unknown os.
func Archs(os string) []string

A few lines. The alternative is every consumer maintaining a private copy that drifts from validate().

3. A Linux declaration that omits libc silently drops an entire wheel family

Target.validate() requires Libc for OS == "linux" (target.go:112-120), and linuxPlatformTags emits manylinux xor musllinux depending on it (:377-390). So a caller expressing "Linux x86_64, I don't care which libc" must compile two targets and union them. Forgetting the musl one silently drops every musllinux_* wheel, with nothing to indicate it.

That is a footgun rather than a bug, and it is better solved here than repeatedly:

// CompileAnyLibc compiles a linux Target for BOTH glibc and musl at the given
// version and returns a Matcher accepting either family. Errors for non-linux.
func (t Target) CompileAnyLibc() (*Matcher, error)

Bare linux_<arch> and py3-none-any are accepted by both families already, so the union does not double-count. Verified.

4. A PyPy target without an implementation version accepts no PyPy extension wheels, silently

Target{Implementation: "pp", PyMajor: 3, PyMinor: 10} with ImplMajor/ImplMinor both zero compiles successfully and emits only the pp310-none-<plat> tier plus the compatible tier (implABI, :199-204). Every real PyPy extension wheel is tagged pp310-pypy310_pp73-…, so such a target collects no PyPy C-extension wheels at all.

The Target doc does state this ("Both zero means 'implementation version unknown': the target then accepts no implementation-specific ABI wheels"), so it is documented rather than hidden — but of the three possible behaviors (error, emit all known PyPy ABIs, emit none) silently emitting none is the worst, because it looks like success.

Proposal: validate() rejects Implementation == "pp" with no implementation version, so the caller is forced to say pypy3.10-7.3. That is a breaking change for anyone relying on the current tier, hence grouping it into one minor bump.

Sequencing note

Item 1 is the one that removes machinery from consumers (a baked constant plus a drift test). Items 2-4 are small. Item 4 is breaking; 1-3 are additive.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions