reqtxt: record entry provenance, and close three pip-parity gaps - #49
Merged
Merged
Conversation
Flatten splices every include level into one flat File, consuming and
discarding each IncludeEntry, and OptionEntry carried only {Name, Value}.
So after flattening there was nothing left to attribute an entry to a file
with, and no line number on any entry at all.
That matters most for whole-file options, because File.Pre() and
IndexURL() are any-wins across the flattened result. A single "--pre"
nested three includes deep -- even inside a "-c" subtree -- silently
changes what a caller collects, and "pre-releases were enabled" is a
materially worse diagnostic than "enabled by line 2 of extra.txt". For a
consumer building an air-gapped bundle, the first is undiagnosable on a
machine with no network.
Source{Path, Line} is now on all four entry types, plus SourceOf for a
caller that only wants provenance and not a type switch. WithPath lets a
bare Parse record a path it cannot otherwise know, since Parse takes
content rather than a filename; Flatten supplies it per file
automatically.
Stamped in Parse's line loop rather than at the eight construction sites.
Every entry from a logical line shares that line, including the
per-requirement options gathered from its continuations, so one pass
cannot miss a site the way eight edits could.
Flatten APPENDS its WithPath after the caller's opts rather than
prepending, so a caller's own WithPath cannot attribute every included
file to a single path. TestSource_CallerWithPathDoesNotOverrideFlatten
pins that ordering, since getting it backwards would be invisible.
Test churn: 13 existing subtests compared whole entry structs and now
carry an expected Source. I set the real line numbers rather than zeroing
the field, so those assertions gained coverage instead of an exemption.
Two consumer-facing diagnostics are covered end to end because they are
the reason this exists: --pre leaking out of a constraints subtree, and a
"-r" nested inside a "-c" resetting constraint-ness so those pins become
requirements.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three real divergences from pip, verified against pip 26.1's own source
and packaging 26.3 rather than from reading.
1. --all-releases, --only-final and --use-feature were missing from
knownOptions, and all three are in pip's SUPPORTED_OPTIONS. The
consequence was worse than "not normalized": an option missing from the
table is assumed boolean, so its argument was dispatched as its own
line and became a FABRICATED requirement. "--use-feature 2020-resolver"
yielded a package named "2020-resolver" from a valid pip file, and a
consumer resolving the closure of every requirement then hunts for a
package that does not exist. All three take a value (pip: type="str").
TestKnownOptions_CoversPipSupportedOptions now asserts every long option
in pip's SUPPORTED_OPTIONS is handled, so this gap cannot silently
reopen.
Worth noting for consumers: --all-releases and --only-final are pip's
per-package replacements for --pre, which pip refuses to combine with
them.
2. A standalone --hash was a hard error. pip logs "line %s has --hash but
no requirement, and will be ignored" (req_file.py:217-219) and carries
on, so the file installs fine and rejecting it made this package
stricter than the thing it models. It is now surfaced as a file-level
option rather than dropped, so a caller can reproduce pip's warning,
and a test pins that it does not attach to a preceding requirement --
which is why it was an error in the first place.
3. Flatten(path, nil) panicked on the first dereference. It returns an
error now; in a CLI a stack trace is a materially worse failure and a
caller cannot tell the two apart from outside.
What did NOT change, deliberately: "requests;" with an empty marker stays
rejected. A review claimed pip accepts it. Checked against packaging 26.3,
Requirement("requests;") raises InvalidRequirement, so the rejection is
faithful. TestTrailingSemicolonStillRejected records that so the claim is
not re-litigated from memory.
Also considered and rejected: making unknown options a hard error to match
pip exactly, and discarding an unknown option's trailing tokens. Both
replace one arity guess with another -- assuming boolean is RIGHT for
"--frob foo", where foo really is a requirement, and wrong for
"--timeout 60". The fix is to leave fewer options unknown, not to change
the guess.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
jonyoder
force-pushed
the
feat/46-reqtxt-gaps
branch
from
August 28, 2026 10:39
39cad49 to
3e96ba8
Compare
jonyoder
added a commit
that referenced
this pull request
Aug 28, 2026
Dates [Unreleased] as 0.9.0 (2026-08-28) and adds its compare link. CHANGELOG only. A minor bump under the 0.x policy because two changes alter observable behaviour: reqtxt now recognizes --all-releases/--only-final/--use-feature (an unrecognized option was assumed boolean, so its argument became a fabricated requirement), and a standalone --hash line is accepted rather than rejected, matching pip. Contents since v0.8.0: tags manylinux floor fix (#43), marker EvaluateUndecidable/Variables (#47), tags IsCompatibleOrNewer/CompileAnyLibc/Archs (#48), reqtxt Source and pip-parity fixes (#49), dependency bumps (#50).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #46.
Two commits: provenance, then three pip-parity fixes.
Source on every entry (the substantial piece)
Flattensplices every include level into one flatFileand consumes eachIncludeEntry, andOptionEntrycarried only{Name, Value}. So after flattening there was nothing left to attribute an entry to a file with, and no entry carried a line number at all.That matters most for whole-file options, because
Pre()andIndexURL()are any-wins across the flattened result. A single--prenested three includes deep, even inside a-csubtree, silently changes what a caller collects. "Pre-releases were enabled" is a materially worse diagnostic than "enabled by line 2 of extra.txt", and on a machine with no network the first is undiagnosable.Source{Path, Line}is now on all four entry types, withSourceOffor a caller that wants provenance without a type switch, andWithPathso a bareParsecan record a path it otherwise cannot know.Flattensupplies it per file.Stamped in
Parse's line loop rather than at the eight construction sites: every entry from a logical line shares that line, including per-requirement options gathered from continuations, so one pass cannot miss a site the way eight edits could.Flattenappends itsWithPathafter the caller's opts rather than prepending, so a caller's ownWithPathcannot attribute every included file to one path.TestSource_CallerWithPathDoesNotOverrideFlattenpins that, since getting it backwards would be invisible.Two consumer diagnostics are covered end to end because they are why this exists:
--preleaking out of a constraints subtree, and a-rnested inside a-cresetting constraint-ness so those pins become requirements.Test churn: 13 existing subtests compared whole entry structs and now carry an expected
Source. I set the real line numbers rather than zeroing the field, so those assertions gained coverage instead of an exemption.Three pip-parity fixes
All three verified against pip 26.1's own source and packaging 26.3, not from reading this package.
--all-releases,--only-final,--use-featurewere missing fromknownOptionsand all three are in pip'sSUPPORTED_OPTIONS. The consequence was worse than losing normalization: an option missing from the table is assumed boolean, so its argument was dispatched as its own line and became a fabricated requirement.--use-feature 2020-resolverproduced a package named2020-resolverfrom a valid pip file, and a consumer resolving the closure of every requirement then hunts for something that does not exist.TestKnownOptions_CoversPipSupportedOptionsnow asserts every long option inSUPPORTED_OPTIONSis handled, so the gap cannot silently reopen.Worth flagging for consumers:
--all-releasesand--only-finalare pip's per-package replacements for--pre, which pip refuses to combine with them.A standalone
--hashwas a hard error. pip logs "line %s has --hash but no requirement, and will be ignored" (req_file.py:217-219) and carries on, so the file installs fine and rejecting it made this package stricter than pip. Now surfaced as a file-level option rather than dropped, so a caller can reproduce pip's warning — with a test pinning that it does not attach to a preceding requirement, which is why it was an error to begin with.Flatten(path, nil)panicked on the first dereference. Returns an error now.What deliberately did not change
requests;with an empty marker stays rejected. #46 claimed pip accepts it; checked against packaging 26.3,Requirement("requests;")raisesInvalidRequirement: Expected a marker variable or quoted string. The rejection is faithful, andTestTrailingSemicolonStillRejectedrecords that so it is not re-litigated from memory.I also considered making unknown options a hard error to match pip exactly, and discarding an unknown option's trailing tokens. Both replace one arity guess with another: assuming boolean is the right guess for
--frob foo, wherefooreally is a requirement, and the wrong one for--timeout 60. The fix is to leave fewer options unknown, not to change the guess. The reasoning is in a comment at that branch so the next reader does not redo it.Verification
gofmtclean,go vet ./...clean, full module suite green.🤖 Generated with Claude Code