`lazy import a.b as c` resolves `b` as an attribute of `a` instead of importing the module `a.b`

[grgalex]

Bug report

Bug description:

lazy import a.b as c never imports the module a.b by name. On first use it imports a and reads the attribute b off it, as lazy from a import b would, so c becomes whatever that attribute lookup finds. This manifests in two ways.

Case A: a.b is not a module. The eager statement raises; the lazy one binds the attribute.

lazy import math.pi as lazy_x
print(lazy_x)  # 3.141592653589793

import math.pi as x  # ModuleNotFoundError: No module named 'math.pi'; 'math' is not a package

The bare lazy import math.pi was fixed in #155194; the as form was not.

Case B: a.b is a module, but a has an attribute b that shadows it and a.b has not been imported yet. The eager statement imports the submodule, which rebinds a.b to the module; the lazy one binds the attribute and never imports the submodule. json does not import json.tool, so the assignment below stands in for an attribute a package's __init__.py would define:

import json
json.tool = "attribute"

lazy import json.tool as lazy_c
print(type(lazy_c).__name__)  # str

import json.tool as c
print(type(c).__name__)  # module

Case B is related to #151208, which reports the same shadowing for import a.foo under -X lazy_imports=all followed by an attribute access a.foo, resolved in Objects/moduleobject.c. That script now passes on main; this statement still fails, because it is resolved on a different path.

Possible root cause

import a.b as c compiles to IMPORT_NAME a.b followed by IMPORT_FROM b. Lazily, IMPORT_NAME leaves a placeholder holding a.b, and _PyEval_LazyImportFrom (Python/ceval.c:3329-3384) replaces it with one holding lz_from = "a", lz_attr = "b". That is the same placeholder lazy from a import b produces.

Reification in _PyImport_LoadLazyImportTstate (Python/import.c:3902-4093) therefore runs __import__("a", fromlist=("b",)) followed by getattr(a, "b") for both statements. Nothing records that the statement was an import, so the module a.b is never imported by its own name: the "is not a package" check never runs, and an existing attribute b is never overwritten by the submodule.

This is not #157614, which is about the sys.modules fast path in _PyEval_LazyImportFrom consulting the wrong module. The present issue reproduces with the fix in #157626 applied.

CPython versions tested on:

3.15

Operating systems tested on:

Linux

Linked PRs

• gh-157757: Fix to make lazy import a.b as c import the module a.b #158092
• [3.15] gh-157757: Fix to make lazy import a.b as c import the module a.b (GH-158092) #158257

[johnslavik]

@brittanyrey

[encukou]

@brittanyrey, do you need any help?
The 3.15.0 release is next week.

[brittanyrey]

Hey, @encukou I would really appreciate that!

I had started working on this one a little bit last week but I wasn't feeling great about the fix I had. I'll put it up as an RFC in case that helps at all. Please take a look and let me know what you think #158092

(If you don't mind, I'm going to cc you on a few more open lazy imports PRs)

[johnslavik]

I'll help too! Will review until the end of the week.

[brittanyrey]

Thanks @encukou & @johnslavik. I iterated a bit more on the fix and I think it actually might be good for review & to hopefully land 🤞.