Skip to content

gh-157216: Clarify that set_forkserver_preload() does not cover lazy imports - #157229

Closed
VimalN2005 wants to merge 1 commit into
python:mainfrom
VimalN2005:main
Closed

gh-157216: Clarify that set_forkserver_preload() does not cover lazy imports#157229
VimalN2005 wants to merge 1 commit into
python:mainfrom
VimalN2005:main

Conversation

@VimalN2005

Copy link
Copy Markdown

Description

Clarifies in Doc/library/multiprocessing.rst that set_forkserver_preload() only preloads modules imported at the top level.

Transitive dependencies that are imported lazily inside function or method bodies (for example, _strptime via datetime.datetime.strptime()) are not loaded in the forkserver process and will instead be imported separately in each child process upon first use.

… lazy imports

Clarify behavior of module preloading in forkserver.
@VimalN2005
VimalN2005 requested a review from gpshead as a code owner September 9, 2026 18:39
@python-cla-bot

python-cla-bot Bot commented Sep 9, 2026

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed

@read-the-docs-community

Copy link
Copy Markdown

Documentation build overview

📚 cpython-previews | 🛠️ Build #34477332 | 📁 Comparing 9367bc8 against main (52ffffe)

  🔍 Preview build  

1 file changed
± library/multiprocessing.html

@mohsen-rahmati-dev

Copy link
Copy Markdown

Thanks for picking this up. I filed the issue and took the measurements, so two substantive notes:

  1. "directly imported at top level" is wrong. Module-level imports are followed transitively — if a preloaded module imports A at module level and A imports B at module level, B is in the forkserver as well. Docs: set_forkserver_preload() does not cover lazily imported modules #157216 shows import importlib.metadata bringing in 75 modules that way, all inherited. The line is module-level vs. inside a function body, not depth. As written the paragraph makes preloading sound much weaker than it is.

  2. The consequence is missing. "imported separately in each child process" doesn't tell the reader why to care: each child allocates that memory privately, so the cost scales with the number of children (~5.9 MiB per child for the two modules in the issue).

Also :meth:~datetime.datetime.strptime`` for the shortened form, and I'd put the paragraph right after the opening description rather than between the launch-order note and the on_error parameter.

I've opened docs-gh-157216 covering these. Fine by me whichever a reviewer picks.

@picnixz picnixz closed this Sep 9, 2026
@picnixz

picnixz commented Sep 9, 2026

Copy link
Copy Markdown
Member

I will be picking the other PR as it ks the OP's PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting review docs Documentation in the Doc dir skip news

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

3 participants