fix(docs): stop the docs build and docs tests deleting a cwd-relative data/ directory - #4339
Merged
d-v-b merged 3 commits intoSep 11, 2026
Merged
Conversation
… data/ directory
Two executable docs sessions opened with shutil.rmtree('data',
ignore_errors=True) so their examples could be re-run. Executed docs
blocks run in the process working directory -- markdown-exec inside the
mkdocs process with no workdir, pytest-examples inside the pytest process
with no chdir -- so `mkdocs build -f <repo>/mkdocs.yml` or
`pytest tests/test_docs.py` started from any directory holding a data/
folder silently emptied it. Reproduced both ways; started from `/` the
casualty is /data. The sdist ships docs/ and tests/ and testpaths collects
docs/user-guide, so this reached users running the shipped suite.
Drop the deletions and make every on-disk example idempotent with
overwrite=True (the zarr.save example moves to save_array(mode="w"),
since zarr.save cannot replace an existing array). Once nothing wiped
data/ between sessions, groups.md, storage.md and performance.md turned
out to depend on the wipe as well; they get the same treatment. A new
docs test rejects any executed block that calls a filesystem deletion,
and fails on the original docs naming both lines.
Verified: the docs suite passes three consecutive runs from a foreign
cwd; mkdocs build --strict from a foreign cwd exits 0 with the unrelated
data/ intact.
Assisted-by: ClaudeCode:claude-fable-5-1
Assisted-by: ClaudeCode:claude-fable-5-1
d-v-b
marked this pull request as ready for review
September 10, 2026 18:11
d-v-b
force-pushed
the
fix/docs-no-cwd-relative-rmtree
branch
from
September 10, 2026 18:22
d94a4ce to
52c1929
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4339 +/- ##
=======================================
Coverage 94.34% 94.34%
=======================================
Files 92 92
Lines 12935 12935
=======================================
Hits 12203 12203
Misses 732 732 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AI-authored PR that prevents our docs build / doctests from blindly nuking user directories on the filesystem.
🤖 AI text below 🤖
Summary
The first executable fence of two docs sessions —
docs/quick-start.mdanddocs/user-guide/arrays.md— ranshutil.rmtree('data', ignore_errors=True)so their examples could be re-run. Executed docs blocks run in the process working directory, not the docs tree: markdown-exec executes them inside the mkdocs process with noworkdir, and pytest-examples inside the pytest process with no chdir. A relative'data'therefore resolves against wherevermkdocs buildorpytestwas started, andignore_errors=Truehid every consequence. Reproduced both ways from a scratch directory containingdata/precious/file.txt:mkdocs build -f <repo>/mkdocs.ymlexits 0 and the file is gone, replaced by ~20 example stores;pytest tests/test_docs.py -k quick-startpasses and the file is gone. Started from/, that is/data. Since the sdist shipsdocs/andtests/andtestpathscollectsdocs/user-guide, this reached anyone running the shipped test suite, not just contributors. RTD and the docs CI job never noticed because they run from the repo root, where the only casualty is the repo's own gitignoreddata/.The
rmtreeexisted for exactly one reason, verified by removing it and re-running the sessions: on-diskcreate_array/create_group/zarr.group/create_hierarchycalls withoutoverwrite=Truefail withContainsArrayError/ContainsGroupErrorthe second time — and, once nothing wipesdata/between sessions, quick-start'sexample-3group and arrays.md'sexample-3array collide on the same path, and groups.md, storage.md and performance.md turn out to have relied on quick-start's wipe too. This PR:rmtreelines (and the now-unusedimport shutil);overwrite=Trueto every on-disk creation in those five pages, which is also what a reader copy-pasting an example a second time needs;zarr.saveexample in arrays.md tozarr.save_array(..., mode="w")with a sentence explaining why, becausezarr.savecannot replace an existing array (see below);test_no_destructive_filesystem_callstotests/test_docs.py, which fails on any executed docs block callingshutil.rmtree,os.remove/unlink/rmdir,Path.unlink/rmdir, orrm -rf, with a docstring recording why. Run against the original docs it names both offending lines.Verified after the change: the full docs suite (
-m "not s3 and not gpu") passes three consecutive runs from a foreign cwd (61 passed each),mkdocs build --strict -f <repo>/mkdocs.ymlfrom a foreign cwd exits 0 and leavesdata/precious/file.txtintact, and the only warnings in the strict build log are the two thatmainalready produces.Found during the pre-release review for #4256; the sdist hazard argues for landing it before 3.4.0.
For reviewers
Two things beyond the mechanical change. First,
zarr.savegenuinely cannot overwrite: with an existing array it raisesContainsArrayError, and passingmode="w"oroverwrite=TrueraisesTypeErrorbecausesave's**kwargsare interpreted as named arrays. zarr-python 2.18.7 overwrote silently (zarr.save(p, a); zarr.save(p, b)leavesb), so this is a 2.x→3.x behaviour change with no opt-in;save_arraydoes acceptmode="w". The API is deliberately left alone here (anoverwriteflag was prototyped and dropped: at a path holding a group it would delete the group and all its children, and it would break callers passing an array namedoverwriteby keyword), and the docs are adjusted to match it. Second, #4289 (in-memory stores for non-persistence examples) touches the same files and currently adds narrower per-examplermtreecalls, which the new guard will reject; whichever lands second needs a small rebase, and #4289 should switch tooverwrite=Truefor the examples it keeps on disk.Author attestation
TODO
docs/user-guide/*.mdchanges/