Skip to content

fix(docs): stop the docs build and docs tests deleting a cwd-relative data/ directory - #4339

Merged
d-v-b merged 3 commits into
zarr-developers:mainfrom
d-v-b:fix/docs-no-cwd-relative-rmtree
Sep 11, 2026
Merged

fix(docs): stop the docs build and docs tests deleting a cwd-relative data/ directory#4339
d-v-b merged 3 commits into
zarr-developers:mainfrom
d-v-b:fix/docs-no-cwd-relative-rmtree

Conversation

@d-v-b

@d-v-b d-v-b commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

AI-authored PR that prevents our docs build / doctests from blindly nuking user directories on the filesystem.

🤖 AI text below 🤖

Summary

The first executable fence of two docs sessions — docs/quick-start.md and docs/user-guide/arrays.md — ran shutil.rmtree('data', ignore_errors=True) so their examples could be re-run. Executed docs blocks run in the process working directory, not the docs tree: markdown-exec executes them inside the mkdocs process with no workdir, and pytest-examples inside the pytest process with no chdir. A relative 'data' therefore resolves against wherever mkdocs build or pytest was started, and ignore_errors=True hid every consequence. Reproduced both ways from a scratch directory containing data/precious/file.txt: mkdocs build -f <repo>/mkdocs.yml exits 0 and the file is gone, replaced by ~20 example stores; pytest tests/test_docs.py -k quick-start passes and the file is gone. Started from /, that is /data. Since the sdist ships docs/ and tests/ and testpaths collects docs/user-guide, this reached anyone running the shipped test suite, not just contributors. RTD and the docs CI job never noticed because they run from the repo root, where the only casualty is the repo's own gitignored data/.

The rmtree existed for exactly one reason, verified by removing it and re-running the sessions: on-disk create_array / create_group / zarr.group / create_hierarchy calls without overwrite=True fail with ContainsArrayError / ContainsGroupError the second time — and, once nothing wipes data/ between sessions, quick-start's example-3 group and arrays.md's example-3 array collide on the same path, and groups.md, storage.md and performance.md turn out to have relied on quick-start's wipe too. This PR:

  • deletes both rmtree lines (and the now-unused import shutil);
  • adds overwrite=True to every on-disk creation in those five pages, which is also what a reader copy-pasting an example a second time needs;
  • switches the zarr.save example in arrays.md to zarr.save_array(..., mode="w") with a sentence explaining why, because zarr.save cannot replace an existing array (see below);
  • adds test_no_destructive_filesystem_calls to tests/test_docs.py, which fails on any executed docs block calling shutil.rmtree, os.remove / unlink / rmdir, Path.unlink / rmdir, or rm -rf, with a docstring recording why. Run against the original docs it names both offending lines.

Verified after the change: the full docs suite (-m "not s3 and not gpu") passes three consecutive runs from a foreign cwd (61 passed each), mkdocs build --strict -f <repo>/mkdocs.yml from a foreign cwd exits 0 and leaves data/precious/file.txt intact, and the only warnings in the strict build log are the two that main already produces.

Found during the pre-release review for #4256; the sdist hazard argues for landing it before 3.4.0.

For reviewers

Two things beyond the mechanical change. First, zarr.save genuinely cannot overwrite: with an existing array it raises ContainsArrayError, and passing mode="w" or overwrite=True raises TypeError because save's **kwargs are interpreted as named arrays. zarr-python 2.18.7 overwrote silently (zarr.save(p, a); zarr.save(p, b) leaves b), so this is a 2.x→3.x behaviour change with no opt-in; save_array does accept mode="w". The API is deliberately left alone here (an overwrite flag was prototyped and dropped: at a path holding a group it would delete the group and all its children, and it would break callers passing an array named overwrite by keyword), and the docs are adjusted to match it. Second, #4289 (in-memory stores for non-persistence examples) touches the same files and currently adds narrower per-example rmtree calls, which the new guard will reject; whichever lands second needs a small rebase, and #4289 should switch to overwrite=True for the examples it keeps on disk.

Author attestation

  • I am a human, these are my changes, and I have reviewed and understood every change and can explain why each is correct.

TODO

  • Add unit tests and/or doctests in docstrings
  • Add docstrings and API docs for any new/modified user-facing classes and functions (n/a)
  • New/modified features documented in docs/user-guide/*.md
  • Changes documented as a new file in changes/
  • GitHub Actions have all passed
  • Test coverage is 100% (Codecov passes)

… data/ directory

Two executable docs sessions opened with shutil.rmtree('data',
ignore_errors=True) so their examples could be re-run. Executed docs
blocks run in the process working directory -- markdown-exec inside the
mkdocs process with no workdir, pytest-examples inside the pytest process
with no chdir -- so `mkdocs build -f <repo>/mkdocs.yml` or
`pytest tests/test_docs.py` started from any directory holding a data/
folder silently emptied it. Reproduced both ways; started from `/` the
casualty is /data. The sdist ships docs/ and tests/ and testpaths collects
docs/user-guide, so this reached users running the shipped suite.

Drop the deletions and make every on-disk example idempotent with
overwrite=True (the zarr.save example moves to save_array(mode="w"),
since zarr.save cannot replace an existing array). Once nothing wiped
data/ between sessions, groups.md, storage.md and performance.md turned
out to depend on the wipe as well; they get the same treatment. A new
docs test rejects any executed block that calls a filesystem deletion,
and fails on the original docs naming both lines.

Verified: the docs suite passes three consecutive runs from a foreign
cwd; mkdocs build --strict from a foreign cwd exits 0 with the unrelated
data/ intact.

Assisted-by: ClaudeCode:claude-fable-5-1
Assisted-by: ClaudeCode:claude-fable-5-1
@d-v-b
d-v-b marked this pull request as ready for review September 10, 2026 18:11
@d-v-b
d-v-b force-pushed the fix/docs-no-cwd-relative-rmtree branch from d94a4ce to 52c1929 Compare September 10, 2026 18:22
@codecov

codecov Bot commented Sep 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.34%. Comparing base (8a11de8) to head (52c1929).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4339   +/-   ##
=======================================
  Coverage   94.34%   94.34%           
=======================================
  Files          92       92           
  Lines       12935    12935           
=======================================
  Hits        12203    12203           
  Misses        732      732           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@d-v-b
d-v-b merged commit 26820c7 into zarr-developers:main Sep 11, 2026
65 of 73 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant