Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
aa9af62
feat: start Python 3.12+ and Windows-safe runtime compatibility
ReallocAll Oct 9, 2026
e041fa4
ci: validate EndKeep and Amulet backports on six Python/OS combinations
ReallocAll Oct 9, 2026
237a433
fix(py312): parenthesize multi-exception handlers for Python 3.12/3.13
ReallocAll Oct 9, 2026
0423c27
fix(py312): parenthesize multi-exception handlers for Python 3.12/3.13
ReallocAll Oct 9, 2026
2870791
fix(py312): parenthesize multi-exception handlers for Python 3.12/3.13
ReallocAll Oct 9, 2026
a030a52
fix(windows): tolerate unsupported directory fsync and escape TOML paths
ReallocAll Oct 9, 2026
192aca2
fix(test): emit valid cross-platform TOML basic string escape sequences
ReallocAll Oct 9, 2026
a749d7e
fix(ci): reuse upstream Visual Studio 2026 toolchain on Windows
ReallocAll Oct 9, 2026
43937ff
style: match Ruff formatter for Windows-safe TOML test
ReallocAll Oct 9, 2026
a3b20ea
fix(windows): flush restored and staged file data using writable handles
ReallocAll Oct 9, 2026
ec1b398
test: fail early if the third logical snapshot loses records on Windows
ReallocAll Oct 9, 2026
125ef8e
test: isolate Windows LevelDB record loss across raw/clone boundary
ReallocAll Oct 9, 2026
a35c720
test: compare Windows source and cloned LevelDB file hashes before op…
ReallocAll Oct 9, 2026
168ab3a
fix(windows): use binary CRT handles for byte-exact LevelDB cloning a…
ReallocAll Oct 9, 2026
b115172
style: satisfy Ruff after Windows clone diagnostics
ReallocAll Oct 9, 2026
17d886e
style: resolve remaining Ruff formatting errors
ReallocAll Oct 9, 2026
3b61344
ci: collect verified compatibility wheels and document six-platform r…
ReallocAll Oct 9, 2026
b12091e
test: require fatal data-file flush errors on Linux and Windows
ReallocAll Oct 9, 2026
e4c8483
fix(windows): restore SIDECAR files with binary CRT descriptors
ReallocAll Oct 9, 2026
9f828ff
test(windows): cover arbitrary binary SIDECAR restore
ReallocAll Oct 9, 2026
f1caa7a
fix(staging): reject unsafe and foreign-world paths before filesystem…
ReallocAll Oct 9, 2026
37be46f
fix(bds): reject Windows drive and ADS paths in save query output
ReallocAll Oct 9, 2026
7262471
test(staging): fail closed before creating unsafe directories
ReallocAll Oct 9, 2026
51fd762
test(bds): reject drive and alternate-stream manifests
ReallocAll Oct 9, 2026
09f8d3f
fix(restore): validate SIDECAR paths before creating output directories
ReallocAll Oct 9, 2026
afe5150
test(restore): reject SIDECAR traversal and Windows ADS payloads
ReallocAll Oct 9, 2026
5425ab2
style: organize SIDECAR test imports and formatting
ReallocAll Oct 9, 2026
dee5501
style: format SIDECAR path validation
ReallocAll Oct 9, 2026
8e5c92d
feat(deps): resolve pinned Amulet compatibility Wheels automatically …
ReallocAll Oct 9, 2026
8c4c8bb
fix(deps): use TOML literal strings for PEP 508 marker expressions
ReallocAll Oct 9, 2026
21a2389
fix(build): ignore non-EndKeep tags during hatch-vcs version resolution
ReallocAll Oct 9, 2026
e8fc4b3
fix(build): allow pinned GitHub Wheel references in hatchling metadata
ReallocAll Oct 9, 2026
92d32c5
docs: publish tested platform requirements and BDS validation evidence
ReallocAll Oct 9, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions .github/scripts/amulet_leveldb_smoke.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
"""Isolated smoke test for Amulet-LevelDB 3.0.7a0 experimental manylinux wheels."""

from importlib.metadata import version
from pathlib import Path
from tempfile import TemporaryDirectory

from amulet.leveldb import LevelDB


def main() -> None:
assert version("amulet-leveldb").split("+", 1)[0] == "3.0.7a0"
expected = {b"\x00": b"alpha", b"\x02": b"bravo", b"\xff": b"\x00\xff"}
with TemporaryDirectory(prefix="amulet-backport-") as tmp:
location = Path(tmp) / "db"
db = LevelDB(str(location), True)
db.put(b"\x00", b"alpha")
db.put_batch({b"\x02": b"bravo", b"\xff": b"\x00\xff"})
for key, value in expected.items():
assert db.get(key) == value, (key, value)
iterator = db.create_iterator()
iterator.seek_to_first()
got = []
while iterator.valid():
got.append((iterator.key(), iterator.value()))
iterator.next()
assert got == sorted(expected.items()), got
del iterator
db.close()

reopened = LevelDB(str(location), False)
for key, value in expected.items():
assert reopened.get(key) == value
reopened.close()
print("OK: CPython wheel import, KV write/batch, sorted iterator, close/reopen")


if __name__ == "__main__":
main()
172 changes: 172 additions & 0 deletions .github/workflows/compatibility.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
name: Python and Windows compatibility

on:
pull_request:
branches: [main]
workflow_dispatch:

concurrency:
group: compatibility-${{ github.head_ref || github.ref_name }}
cancel-in-progress: true

permissions:
contents: read

jobs:
full-matrix:
name: ${{ matrix.os }} / Python ${{ matrix.python }}
runs-on: ${{ matrix.os }}
timeout-minutes: 50
strategy:
fail-fast: false
matrix:
include:
- { os: ubuntu-24.04, python: '3.12', wheel: cp312-manylinux_x86_64 }
- { os: ubuntu-24.04, python: '3.13', wheel: cp313-manylinux_x86_64 }
- { os: ubuntu-24.04, python: '3.14', wheel: '' }
- { os: windows-2025, python: '3.12', wheel: cp312-win_amd64 }
- { os: windows-2025, python: '3.13', wheel: cp313-win_amd64 }
- { os: windows-2025, python: '3.14', wheel: '' }
defaults:
run:
shell: bash
steps:
- name: Checkout EndKeep
uses: actions/checkout@v4
with:
fetch-depth: 0
persist-credentials: false
- name: Checkout upstream Amulet 3.0.7a0 at verified commit
if: matrix.python != '3.14'
uses: actions/checkout@v4
with:
repository: Amulet-Team/Amulet-LevelDB
ref: 736228e375e9f535b6522bd28168fc306a5354b9
path: amulet-src
fetch-depth: 0
persist-credentials: false
- name: Patch Amulet Python floor
if: matrix.python != '3.14'
run: |
python - <<'PY'
from pathlib import Path
project = Path("amulet-src/pyproject.toml")
content = project.read_text()
needle = 'requires-python = ">=3.14"'
assert content.count(needle) == 1
project.write_text(content.replace(needle, 'requires-python = ">=3.12"'))

PY
cp .github/scripts/amulet_leveldb_smoke.py amulet-src/tests/test_backport_smoke.py
git -C amulet-src diff --check
- name: Setup Python
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python }}
- name: Install cibuildwheel
if: matrix.python != '3.14'
run: python -m pip install 'cibuildwheel~=4.1'
- name: Build and test Amulet wheel
if: matrix.python != '3.14'
env:
CIBW_BEFORE_BUILD: 'python -m pip install "cmake>=4.1,<5"'
CIBW_BEFORE_TEST: 'python -m pip install "cmake>=4.1,<5"'
CIBW_TEST_COMMAND: 'python {project}/tests/test_backport_smoke.py && python {project}/tests/test_db.py'
CIBW_TEST_SOURCES: tests
run: |
mkdir -p amulet-dist
cd amulet-src
python -m cibuildwheel --only '${{ matrix.wheel }}' --output-dir ../amulet-dist .
- name: Setup uv
uses: astral-sh/setup-uv@v7
with:
enable-cache: true
- name: Test installed EndKeep with matched Amulet
run: |
uv venv .venv --python '${{ matrix.python }}'
if [[ "${{ runner.os }}" == "Windows" ]]; then
PYTHON=.venv/Scripts/python.exe
else
PYTHON=.venv/bin/python
fi
# Fresh dependency resolution: do NOT seed the local Amulet build.
# This must fetch the exact published GitHub Release Wheel on 3.12/3.13.
uv pip install --python "$PYTHON" -e '.[dev]'
"$PYTHON" - <<'PY'
from importlib.metadata import distribution
from packaging.requirements import Requirement
requires = distribution("endstone-endkeep").requires or []
amulet = [Requirement(req) for req in requires if Requirement(req).name == "amulet-leveldb"]
assert len(amulet) == 5, [str(req) for req in amulet]
print("Packaged Amulet selectors:", *amulet, sep="\\n")
PY
uv pip check --python "$PYTHON"
"$PYTHON" - <<'PY'
from importlib.metadata import version
from amulet.leveldb import LevelDB
assert version("amulet-leveldb").split("+")[0] == "3.0.7a0"
print("Amulet:", version("amulet-leveldb"))
PY
"$PYTHON" -m pytest -q
- name: Save tested experimental backports (no Release)
if: matrix.python != '3.14'
uses: actions/upload-artifact@v4
with:
name: amulet-compat-${{ matrix.os }}-${{ matrix.python }}
path: amulet-dist/*.whl
if-no-files-found: error
retention-days: 7

verified-backports:
name: Collect verified CPython 3.12/3.13 Wheels
needs: full-matrix
runs-on: ubuntu-24.04
steps:
- name: Download four tested wheel artifacts
uses: actions/download-artifact@v4
with:
pattern: amulet-compat-*
path: dist
merge-multiple: true

- name: Validate Wheels and create SHA256 manifest
run: |
python - <<'PY'
from hashlib import sha256
from pathlib import Path
from zipfile import ZipFile

root = Path("dist")
wheels = sorted(root.glob("*.whl"))
assert len(wheels) == 4, [p.name for p in wheels]
for abi in ("cp312", "cp313"):
for platform in ("manylinux_", "win_amd64"):
matches = [
p for p in wheels
if f"-{abi}-{abi}-" in p.name and platform in p.name
]
assert len(matches) == 1, (abi, platform, [p.name for p in wheels])
for wheel in wheels:
with ZipFile(wheel) as archive:
meta = next(path for path in archive.namelist() if path.endswith(".dist-info/METADATA"))
data = archive.read(meta).decode("utf-8")
assert "Name: amulet-leveldb" in data
assert "Version: 3.0.7a0+0.g736228e.dirty" in data
assert "Requires-Python: >=3.12" in data
assert any(path.endswith(".dist-info/licenses/LICENSE") for path in archive.namelist())
(root / "SHA256SUMS").write_text("".join(
f"{sha256(path.read_bytes()).hexdigest()} {path.name}\n"
for path in wheels
))
print((root / "SHA256SUMS").read_text())
PY

- name: Upload verified wheel set for review (not a GitHub Release)
uses: actions/upload-artifact@v4
with:
name: amulet-backports-verified-four-wheels
path: |
dist/*.whl
dist/SHA256SUMS
if-no-files-found: error
retention-days: 14
20 changes: 14 additions & 6 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
# Working on EndKeep

EndKeep is a Python plugin for Endstone that creates incremental backups of
Minecraft Bedrock worlds. This repository targets Linux, Python 3.14 and
Endstone 0.11. For Endstone API usage, see [Endstone's documentation](https://endstone.dev/).
Minecraft Bedrock worlds. EndKeep targets Python 3.12–3.14
on Linux and Windows x86-64 with Endstone 0.11. For Endstone API usage, see
[Endstone's documentation](https://endstone.dev/).

## Project layout

Expand All @@ -16,10 +17,11 @@ Endstone 0.11. For Endstone API usage, see [Endstone's documentation](https://en
## Development

```sh
uv sync --python 3.14 --extra dev
uv run ruff check src tests tools
uv run ruff format --check src tests tools
uv run pytest
uv venv --python 3.14
uv pip install --python .venv -e ".[dev]"
uv run --no-sync ruff check src tests tools
uv run --no-sync ruff format --check src tests tools
uv run --no-sync pytest
```

## Important constraints
Expand All @@ -34,6 +36,12 @@ uv run pytest
- Update user documentation for visible behavior changes. Explain what server
owners need to do; keep internal implementation notes out of usage guides.
- Do not create releases or rewrite version tags unless explicitly requested.
- Python 3.12/3.13 resolves pinned, SHA256-verified Amulet-LevelDB compatibility
Wheels from the dedicated GitHub Release. Test a clean EndKeep install without
pre-installing the dependency. The upstream Amulet version metadata currently prevents regenerating a
universal uv.lock; prefer `uv pip install` instead of `uv sync`.
- The six-combination compatibility workflow is not a substitute for
testing real BDS worlds, especially NTFS crash and directory metadata durability.

See [Contributing](CONTRIBUTING.md) and the
[release checks](docs/manual-validation.md) for the development workflow.
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,20 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

## [Unreleased]

### Added
- Support Python 3.12–3.14 on Linux and Windows x86-64.
- Automatically resolve pinned Amulet-LevelDB compatibility Wheels for Python 3.12/3.13.
- Add cross-platform CI and real BDS tests through bds-test-lab.

### Fixed
- Preserve binary LevelDB and sidecar data on Windows during backup and recovery.
- Improve worker process handling, path safety and filesystem compatibility.
- Exclude non-EndKeep tags from VCS version generation.

### Documentation
- Update installation, recovery and platform validation guides.


## [0.2.0] - 2026-10-09

### Added
Expand Down
15 changes: 10 additions & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,17 @@

Bug reports, fixes and documentation improvements are welcome.

EndKeep targets Linux, Python 3.14 and Endstone 0.11. To work on the project:
EndKeep supports Linux and Windows x86-64, Python 3.12–3.14 and Endstone 0.11.
To work on the project:

```sh
git clone https://github.com/ReallocAll/endkeep.git
cd endkeep
uv sync --python 3.14 --extra dev
uv run ruff check src tests tools
uv run ruff format --check src tests tools
uv run pytest
uv venv --python 3.14
uv pip install --python .venv -e ".[dev]"
uv run --no-sync ruff check src tests tools
uv run --no-sync ruff format --check src tests tools
uv run --no-sync pytest
```

Open a pull request with a short explanation of the change and tests for
Expand All @@ -19,3 +21,6 @@ on a disposable repository; see [Release checks](docs/manual-validation.md).

For bugs, use [GitHub Issues](https://github.com/ReallocAll/endkeep/issues).
For security reports, see [SECURITY.md](SECURITY.md).

The upstream Amulet package currently contains version metadata rejected by
some `uv lock` versions. Use `uv pip install` as above for development.
11 changes: 9 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,20 @@ without storing a complete world copy each time, and lets you restore an earlier

## Installation

Requires **Linux**, **Python 3.14** and **Endstone 0.11**. Tested on Ubuntu 26.04 LTS (x86-64).
Requires **Linux or Windows x86-64**, **Python 3.12–3.14** and **Endstone 0.11**.
Verified on BDS 1.26.52.3 / Endstone 0.11.13 on Linux and Windows.
Python 3.12/3.13 automatically install SHA-256-pinned, unofficial
Amulet-LevelDB compatibility Wheels from the
[compatibility Release](https://github.com/ReallocAll/endkeep/releases/tag/amulet-leveldb-3.0.7a0-compat-cp312-cp313); Python 3.14
uses the upstream PyPI package. GitHub Releases and PyPI must be reachable
for the initial installation.

1. Download the plugin `.whl` from [Releases](https://github.com/ReallocAll/endkeep/releases/latest).
2. Place it in your server's `plugins/` directory.
3. Restart the server. Endstone installs the required Python packages automatically.

The server needs internet access and compatible dependency packages for the initial installation.
On Windows, sudden-power-loss durability may differ from POSIX directory fsync.
Keep an independent off-server copy of backups.
See [Using EndKeep](docs/using-endkeep.md) for details.

## Quick start
Expand Down
44 changes: 42 additions & 2 deletions docs/manual-validation.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,13 @@
These checks complement automated tests. Perform them with a release candidate
on a test world before publishing a version.

**Validated environment:** Ubuntu 26.04 LTS x86-64, Python 3.14, Endstone 0.11.
The automated CI runner is not a substitute for testing a real Bedrock server.
**Automated validation:** Python 3.12/3.13/3.14 on Linux x86-64 and Windows
x64; [six-combination CI](https://github.com/ReallocAll/endkeep/actions/runs/37976573011).
Real BDS 1.26.52.3 / Endstone 0.11.13 with Python 3.14 passed on both platforms
in [bds-test-lab](https://github.com/ReallocAll/bds-test-lab/actions/runs/37977425842):
two online snapshots, maintenance, deep verify, export, graceful BDS stop,
explicit authenticated worker shutdown and offline restore. The tests do
not establish equivalent NTFS/POSIX sudden-power-loss guarantees.

## Installation and backup

Expand All @@ -29,3 +34,38 @@ The automated CI runner is not a substitute for testing a real Bedrock server.
- Check that disabling bStats does not affect backup operation.

Keep these tests on disposable worlds or independent copies of backup data.

## Cross-platform release acceptance (Python 3.12–3.14)

The Python 3.14 smoke scenarios were completed on disposable Linux and Windows
BDS worlds. Repeat the following when validating additional server versions,
large repositories or production configurations.

1. Test the exact published plugin Wheel and a clean installation on Endstone
0.11 for Python 3.12/3.13/3.14. The 3.12/3.13 compatibility Wheels of
Amulet-LevelDB must be publicly retrievable and their checksums verified;
EndKeep uses the fixed compatibility Release rather than expiring CI artifacts.
2. With an isolated BDS world, exercise `save hold`, `save query`, `save resume`,
and two snapshots containing LevelDB keys with binary data. Verify no tick
stall or held-save state persists after a failed capture.
3. Capture BASE and at least two DELTAs. Compare each exported state's records,
world files and SHA-256 with the original world at that recovery point.
4. Test delete-middle, rollover, GC, `verify deep`, and offline restore to a new
directory. Load the restored disposable world in BDS.
5. Crash a disposable backup worker while writing an incoming object, after an
object rename, after manifest publication and while updating HEAD. Restart,
verify the correct recovery point survives, and confirm that suspicious
repository states block destructive operations.
6. On Windows, separately test NTFS file locking/sharing, open LevelDB handles,
reparse points/junctions and interrupted file/directory rename sequences.
Windows' CRT does not provide POSIX directory-fsync behavior; passing unit
tests is not evidence of identical power-failure durability guarantees.

The repository worker intentionally stays alive across plugin reloads.
The headless lab explicitly shuts down the authenticated worker after
stopping BDS so the process tree is clean; this is separate from checking
whether BDS itself stops normally.

Sudden power failure cannot be simulated by clean shutdown. The Windows
NTFS directory-metadata durability guarantees remain different from POSIX
directory fsync; test these separately before making stronger claims.
Loading
Loading