Skip to content

Fix device parametrization for optimizer benchmarks - #2092

Open
emme1t wants to merge 1 commit into
bitsandbytes-foundation:mainfrom
emme1t:fix/optimizer-benchmark-device
Open

emme1t wants to merge 1 commit into
bitsandbytes-foundation:mainfrom
emme1t:fix/optimizer-benchmark-device

Conversation

@emme1t

@emme1t emme1t commented Sep 22, 2026

Copy link
Copy Markdown

Fixes #2084.

test_benchmark_blockwise accepts device without parametrizing it, so selecting optimizer benchmarks fails during setup with 27 fixture 'device' not found errors. Parametrize it with the existing get_available_devices() helper and use the same CPU/Windows paged-optimizer skips as test_optimizer32bit.

Validation on Windows, Python 3.12.13 and PyTorch 2.14.0+cpu:

  • python -m pytest tests/test_optim.py -k test_benchmark_blockwise -m benchmark --setup-only -q: reproduced 27 setup errors on main; all 27 cases set up successfully with the patch.
  • Executed the existing benchmark functions with 64x64 tensors, one CPU thread and TORCHDYNAMO_DISABLE=1, retaining the 500-step loops: 12 passed, 15 skipped. This exercises the 8-bit state path at 4096 elements. Tensor sizes were adjusted by a local pytest collection plugin; the committed benchmark dimensions remain 4096x4096.
  • The same setup-only command with BNB_TEST_DEVICE=cuda successfully resolves the explicit device selection.
  • Changed-file pre-commit hooks and git diff --check pass.

FreeBSD and accelerator execution were not available locally. Full-size benchmark timings were not measured.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Several tests fail: fixture 'device' not found

1 participant