Make the timing tests independent of sleep accuracy - #53
Merged
Merged
Conversation
The timeout tests counted items against real sleeps and left 10 to 40 ms of slack. A sleep only promises to take at least as long as requested, so the counts changed on a busy machine and on a coarse clock: - Blocking sleeps that overshoot by 40 ms or more made timeout_generator yield one item fewer, in five test cases and in its doctest. - A 15.6 ms event loop clock resolution, the Windows default, let the 0.05 s timeout fire together with a 0.04 s sleep, so the detector tests stopped at 3 instead of 4. The sync tests and the doctest now run on a fake clock that only moves when it is slept on, and they check the requested sleeps as well. The total timeout tests advance the same clock. The per-item timeout tests yield without waiting and then stall for 10 s against a 0.05 s timeout. One test stays on the real clock and only checks what holds for any sleep accuracy. The fixtures are loaded from a conftest.py in the repository root so the doctests can use them, and the sdist ships that file.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
| root because the doctests in ``python_utils`` need them as well. | ||
| """ | ||
|
|
||
| pytest_plugins: tuple[str, ...] = ('_python_utils_tests.clock',) |
This was referenced Oct 2, 2026
test_aio_timeout_generator still counted items against real sleeps. The case with five sleeps of 0.06 s against a 0.3 s timeout ends one item short as soon as the sleeps run 15 ms late in total. It failed 3 of 25 runs on a busy machine, and fails every time when asyncio.sleep is made 20 ms late. The test now lets asyncio.sleep advance the fake clock. The default iterable test in test_lazy_imports uses the fake clock too, so its 0.05 s timeout cannot end the loop before the second item.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
The timeout tests count items against real sleeps and leave 10 to 40 ms of slack. A sleep only promises to take at least as long as requested, so the counts change with the machine.
Three failures, three measurements:
time.sleep(0.06)took 0.11 s at the median, whileasyncio.sleep(0.06)took 0.0604 s.timeout_generator(0.1, 0.06)then yields 2 items where the test and the doctest expect 3. Five cases oftest_timeout_generatorand thetimeout_generatordoctest failed in 5 out of 5 runs.loop._clock_resolution = 0.0156reproducesi == 3instead of4intest_aio_generator_timeout_detector_decorator_clean_exiton every run, which is the failure reported on Windows in Fix UniqueList membership after indexed replacement #51.test_aio_timeout_generatorexpects exactly five items from five sleeps of 0.06 s against a 0.3 s timeout. It failed 3 of 25 runs on a busy machine, and fails every time whenasyncio.sleepis made 20 ms late.The change
No library code changes.
timeout_generatortests and the doctest run on a fake clock that only moves when it is slept on. The tests now check the requested sleeps too, sointerval_multiplierandmaximum_intervalare verified instead of implied.test_aio_timeout_generatorletsasyncio.sleepadvance the same fake clock._python_utils_tests/clock.pyand are loaded from a rootconftest.py, because the doctests inpython_utilsneed them. The sdist shipsconftest.py.This follows the fake clock that
test_abatcher_timedalready uses.Verification
timeout_generatorare caught: one without the interval multiplier, one that stops an item late.