Repository navigation
docs: rewrite the performance page around the results - #638
Merged
Merged
Conversation
bench-report now prints an "At a glance" table, which reads each paired ratio as "Nx faster" or "Nx slower". It takes each rival from the table whose API matches it and rounds the unrounded median. Each rival cell in the detailed tables now shows that rival's time, with the ratio in brackets.
The release-by-release record of resolve-path changes, the 3.x-to-4.0 comparison table and the notes on each publication now live in docs/dev/performance-history.md, with links to the PRs and issues they cite.
The page now opens with where modern-di is faster and slower. It then gives the at-a-glance table and the regenerated tables (8845e6a, same machine and CPython build as the previous publication), followed by the scenarios, a single caveats list, a short explanation of the design and the reproduce steps. The history moved to dev/performance-history.md, which cuts the page from 4,590 to about 1,400 words.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Benchmark
Details
| Benchmark suite | Current: ff6a6e0 | Previous: 8845e6a | Ratio |
|---|---|---|---|
benchmarks/test_guard_by_type.py::test_g16_resolve_by_type |
3761263.9983491222 iter/sec (stddev: 1.9390687836176093e-8) |
4944432.734406896 iter/sec (stddev: 1.5433237190808405e-8) |
1.31 |
benchmarks/test_guard_by_type.py::test_g17_resolve_by_type_large_registry |
3873124.647571283 iter/sec (stddev: 1.2058416929287069e-8) |
4794010.478270802 iter/sec (stddev: 1.8872510916587298e-8) |
1.24 |
benchmarks/test_guard_cold.py::test_g8_cold_first_resolve |
22336.92577140693 iter/sec (stddev: 0.00004461584798119576) |
23379.907771025722 iter/sec (stddev: 0.00004251840467054203) |
1.05 |
benchmarks/test_guard_cold.py::test_g8b_cold_first_resolve_cached |
17736.878331368043 iter/sec (stddev: 0.00013807458571564506) |
18631.066883930645 iter/sec (stddev: 0.00012915608390466523) |
1.05 |
benchmarks/test_guard_concurrency.py::test_g14_concurrent_cached_hit[1] |
409.09830987067164 iter/sec (stddev: 0.00008043443515386216) |
645.6690271789628 iter/sec (stddev: 0.00008580711203073574) |
1.58 |
benchmarks/test_guard_concurrency.py::test_g14_concurrent_cached_hit[2] |
387.84670885169965 iter/sec (stddev: 0.00010941468416767338) |
640.6881767734664 iter/sec (stddev: 0.000052255992861963645) |
1.65 |
benchmarks/test_guard_concurrency.py::test_g14_concurrent_cached_hit[4] |
381.63047338775544 iter/sec (stddev: 0.00016196000193778395) |
574.207945832185 iter/sec (stddev: 0.00008621649216688135) |
1.50 |
benchmarks/test_guard_concurrency.py::test_g15_concurrent_first_resolve[1] |
13319.364280141459 iter/sec (stddev: 0.000008204183222723626) |
13603.602142867556 iter/sec (stddev: 0.000008748806155415457) |
1.02 |
benchmarks/test_guard_concurrency.py::test_g15_concurrent_first_resolve[2] |
6443.183763950714 iter/sec (stddev: 0.00001991764978743668) |
6442.502650136634 iter/sec (stddev: 0.00002158362719093651) |
1.00 |
benchmarks/test_guard_concurrency.py::test_g15_concurrent_first_resolve[4] |
3377.627065553969 iter/sec (stddev: 0.00004168634147843566) |
3291.2383039734004 iter/sec (stddev: 0.00005018183204744461) |
0.97 |
benchmarks/test_guard_concurrency.py::test_g15b_concurrent_first_resolve_sibling_children[1] |
9410.810338992665 iter/sec (stddev: 0.000009777578841198606) |
10373.874434649484 iter/sec (stddev: 0.000009255665511041216) |
1.10 |
benchmarks/test_guard_concurrency.py::test_g15b_concurrent_first_resolve_sibling_children[2] |
3868.4438942596917 iter/sec (stddev: 0.000026404066756041128) |
3923.3093715774007 iter/sec (stddev: 0.000027927028697243745) |
1.01 |
benchmarks/test_guard_concurrency.py::test_g15b_concurrent_first_resolve_sibling_children[4] |
1892.4311239440817 iter/sec (stddev: 0.00004268895045444918) |
1974.4755595668735 iter/sec (stddev: 0.000047472258999862) |
1.04 |
benchmarks/test_guard_concurrency.py::test_g15c_worker_pool_floor_control[1] |
41087.391512418995 iter/sec (stddev: 0.0000046331762229240955) |
30093.76464333532 iter/sec (stddev: 0.000004384203871752712) |
0.73 |
benchmarks/test_guard_concurrency.py::test_g15c_worker_pool_floor_control[2] |
13656.165075574057 iter/sec (stddev: 0.000011930249281922118) |
12464.966950187316 iter/sec (stddev: 0.000019308270324908097) |
0.91 |
benchmarks/test_guard_concurrency.py::test_g15c_worker_pool_floor_control[4] |
5839.939805799735 iter/sec (stddev: 0.000028502379014712098) |
5385.992541485295 iter/sec (stddev: 0.000039411866561451894) |
0.92 |
benchmarks/test_guard_lifecycle.py::test_g6_build_child_container |
1577599.3512949378 iter/sec (stddev: 4.008381978979722e-8) |
1904477.0494249293 iter/sec (stddev: 3.223495739376149e-8) |
1.21 |
benchmarks/test_guard_lifecycle.py::test_g6b_build_child_container_auto_scope |
1680066.2450117082 iter/sec (stddev: 2.9823986326332736e-8) |
2158513.923575061 iter/sec (stddev: 1.8048597318579156e-8) |
1.28 |
benchmarks/test_guard_lifecycle.py::test_g7_request_lifecycle_batch |
3002.9666706991034 iter/sec (stddev: 0.000017321650881778388) |
3071.4175219185095 iter/sec (stddev: 0.000010164944555708832) |
1.02 |
benchmarks/test_guard_lifecycle.py::test_g7c_event_loop_floor_control |
57509.204475695726 iter/sec (stddev: 0.0000020267658377197047) |
70739.47801436351 iter/sec (stddev: 0.0000014104827498090188) |
1.23 |
benchmarks/test_guard_lifecycle.py::test_g7b_request_cycle_sync |
379634.240252047 iter/sec (stddev: 6.728798197377859e-8) |
449247.02492486045 iter/sec (stddev: 9.734909639107606e-8) |
1.18 |
benchmarks/test_guard_lifecycle.py::test_g13_teardown_at_scale |
57805.38422325825 iter/sec (stddev: 0.0000018017497252879282) |
64327.70890705498 iter/sec (stddev: 0.0000014695393285292838) |
1.11 |
benchmarks/test_guard_lifecycle.py::test_g13b_teardown_at_scale_async_no_finalizers |
534.1441340042708 iter/sec (stddev: 0.0009957871812279182) |
590.8704902587542 iter/sec (stddev: 0.0011552607001117706) |
1.11 |
benchmarks/test_guard_resolve.py::test_g1_transient_resolve |
2092226.1662968134 iter/sec (stddev: 2.688456253951475e-8) |
2874717.64884099 iter/sec (stddev: 3.2376833554889374e-8) |
1.37 |
benchmarks/test_guard_resolve.py::test_g2_cached_resolve |
3161227.8689491814 iter/sec (stddev: 3.7000359048846516e-8) |
4842842.496634139 iter/sec (stddev: 1.733794528424132e-8) |
1.53 |
benchmarks/test_guard_resolve.py::test_g3_deep_chain |
679758.6945004497 iter/sec (stddev: 5.802016151919153e-8) |
936594.8688616627 iter/sec (stddev: 6.405220966673408e-8) |
1.38 |
benchmarks/test_guard_resolve.py::test_g4_wide_resolve |
422301.8627226056 iter/sec (stddev: 3.890334070477516e-7) |
541366.0799173635 iter/sec (stddev: 3.8151316686214023e-7) |
1.28 |
benchmarks/test_guard_resolve.py::test_g5_cross_scope |
1932813.0973599844 iter/sec (stddev: 2.8777642486449297e-8) |
2405069.3571518627 iter/sec (stddev: 3.406940432687613e-8) |
1.24 |
benchmarks/test_guard_resolve.py::test_g9_context_resolve |
1296494.774040306 iter/sec (stddev: 1.475936303506387e-7) |
1813361.0801576641 iter/sec (stddev: 1.5674780053189866e-7) |
1.40 |
benchmarks/test_guard_resolve.py::test_g12_override_active_resolve |
695872.6263594816 iter/sec (stddev: 6.036974515257598e-8) |
949960.3177650786 iter/sec (stddev: 7.690823713920681e-8) |
1.37 |
benchmarks/test_guard_resolve.py::test_g18_alias_hop |
3589098.601111863 iter/sec (stddev: 2.3725683424504437e-8) |
4969249.785424692 iter/sec (stddev: 1.574082646522229e-8) |
1.38 |
benchmarks/test_guard_validate.py::test_g10_validate_deep_chain |
31233.397172001965 iter/sec (stddev: 0.00023428110843988435) |
35884.53562137941 iter/sec (stddev: 0.000019160675538861134) |
1.15 |
benchmarks/test_guard_validate.py::test_g11_validate_wide |
20871.900692348103 iter/sec (stddev: 0.00002712480826395207) |
21094.67378196473 iter/sec (stddev: 0.000027522008242584267) |
1.01 |
This comment was automatically generated by workflow using github-action-benchmark.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The performance page had grown to 4,590 words, and most of it was optimization history. A reader had to get through about 900 words of method before the first number, and then read ratios such as "0.02" where "60x faster" was meant.
bench-reportnow prints an "At a glance" table before the detailed ones. It reads each paired ratio as "Nx faster", "Nx slower" or "level", takes each rival from the table whose API matches it, and rounds the unrounded median. On the detailed tables, each rival cell now shows that rival's time, with the ratio and its±in brackets.docs/dev/performance-history.md, in the nav under Development. It covers the release-by-release resolve-path changes with their guard-tier figures, the 3.x to 4.0 comparison table and the notes on each publication. PR and issue references are links, and the 3.1.1 reference-cycle story, which the old page told twice, appears once.docs/introduction/performance.mdis down to about 1,400 words. It opens with where modern-di is faster and where it is slower, then the tables, what each scenario measures, the caveats in one list, a short explanation of why the design is fast, and how to reproduce the numbers.Design decisions
8845e6a. The library code is unchanged sinceab327b3. Every ratio is within 0.04 of 4.0.0, and no cell crosses 1.0.±on each cell stays in the detailed tables. It is what the paired-ratio work exists for, and the summary table now serves readers who don't need it.xinstead of×, because ruff's RUF001-003 flag the multiplication sign.Test plan
just test: 1006 passed, including the new and updatedtests/test_bench_report.pycasesjust lint-cijust docs-build(strict)benchmarks/README.mdbuild_table()output from the raw JSON of the 5-runbench-report