Repository navigation
Speed up dependency solver preparation - #803
Merged
Merged
Conversation
f-f
requested changes
Oct 11, 2026
Member
Author
|
@f-f addressed your comments and updated the text of the PR accordingly; I didn't change the implementation besides bringing in the test helpers, so your original review still stands. |
thomashoneyman
force-pushed
the
solver-performance-clean
branch
from
October 12, 2026 16:08
8d89bb4 to
0edfbc7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
While looking into #801, I started wondering if we've left easy solver performance improvements on the table. Indeed, we have! The previous PR made calculating the matrix cascade ~18x faster, and this PR makes solving an individual package considerably faster — anywhere from ~1.3x to ~80x faster. It doesn't require a lot of code; in fact, this is a small net reduction of the solver implementation, and the majority of the new code is improved tests and the benchmarking code.
In short, the solver previously found reachable versions by repeatedly merging dependency maps until they stopped changing. In this rework, we instead use a worklist, expanding each reachable package version once. It also waits until after pruning to initialize diagnostic data, and stops merging source information that would immediately be discarded.
Benchmarks
Solving the latest manifest's dependencies for every package in the most recent registry snapshot went from 82.92s to 37.62s, or 2.20× faster. All 1,176 complete plans were identical.
The version index here is 5,573 manifests from the registry, and each package's dependencies are solved independently and without a compiler constraint. Individual workloads improve by different amounts. Halogen's dependency solve went from 72.56ms to 10.68ms; a synthetic batch with a propagation-heavy, 60-package chain only improved from 242.91ms to 227.98ms. (This at least shows we aren't regressing — there is no case I could put together in which the new implementation is slower).
Here are some benchmark results, visually:
The charts are showing means from
minibenchfrom 40 solves per process, and two processes per variant. The whiskers in the chart are the two means. As stated before, the resulting plans are the same.The fixed snapshot uses registry revision d51cb8e and registry-index revision 23d1e62.