perf(@angular/build): verify cache metadata before reading file and eliminate sqlite read-locks - #34042
Merged
alan-agius4 merged 1 commit intoSep 8, 2026
Conversation
There was a problem hiding this comment.
Code Review
This pull request simplifies the cache key calculation in PersistentLoadResultCache by removing the file content from the key generation, which avoids redundant disk reads when checking the L2 cache. Additionally, it adjusts the flushing behavior in SqliteCacheStore by removing the immediate flush threshold and increasing the timeout. The reviewer suggested simplifying and unifying the deduplication of watch files using a single Set initialization to prevent redundant operations even when filePath is undefined.
…liminate sqlite read-locks Previously, PersistentLoadResultCache.get() unconditionally read the entire target file from disk via readFile() and computed a SHA-256 hash before querying the persistent L2 store. This introduced two primary bottlenecks: 1. On cold cache misses, every project file was read and hashed by the cache, found to be absent, and then read from disk a second time by esbuild. On warm cache hits, reading and hashing multi-megabyte JavaScript files from disk was redundant when a fast stat() check (mtimeMs + size) would confirm whether the cached entry remains valid. 2. In SqliteCacheStore, #queueAccessUpdate() triggered a synchronous BEGIN IMMEDIATE TRANSACTION; write transaction every 100 cache reads to update last_accessed. In multi-process worker pools and parallel builds, this converted concurrent read operations into serialized write transactions that blocked on SQLite busy timeouts. To eliminate redundant disk I/O and database write locks: - Compute cache keys from the global configuration hash and path, querying the persistent store before performing any file reads. - Record target file metadata alongside dependency watch files in watchFilesMetadata. - Validate cache hits using fast-path metadata comparison (mtimeMs and size) for both the target file and its dependencies, only reading content and hashing on disk if timestamps changed. - Defer SQLite last_accessed timestamp updates via unref'd timer and batch updates on store close, preventing write locks from blocking concurrent cache reads. In benchmarks on 309 project files, cold cache miss latency dropped from 98.5 ms to 1.5 ms (65.2x faster), warm cache hit latency dropped from 77.7 ms to 17.9 ms (4.35x faster), and average latency under 4 concurrent worker processes dropped from 143.6 ms to 17.5 ms (8.2x faster).
alan-agius4
force-pushed
the
perf/persistent-cache-metadata-verification
branch
from
September 8, 2026 13:13
61ee772 to
57c2ceb
Compare
clydin
approved these changes
Sep 8, 2026
Collaborator
Author
|
This PR was merged into the repository. The changes were merged into the following branches:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When using persistent caching in
@angular/build,PersistentLoadResultCache.get()previously read the entire target file from disk viareadFile()and computed a SHA-256 hash before querying the persistent L2 store.This introduced two primary bottlenecks:
stat()check (mtimeMs+size) would confirm whether the cached entry remains valid.SqliteCacheStore,#queueAccessUpdate()triggered a synchronousBEGIN IMMEDIATE TRANSACTION;write transaction every 100 cache reads to updatelast_accessed. In multi-process worker pools and parallel builds, this converted concurrent read operations into serialized write transactions that blocked on SQLite busy timeouts.Optimizations
watchFilesMetadata.mtimeMsandsize) for both the target file and its dependencies, only reading content and hashing on disk if timestamps changed.last_accessedtimestamp updates via unref'd timer and flushes batch updates on store close, preventing write locks from blocking concurrent cache reads.Benchmark Results
In benchmarks on 309 project files: