Skip to content

perf(@angular/build): verify cache metadata before reading file and eliminate sqlite read-locks - #34042

Merged
alan-agius4 merged 1 commit into
angular:mainfrom
alan-agius4:perf/persistent-cache-metadata-verification
Sep 8, 2026
Merged

perf(@angular/build): verify cache metadata before reading file and eliminate sqlite read-locks#34042
alan-agius4 merged 1 commit into
angular:mainfrom
alan-agius4:perf/persistent-cache-metadata-verification

Conversation

@alan-agius4

Copy link
Copy Markdown
Collaborator

Summary

When using persistent caching in @angular/build, PersistentLoadResultCache.get() previously read the entire target file from disk via readFile() and computed a SHA-256 hash before querying the persistent L2 store.

This introduced two primary bottlenecks:

  1. Redundant Disk I/O & Hashing: On cold cache misses, every project file was read and hashed by the cache, found to be absent in the store, and then read from disk a second time by esbuild. On warm cache hits, reading and hashing multi-megabyte JavaScript files from disk was redundant when a fast stat() check (mtimeMs + size) would confirm whether the cached entry remains valid.
  2. SQLite Read-Lock Contention: In SqliteCacheStore, #queueAccessUpdate() triggered a synchronous BEGIN IMMEDIATE TRANSACTION; write transaction every 100 cache reads to update last_accessed. In multi-process worker pools and parallel builds, this converted concurrent read operations into serialized write transactions that blocked on SQLite busy timeouts.

Optimizations

  • Metadata Verification Before Read: Cache keys are now derived from the global configuration hash and path, querying the persistent store before performing disk file reads.
  • Watch Files & Target Metadata: Records target file metadata alongside dependency watch files in watchFilesMetadata.
  • Fast-Path Metadata Validation: Validates cache hits using fast-path metadata comparison (mtimeMs and size) for both the target file and its dependencies, only reading content and hashing on disk if timestamps changed.
  • Deferred Access Updates: Defers SQLite last_accessed timestamp updates via unref'd timer and flushes batch updates on store close, preventing write locks from blocking concurrent cache reads.

Benchmark Results

In benchmarks on 309 project files:

  • Cold cache miss latency: dropped from 98.5 ms to 1.5 ms (65.2x faster).
  • Warm cache hit latency: dropped from 77.7 ms to 17.9 ms (4.35x faster).
  • Concurrent latency (4 worker processes): dropped from 143.6 ms to 17.5 ms (8.2x faster).

@angular-robot angular-robot Bot added area: performance Issues related to performance area: @angular/build labels Sep 8, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request simplifies the cache key calculation in PersistentLoadResultCache by removing the file content from the key generation, which avoids redundant disk reads when checking the L2 cache. Additionally, it adjusts the flushing behavior in SqliteCacheStore by removing the immediate flush threshold and increasing the timeout. The reviewer suggested simplifying and unifying the deduplication of watch files using a single Set initialization to prevent redundant operations even when filePath is undefined.

Comment thread packages/angular/build/src/tools/esbuild/persistent-load-result-cache.ts Outdated
…liminate sqlite read-locks

Previously, PersistentLoadResultCache.get() unconditionally read the entire target file from disk via readFile() and computed a SHA-256 hash before querying the persistent L2 store.

This introduced two primary bottlenecks:
1. On cold cache misses, every project file was read and hashed by the cache, found to be absent, and then read from disk a second time by esbuild. On warm cache hits, reading and hashing multi-megabyte JavaScript files from disk was redundant when a fast stat() check (mtimeMs + size) would confirm whether the cached entry remains valid.
2. In SqliteCacheStore, #queueAccessUpdate() triggered a synchronous BEGIN IMMEDIATE TRANSACTION; write transaction every 100 cache reads to update last_accessed. In multi-process worker pools and parallel builds, this converted concurrent read operations into serialized write transactions that blocked on SQLite busy timeouts.

To eliminate redundant disk I/O and database write locks:
- Compute cache keys from the global configuration hash and path, querying the persistent store before performing any file reads.
- Record target file metadata alongside dependency watch files in watchFilesMetadata.
- Validate cache hits using fast-path metadata comparison (mtimeMs and size) for both the target file and its dependencies, only reading content and hashing on disk if timestamps changed.
- Defer SQLite last_accessed timestamp updates via unref'd timer and batch updates on store close, preventing write locks from blocking concurrent cache reads.

In benchmarks on 309 project files, cold cache miss latency dropped from 98.5 ms to 1.5 ms (65.2x faster), warm cache hit latency dropped from 77.7 ms to 17.9 ms (4.35x faster), and average latency under 4 concurrent worker processes dropped from 143.6 ms to 17.5 ms (8.2x faster).
@alan-agius4 alan-agius4 added the target: minor This PR is targeted for the next minor release label Sep 8, 2026
@alan-agius4
alan-agius4 force-pushed the perf/persistent-cache-metadata-verification branch from 61ee772 to 57c2ceb Compare September 8, 2026 13:13
@alan-agius4
alan-agius4 requested a review from clydin September 8, 2026 14:56
@alan-agius4 alan-agius4 added the action: review The PR is still awaiting reviews from at least one requested reviewer label Sep 8, 2026
@alan-agius4 alan-agius4 added action: merge The PR is ready for merge by the caretaker and removed action: review The PR is still awaiting reviews from at least one requested reviewer labels Sep 8, 2026
@alan-agius4
alan-agius4 merged commit d11a663 into angular:main Sep 8, 2026
40 of 41 checks passed
@alan-agius4

Copy link
Copy Markdown
Collaborator Author

This PR was merged into the repository. The changes were merged into the following branches:

@alan-agius4
alan-agius4 deleted the perf/persistent-cache-metadata-verification branch September 8, 2026 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

action: merge The PR is ready for merge by the caretaker area: @angular/build area: performance Issues related to performance target: minor This PR is targeted for the next minor release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants