Skip to content

Asset::exists() loads the entire container listing to check one file #15607

Description

@edalzell

Bug description

Asset::exists() (src/Assets/Asset.php:224) is $this->container()->files()->contains($path). files() resolves through AssetContainerContents::all(), which reads the whole cached recursive listing of the container into memory. AssetRepository::findById() → AssetContainer::asset() calls exists() on every lookup, so resolving a single asset reference on the front end costs a full listing load.

On a large S3 container this is fatal. One production site has a private container with 197,603 files; the cached listing (asset-list-contents-<handle>) is an 80 MB serialized string in Redis, and unserializing it pushes a PHP-FPM worker past a 1 GB memory_limit:

Allowed memory size of 1073741824 bytes exhausted (tried to allocate 80320709 bytes) at vendor/laravel/framework/src/Illuminate/Redis/Connections/Connection.php:124

Any page that renders an asset from that container pays this, not just the CP.

How to reproduce

  1. Container on an S3 disk with a large number of files (tens of thousands is enough to see it; 197k reproduces the fatal at 1 GB).
  2. STATAMIC_STACHE_FILE_WATCHER=false so the listing is cached forever.
  3. Asset::find('private::some/file.jpg') from a front-end template or tinker and watch memory: the whole listing is loaded to answer one contains().

Logs

See above.

Environment

Statamic 6.35.0, Laravel 11, PHP 8.3, Redis cache, S3 (Flysystem v3) container, cache_meta true.

Installation

Existing Laravel app

Additional details

Suggested fix: make exists() answer without the listing when it can.

  • If cache_meta is on and cacheStore()->has($this->metaCacheKey()) (meta cache is written only for assets that exist), return true without touching the listing.
  • If the listing is already loaded in memory for this request (AssetContainerContents::$files is set, or Blink has it), use it as today.
  • Otherwise fall back to a single $this->disk()->exists($path) (one S3 HEAD) instead of loading the listing.

The same pattern applies to metaExists() (src/Assets/Asset.php:316), which calls container()->metaFiles() and so also loads the full listing. A disk check on metaPath() works there too.

Tests to add in tests/Assets/AssetTest.php: exists() returns true from the meta cache without calling files(); falls back to the disk check when nothing is cached; still returns false for a missing path. Related: #11998 (meta as content) and a separate issue for excluding .meta files from the listing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions