feat: improve Instant and Chrome audio using existing local outputs by richiemcilroy · Pull Request #2242 · CapSoftware/Cap

richiemcilroy · 2026-09-08T11:19:25Z

Quiet desktop Instant and Chrome recordings can receive bounded level correction after their original MP4 is published. The existing segment mux or WebM conversion supplies its local output, avoiding another source download or media job. Video packets are copied and verified; only audio is re-encoded.

A newly reported recording exposed limits that would have left very quiet audio unchanged. The worker now accepts valid level measurements down to -55 LUFS and progressively raises quieter recordings toward -22 LUFS, capped at 28 dB and available peak headroom. Recordings at or above -34 LUFS retain the previous planned gain. Encoded output must remain below -1 dBTP. Web publication enforces the updated range and gain bound too.

Audio and video may end independently. Correction preserves each original timeline and checks source/output audio start, duration, sample count, video packets and terminal packet timing. Both track durations remain subject to the processing limit. EQ and denoising remain disabled.

Retain the original output and verification receipt. Select an immutable derivative only after audio/timing/packet checks, uploaded-byte verification and a signed callback that rechecks ownership, storage, source identity and upload state under a row lock.
Cover desktop segment finalization and browser conversion. Playback, downloads, copies, edits, replacement and reprocessing handle selected derivatives and clear stale selection.
Bound correction to one worker per replica, spare processing capacity, two minutes, 15-minute recordings and 256 MiB sources. Capacity, cancellation, timeout, unsupported input/storage or validation failure retain the original. No desktop capture change or historical backfill.
Reuse the local source without another video encode or candidate video decode. Additional transfers are the derivative upload, one full stored-output verification read and two one-byte identity checks, directly between worker and storage.

Validation on the updated implementation:

427 tests passed while building the Linux production image, including media preservation and large-transfer performance gates. Full web project typecheck, scoped media typecheck, touched-code Biome and 202 publication/storage/finalization web tests passed.
The same 60 audio samples produced 42 validated corrections and 18 unchanged originals, with no rejected outputs or previously accepted files becoming ineligible. Every decoded sample count was preserved; duration changes were at most 21 ms and the highest encoded true peak was -1.95 dBTP.
All 42 accepted samples plus the reported recording passed signal comparison: minimum STOI similarity 0.9924, median 0.99986, minimum SI-SDR 25.93 dB and zero measured lag. These compare encoded output with its original, not clean speech ground truth. The reused tuning/holdout cohort is not a new independent holdout.
The exact production-image worker processed 21 complete production MP4s offline: four corrected outputs, seventeen unchanged originals, every source hash preserved and zero source downloads. With two CPUs and 2 GiB, the run took 32.8 seconds, peaked at 1.06 GiB container memory and recorded no CPU throttling. Storage was mocked, so this does not measure production network latency.
The reported 13.93-second recording improved from -50.48 to -22.49 LUFS, with -3.04 dBTP peaks, unchanged audio duration/sample count/LRA and verified video packets. Its local +28 dB comparison received positive listening feedback. Publication regression tests failed under the old bounds and now verify that these measurements select the derivative for playback.

Constant gain also raises existing background noise. These results establish bounded technical compatibility and level improvement, not universal perceptual improvement or a zero-regression guarantee. Legacy/direct MP4 finalization paths, recordings outside the limits and unsupported storage keep their originals. Clearing audio selection fields restores original playback. No database DDL migration is required.

Detailed methodology and limits: audio-quality-benchmarks.md.

Storage verification responsibility: the authenticated worker SHA-256 hashes a full conditional GET of the stored derivative, compares it to the locally validated output, and rechecks strong ETag/size before and after that read. The publication payload now uses the verifier's computed remoteSha256 directly. The web transaction pins that already-verified identity; it does not treat ETag as a cryptographic digest or repeat the full download. Tests assert If-Match on storage reads, the computed digest in publication, rejection of corrupted bytes with unchanged ETag/size, and no extra source read. The focused Linux worker and web publication tests passed again after this clarification.

Greptile Summary

This PR publishes bounded audio-level derivatives for eligible Instant and Chrome recordings while retaining the original media and reusing already-local outputs.

Adds constrained audio correction with independent audio/video timeline validation.
Verifies uploaded derivative bytes before atomically selecting them for playback.
Integrates derivative resolution and invalidation across publication, storage, copying, editing, replacement, and reprocessing.
Adds production-image, media integration, publication, lifecycle, and duration-boundary coverage.
The latest revision publishes the digest calculated from the verified stored bytes.

Confidence Score: 5/5

The PR appears safe to merge, with no outstanding correctness, security, or repository-rule issue identified.

The latest stored-digest change remains bound to the locally validated candidate through a complete conditional read and rejects mismatched bytes before publication. The earlier duration-test finding was manually resolved, and the current tests cover eligible 900-second inputs as well as values above the limit.