fix(engine): preserve composition root background in alpha exports - #3976
Open
miga-heygen wants to merge 3 commits into
Open
miga-heygen wants to merge 3 commits into
miga-heygen wants to merge 3 commits into
Conversation
initTransparentBackground() forced html, body, AND any [data-composition-id] element (the composition root, or a nested sub-composition root) to transparent for every alpha-carrying capture session. That's correct for the HDR two-pass layered DOM pass, where the HDR video itself is the backdrop — but for a plain alpha export (MOV ProRes4444, webm+alpha) of a normal composition, an author's own background painted on the root is real, intentional content, and it was being silently dropped. Add CaptureOptions.clearCompositionRootBackground, threaded through session.options into every general-session call site (default false — preserve the root's authored background). The two HDR-layered-DOM- session call sites set it to true, keeping their existing (correct) behavior. initTransparentBackground's injected stylesheet is now idempotently rewritten rather than insert-once/skip, so a caller's choice is never silently ignored by an earlier call on the same page. Add a producer test fixture + transparency-test.ts check proving a composition root's authored background survives into alpha output, plus unit tests for both flag values and the rewrite-on-repeat-call behavior directly. Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
…planner transparency-regression-root-bg is exercised by transparency-test.ts directly (asserts a real alpha channel), not by the sharded regression harness, same as its sibling transparency-regression. The shard planner hard-fails on any fixture directory it doesn't recognize as either scheduled or explicitly excluded, so register it in the excluded list with the same reason as its sibling. Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
…ered sessions The previous commit removed the two HDR-layered-DOM-session call sites' own explicit initTransparentBackground() call, relying on initializeSession()'s internal call (gated behind session.options.format === "png") plus a new clearCompositionRootBackground session option. That was wrong: a plain HDR render's own DOM session captures "jpeg", not "png" — the final HDR MP4/HEVC output doesn't itself carry an alpha channel, only the layered-compositing step does. So initializeSession()'s internal call never fired at all for this common case, and the HDR video layer never got a transparent DOM backdrop to composite into. Reproduced directly against the hdr-regression fixture (part of the regression-shards CI suite): a real local render showed a severe PSNR collapse (~8-56dB) in the "Window B: z-order sandwich" segment, where a DOM element behind the HDR video, the video itself, and a translucent DOM overlay in front all need correct z-ordering — with the composition root's own background no longer cleared, it painted solid over the video slot instead of letting the video composite through. Confirmed via bisection: reverting to the prior commit's files reproduces 0 failed frames (passing); the removed-call version reproduces the exact failure; restoring the explicit calls fixes it again (100/100 checkpoints, 0 failed frames). Restore the explicit, unconditional call in both HDR call sites, right after initializeSession(). clearCompositionRootBackground still does its job for the general/plain-alpha-export path (frameCapture.ts), which is unaffected by this fix. Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
initTransparentBackground()sets up the transparent background for any capture session that needs an alpha channel (format: "png"— MOV ProRes4444, webm+alpha, PNG sequences, and the HDR two-pass layered DOM pass). It works by injecting a stylesheet, since Chrome'sEmulation.setDefaultBackgroundColorOverrideonly replaces the page's default background and doesn't override an author's own CSS.That stylesheet forced
html,body, and any[data-composition-id]element (the composition root, or a nested sub-composition root) to transparent, unconditionally. That's correct for the HDR two-pass layered DOM pass — the HDR video itself is the backdrop there, so the DOM layer must contribute only foreground pixels. But for a plain alpha export of a normal composition, an author's own background painted on the composition root is real, intentional content — and it was being silently dropped, even for a simple top-level composition with no nesting at all.Reproduced with a standalone A/B render: a root
<div data-composition-id>withbackground: rgb(0,0,255)renders correctly under--format mp4(opaque, no alpha needed) but decodes as fully transparent under an alpha-carrying format.Fix
initTransparentBackground(page, options)now takes an explicit{ clearCompositionRoot: boolean }instead of always clearing everything.falseclears onlyhtml/body(page-chrome defaults);truealso clears every[data-composition-id]element.CaptureOptions.clearCompositionRootBackgroundso each session states its own intent once, at construction, rather than the decision being re-derived or passed around separately. The engine's general session-init path reads this option (defaulting tofalse— preserve the root's authored background) at all 4 of its call sites.<style>element is now idempotently rewritten on a repeat call instead of insert-once/skip-if-present, so a session's chosen behavior can never be silently ignored by an earlier call that ran with a different value.clearCompositionRootBackground: true, and also callinitTransparentBackground()explicitly right after session init (see "second bug" below for why the explicit call is still required there).Second bug found and fixed in this same PR
An earlier version of this fix assumed
clearCompositionRootBackground: truealone was enough for the two HDR-layered-DOM-session call sites, and dropped their own explicitinitTransparentBackground()call, relying entirely on the engine's internal call insideinitializeSession(). That internal call only fires when the session's own capture format is"png"— but a plain HDR render's DOM session captures"jpeg"(the final HDR MP4/HEVC output doesn't itself carry an alpha channel; only the layered-compositing step needs one), so the internal call never fired at all, and the HDR video layer never got a transparent DOM backdrop to composite into.This reproduced as a real, severe regression against
hdr-regression, a fixture in this repo's standard CI regression suite: a "z-order sandwich" window (DOM element behind an HDR video, the video itself, a translucent DOM element in front) collapsed to ~8-56dB PSNR in the affected time range, because the composition root's own background painted solid over the video slot instead of letting the video composite through. Confirmed via bisection against a real local render (not just CI logs): the prior commit's files reproduce 0 failed frames; the version with the explicit call removed reproduces the exact failure; restoring the explicit call fixes it again (100/100 checkpoints, 0 failed frames, audio unaffected).Fix: restored the explicit, unconditional
initTransparentBackground()call in both HDR call sites, right afterinitializeSession().clearCompositionRootBackgroundstill does its job for the general/plain-alpha-export path, which this second fix doesn't touch.Test plan
transparency-regression-root-bg) + a new check intransparency-test.ts: renders a composition whose root div paints a full-frame blue background over a red card, asserts the blue survives (opaque, not transparent) in webm alpha output while the red card still renders correctly. Verified with a real revert-and-restore.screenshotService.test.tscovering both flag values directly, plus the rewrite-on-repeat-call behavior.hdr-regressionfixture, already part of theregression-shardsCI job — confirmed via direct local bisection (fetched the fixture's real LFS media and rendered it three ways: pre-fix baseline passes, broken-intermediate-version fails identically to CI, final fix passes 100/100 checkpoints).packages/engine:screenshotService.test.ts(35/35) andframeCapture.test.ts(28/28) pass.tsc --noEmit,oxlint,oxfmt --checkclean on all touched files.packages/producer:tsc --noEmitclean. The fulltransparency-test.tsscript passes for webm, the new root-background check, gif, and png-sequence; its one remaining check (a GIF-vs-golden-MP4 shader-transition comparison) fails in this environment only because that goldenoutput.mp4fixture is an unfetched git-LFS pointer here — unrelated to this diff.