Skip to content

[Native] MultiRenderTarget, reverse-Z clear, applyStates, OIT alpha modes#18568

Merged
bkaradzic-microsoft merged 6 commits into
masterfrom
native-multi-render-target
Jul 9, 2026
Merged

[Native] MultiRenderTarget, reverse-Z clear, applyStates, OIT alpha modes#18568
bkaradzic-microsoft merged 6 commits into
masterfrom
native-multi-render-target

Conversation

@bkaradzic-microsoft

@bkaradzic-microsoft bkaradzic-microsoft commented Jun 11, 2026

Copy link
Copy Markdown
Member

Paired native PR: BabylonJS/BabylonNative#1754

What

Foundational native-engine surface so MultiRenderTarget, the reverse depth buffer, and the
order-independent-transparency (depth-peeling) renderer stop dereferencing the null _gl context.
This removes several distinct crash classes. It does not by itself turn the MRT/OIT/FrameGraph
validation tests green — those need further work (see Follow-ups).

Changes (packages/dev/core/src/Engines)

  • thinNativeEngine.pure.ts
    • createMultipleRenderTarget + the MRT helper overrides: bindAttachments, buildTextureLayout,
      restoreSingleAttachment/restoreSingleAttachmentForRenderTarget, generateMipMapsMultiFramebuffer,
      resolveMultiFramebuffer, unBindMultiColorAttachmentFramebuffer,
      updateMultipleRenderTargetTextureSampleCount. bgfx writes every color attachment of the bound
      framebuffer, so the WebGL drawBuffers / MSAA-resolve plumbing becomes no-ops on Native. Reports
      drawBuffersExtension = true.
    • clear(): implement the reverse depth buffer (clear depth to 0 + GEQUAL) instead of throwing.
    • applyStates(): override so it flushes the depth-culling state through the native command path —
      the base implementation drives the null _gl directly, which crashed callers that mutate
      engine.depthCullingState then call applyStates() (e.g. the OIT depth-peeling renderer).
  • Native/nativeHelpers.ts / nativeInterfaces.ts: map alpha modes ALPHA_ONEONE_ONEONE and
    ALPHA_LAYER_ACCUMULATE to the native engine, and declare createMultiFrameBuffer.

Backward compatibility (feature detection)

createMultiFrameBuffer and the two OIT alpha constants only exist on a Babylon Native binary that
carries the paired #1754. They are declared optional on the native interface and feature-detected
at runtime, so this PR runs against the currently-published native without a protocol/version bump:

  • createMultiFrameBuffer absent → Logger.Warn + fall back to a single-attachment createFrameBuffer
    bound to the first color target.
  • ALPHA_ONEONE_ONEONE / ALPHA_LAYER_ACCUMULATE absent → warn once + fall back to ALPHA_ONEONE.

This is also what fixes the previously-red "Native tests (experimental)" CI job, which pulls the
BabylonNative master Playground (no #1754): before the guards it hit createMultiFrameBuffer is not a function / an undefined alpha mode, the harness exited on the first failure, and the step's retries
blew the job timeout. With the guards the MRT/OIT scenes degrade gracefully instead of crashing.

Paired native PR

Pairs with the BabylonNative C++ change (the shared CreateFrameBufferImpl framebuffer helper + the two
alpha blend modes). Thanks to the feature detection above, this JS PR can land before #1754.

Follow-ups (not in this PR)

  • OIT depth-peeling (thinDepthPeelingRenderer) still faults inside the D3D11 driver on submit
    (a multi-output / SRV↔RTV ping-pong state the driver rejects); needs interactive GPU debugging.
  • The blend equation (e.g. MAX) is not yet applied on Native (setAlphaEquation is a no-op),
    so OIT blending would still be incorrect once the crash is resolved.

Testing

  • nx run babylonjs:build (full tsc -b + rollup + declaration) and ESLint pass.
  • Against native with check animations all stop bug. #1754: built babylon.max.js and ran the BabylonNative Playground suite
    (D3D11). MRT, reverse-Z, and the depth-peeling setup no longer crash with COLOR_ATTACHMENT0 /
    reverse depth buffer / depthMask of undefined / Unsupported alpha mode. No regressions in
    existing 2D render-target tests.
  • Against native without check animations all stop bug. #1754: the feature-detection guards keep those same scenes from throwing —
    they warn and degrade to a single-attachment framebuffer + ALPHA_ONEONE.

Related PRs & landing order

These two are co-dependent. With the feature-detection guards, #18568 no longer hard-requires the
native side
and can merge independently:

  1. Babylon.js [Native] MultiRenderTarget, reverse-Z clear, applyStates, OIT alpha modes #18568 first — adds native-engine TS overrides, changes no WebGL behavior, re-enables
    no validation tests on its own, and degrades gracefully on native that predates check animations all stop bug. #1754.
  2. A babylonjs npm release ships that TS change.
  3. BabylonNative check animations all stop bug. #1754 last — after bumping the bundled babylonjs to that release. (This
    foundational pair does not yet turn the MRT/OIT/FrameGraph tests green — that is separate follow-up
    work.)

@bjsplat

bjsplat commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s).
To prevent this PR from going to the changelog marked it with the "skip changelog" label.

@bjsplat

bjsplat commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

Snapshot stored with reference name:
refs/pull/18568/merge

Test environment:
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/18568/merge/index.html

To test a playground add it to the URL, for example:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/18568/merge/index.html#WGZLGJ#4600

Links to test your changes to core in the published versions of the Babylon tools (does not contain changes you made to the tools themselves):

https://playground.babylonjs.com/?snapshot=refs/pull/18568/merge
https://sandbox.babylonjs.com/?snapshot=refs/pull/18568/merge
https://gui.babylonjs.com/?snapshot=refs/pull/18568/merge
https://nme.babylonjs.com/?snapshot=refs/pull/18568/merge

To test the snapshot in the playground with a playground ID add it after the snapshot query string:

https://playground.babylonjs.com/?snapshot=refs/pull/18568/merge#BCU1XR#0

If you made changes to the sandbox or playground in this PR, additional comments will be generated soon containing links to the dev versions of those tools.

@bjsplat

bjsplat commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

@bjsplat

bjsplat commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

🟢 Memory Leak Test Results

13 passed, 0 leaked out of 13 scenarios

🟢 All memory leak tests passed — no leaks detected.

Passed Scenarios (13)
Scenario Package
Core Feature Stack @babylonjs/core
Core Rendering Materials Shadows Stack @babylonjs/core
Core Textures Render Targets PostProcess Stack @babylonjs/core
GUI Fullscreen UI Controls @babylonjs/gui
GUI Mesh ADT Controls @babylonjs/gui
Loaders Boombox Import @babylonjs/loaders
Loaders OBJ Direct Load @babylonjs/loaders
Loaders STL Direct Load @babylonjs/loaders
Materials Library Stack @babylonjs/materials
Serializers glTF Export @babylonjs/serializers
Serializers GLB Export @babylonjs/serializers
PostProcesses Digital Rain Stack @babylonjs/post-processes
Procedural Textures Stack @babylonjs/procedural-textures

@bjsplat

bjsplat commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

@bjsplat

bjsplat commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

@bjsplat

bjsplat commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

⚡ Performance Test Results

🟢 All performance tests passed — no regressions detected.

@sebavan

sebavan commented Jul 3, 2026

Copy link
Copy Markdown
Member

@bkaradzic, any update on this one ?

@bkaradzic-microsoft
bkaradzic-microsoft force-pushed the native-multi-render-target branch from c5da3b6 to f4fdea7 Compare July 6, 2026 14:56
@bjsplat

bjsplat commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

@bjsplat

bjsplat commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

🟢 Memory Leak Test Results

4 passed, 0 leaked out of 4 scenarios

🟢 All memory leak tests passed — no leaks detected.

Passed Scenarios (4)
Scenario Package
Core Playground #2FDQT5#1508 @babylonjs/core
Core Playground #T90MQ4#14 @babylonjs/core
Core Playground #8EDB5N#2 @babylonjs/core
Core Playground #LL5BIQ#636 @babylonjs/core

@bjsplat

bjsplat commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

@bjsplat

bjsplat commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

@bjsplat

bjsplat commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

⚡ Performance Test Results

🟢 All performance tests passed — no regressions detected.

@bghgary bghgary left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Reviewed by Copilot on behalf of @bghgary]

Needs decoupling — the JS side hard-depends on native surface from #1754. Feature-detect instead; details inline.

Comment thread packages/dev/core/src/Engines/thinNativeEngine.pure.ts Outdated
Comment thread packages/dev/core/src/Engines/Native/nativeHelpers.ts Outdated
@bjsplat

bjsplat commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

@bjsplat

bjsplat commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

🟢 Memory Leak Test Results

4 passed, 0 leaked out of 4 scenarios

🟢 All memory leak tests passed — no leaks detected.

Passed Scenarios (4)
Scenario Package
Core Playground #2FDQT5#1508 @babylonjs/core
Core Playground #T90MQ4#14 @babylonjs/core
Core Playground #8EDB5N#2 @babylonjs/core
Core Playground #LL5BIQ#636 @babylonjs/core

@bghgary bghgary left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Reviewed by Copilot on behalf of @bghgary]

LGTM. The description is now stale, though: with the feature-detection this no longer requires #1754 (it warns and falls back on older native), so the "Paired native PR: Requires…" and "co-dependent / landing order" sections no longer apply — the two can land separately, and this can come out of draft.

@bjsplat

bjsplat commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

@bjsplat

bjsplat commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

@bjsplat

bjsplat commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

⚡ Performance Test Results

🟢 All performance tests passed — no regressions detected.

@bkaradzic-microsoft
bkaradzic-microsoft marked this pull request as ready for review July 8, 2026 22:58
Copilot AI review requested due to automatic review settings July 8, 2026 22:58
@bkaradzic-microsoft
bkaradzic-microsoft enabled auto-merge (squash) July 8, 2026 23:01
@bjsplat

bjsplat commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

@bjsplat

bjsplat commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

⚡ Performance Test Results

🟢 All performance tests passed — no regressions detected.

@bjsplat

bjsplat commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

Comment thread packages/dev/core/src/Engines/thinNativeEngine.pure.ts
@bjsplat

bjsplat commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

🟢 Memory Leak Test Results

4 passed, 0 leaked out of 4 scenarios

🟢 All memory leak tests passed — no leaks detected.

Passed Scenarios (4)
Scenario Package
Core Playground #2FDQT5#1508 @babylonjs/core
Core Playground #T90MQ4#14 @babylonjs/core
Core Playground #8EDB5N#2 @babylonjs/core
Core Playground #LL5BIQ#636 @babylonjs/core

@bjsplat

bjsplat commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

@bjsplat

bjsplat commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

@bjsplat

bjsplat commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

🟢 Memory Leak Test Results

4 passed, 0 leaked out of 4 scenarios

🟢 All memory leak tests passed — no leaks detected.

Passed Scenarios (4)
Scenario Package
Core Playground #2FDQT5#1508 @babylonjs/core
Core Playground #T90MQ4#14 @babylonjs/core
Core Playground #8EDB5N#2 @babylonjs/core
Core Playground #LL5BIQ#636 @babylonjs/core

@bjsplat

bjsplat commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

Comment thread packages/dev/core/src/Engines/thinNativeEngine.pure.ts
Comment thread packages/dev/core/src/Engines/thinNativeEngine.pure.ts
@bjsplat

bjsplat commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

⚡ Performance Test Results

🟢 All performance tests passed — no regressions detected.

@bjsplat

bjsplat commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

bkaradzic and others added 6 commits July 9, 2026 08:04
…odes

Foundational native-engine surface so MultiRenderTarget, the reverse depth
buffer, and the order-independent-transparency renderer stop dereferencing the
null _gl context. Removes several crash classes; does not by itself land the
dependent validation tests.

- createMultipleRenderTarget + the MRT helper overrides (bindAttachments,
  buildTextureLayout, restoreSingleAttachment[ForRenderTarget],
  generateMipMapsMultiFramebuffer, resolveMultiFramebuffer,
  unBindMultiColorAttachmentFramebuffer,
  updateMultipleRenderTargetTextureSampleCount). bgfx writes every color
  attachment of the bound framebuffer, so the WebGL drawBuffers / MSAA-resolve
  plumbing becomes no-ops on Native. Reports drawBuffersExtension = true.
- clear(): implement the reverse depth buffer (clear depth to 0 + GEQUAL)
  instead of throwing.
- applyStates(): override so it flushes the depth-culling state through the
  native command path (the base implementation drives the null _gl directly),
  fixing callers that mutate engine.depthCullingState then call applyStates()
  (e.g. the OIT depth-peeling renderer).
- Map alpha modes ALPHA_ONEONE_ONEONE and ALPHA_LAYER_ACCUMULATE to the native
  engine.

Pairs with the BabylonNative change (createMultiFrameBuffer + native alpha
modes). Known follow-ups: OIT depth-peeling still faults in the D3D11 driver on
submit; the blend equation (MAX) is not yet applied natively.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ative

Address review feedback: don't hard-break against Babylon Native binaries
that predate multi render target support (BabylonNative#1754).

- thinNativeEngine: guard createMultiFrameBuffer; when the native binary does
  not expose it, warn and fall back to a single-attachment framebuffer bound
  to the first color target so the scene keeps rendering.
- nativeHelpers.getNativeAlphaMode: ALPHA_ONEONE_ONEONE / ALPHA_LAYER_ACCUMULATE
  now warn once and fall back to ALPHA_ONEONE when the native constant is
  undefined, instead of silently returning undefined.
- nativeInterfaces: mark createMultiFrameBuffer and the two OIT alpha constants
  optional to reflect that older native does not provide them.

Keeps the PR mergeable against the currently published native binary and
unblocks the Native tests CI without requiring a coordinated protocol bump.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
clear() set depthCullingState.depthFunc = GEQUAL to mirror the WebGL
engine, but the native draw path never calls applyStates() before a draw
(only depth-test enable/disable is reconciled in _flushDepthTestState),
so the GEQUAL comparison never reached the backend and reverse-Z had no
effect. Also call setDepthFunction(GEQUAL) to encode the compare via the
native command path, mirroring the WebGPU engine's clear path
(setDepthFunctionToGreaterOrEqual).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
bgfx binds a fixed attachment set at framebuffer creation and cannot re-point an individual attachment the way GL's framebufferTexture2D can. When a multi render target attachment is replaced after creation (e.g. the OIT depth-peeling renderer swaps every attachment via MultiRenderTarget.setInternalTexture), the native framebuffer stayed bound to the original creation-time textures.

Extract the framebuffer build from createMultipleRenderTarget into _createMultiRenderTargetFramebuffer and call it from a NativeRenderTargetWrapper.setTexture override so the framebuffer is rebuilt from the current attachment set on every swap.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…esent

bgfx framebuffers use a dense, ordered attachment list. Building _createMultiRenderTargetFramebuffer from a partial attachment set (attachments deferred via dontCreateTextures / targetTypes[i] === -1, or filled out of order) would compress the attachment indices and produce a framebuffer that does not match the MRT layout. Guard the build so it only runs when the collected color handles match the expected attachment count (rtWrapper._attachments.length), which also guarantees contiguous, in-order handles.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
updateMultipleRenderTargetTextureSampleCount was a no-op that only recorded _samples, so RenderTargetWrapper.setSamples() silently did nothing for MRTs on Native (bgfx couples MSAA to texture creation flags). Mirror updateRenderTargetTextureSampleCount: reissue each non-external color attachment's bgfx handle with the new sample count via initializeTexture, update texture.samples, then recreate the framebuffer so it points at the rotated handles.

Also pass limit=1 to the createMultiFrameBuffer-fallback Logger.Warn so it warns once instead of spamming when the framebuffer is rebuilt during attachment setup/swaps.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bkaradzic-microsoft
bkaradzic-microsoft force-pushed the native-multi-render-target branch from 33cdf3b to ddc7578 Compare July 9, 2026 15:04
@bjsplat

bjsplat commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

@bjsplat

bjsplat commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

🟢 Memory Leak Test Results

4 passed, 0 leaked out of 4 scenarios

🟢 All memory leak tests passed — no leaks detected.

Passed Scenarios (4)
Scenario Package
Core Playground #2FDQT5#1508 @babylonjs/core
Core Playground #T90MQ4#14 @babylonjs/core
Core Playground #8EDB5N#2 @babylonjs/core
Core Playground #LL5BIQ#636 @babylonjs/core

@bjsplat

bjsplat commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

@bjsplat

bjsplat commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

@bjsplat

bjsplat commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

⚡ Performance Test Results

Baseline: Latest · Candidate: Dev

Metric Value
🟡 Average 2.4%25 slower
Median 2.4%25 slower
Tests 1 conclusive, 0 inconclusive
🔘 Not Significant — p ≥ 0.05 (1)
Test Baseline Candidate Diff p-value GPU/frame (B/C)
SSAO2 [#XT1HAS#1] (webgpu) 28.9ms 29.6ms 2.4%25 slower 0.0691 16.60 / 16.62

@bkaradzic-microsoft
bkaradzic-microsoft merged commit a548313 into master Jul 9, 2026
22 checks passed
@bkaradzic-microsoft
bkaradzic-microsoft deleted the native-multi-render-target branch July 9, 2026 15:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants