Add support for WebGPURenderer and TSL example by vincentfretin · Pull Request #5655 · aframevr/aframe · GitHub

Description:

For those that want to play with WebGPURenderer and TSL.
Run

npm run dist # to build dist/aframe-master.module.min.js used in tsl example
npm run start:webgpu

Start from
https://localhost:8080/showcase/tsl/
and port an example from https://threejs.org/examples/?q=tsl

Changes proposed:

Rename a-node isNode to isANode, this was the cause of infinite loop with build function on WebGPURenderer when building a shader
Add a start:webgpu command that will import from three/webgpu instead of three via a webpack alias
Set dummy renderer.xr.setPoseTarget for now when it's WebGPURenderer
Import UniformsUtils and UniformsLib from three/src (used by sdf/msdf text shaders), those are not exposed on the three/webgpu build
Fix vendor/rStats.extras.js where renderer.info.programs is undefined with WebGPURenderer

All the changes here are backward compatible with the default WebGLRenderer umd aframe build.

You can use WebGPURenderer in two ways:

via an importmap defining three and three/webpgu to use three.webgpu.js instead of three.module.js
in your webpack project defining the correct alias:

  resolve: {
    alias: {
      'three$': path.resolve(__dirname, 'node_modules/three/build/three.webgpu.js')
    }
  };

and using in package.json this PR for now:

"dependencies": {
  "aframe": "vincentfretin/aframe#webgpu"
}

Using WebGPURenderer with classic aframe umd build is not supported, we don't generate a build for that.

We should def understand what's going on here

The infinite loop was here https://github.com/mrdoob/three.js/blob/a5734130516db50c1de2cdfe27f477a33ecd1c87/src/nodes/core/Node.js#L640-L648
I don't know much about the new node system to say what that code is doing.

As I said in #5363 (comment) my guess is that in aframe we set a reference to the entity with el on a three object.

Doing a search I get the following places:

$ ag "\.el = " src/
src/core/system.js
31:  this.el = sceneEl;

src/core/component.js
66:  this.el = el;

src/core/a-entity.js
36:    this.object3D.el = this;
111:    this.object3D.el = null;
140:    obj.el = this;
143:        child.el = self;

src/components/material.js
112:    shaderInstance.el = this.el;

src/components/light.js
205:      this.light.el = el;

src/components/text.js
437:  shaderObject.el = el;

My guess is that the one set on shaderInstance.el is the culprit and end up in the shader/material being build, but not sure how, I'm not familiar with the new webgpu code path.

I'm not aware of any community components that is using an isNode check, I think it's fine to rename it to isANode and add a note about it in the release notes.

Cool. I would postpone this to 1.8.0. I will be cool to have a WebGPU path.

sdf/msdf text is completely broken
WebGLRenderer

WebGPURenderer

http://127.0.0.1:8080/docs/basic-scene/
2 errors in console: NodeMaterial: Material "ShaderMaterial" is not compatible.
may be related to the text shader

http://127.0.0.1:8080/showcase/curved-mockups/
background image is black on Chrome Android where is really run on webgpu. On desktop it's okay but it fallbacks to webgl2 on Ubuntu, so not sure if it's a webgpu on mobile issue.

http://127.0.0.1:8080/boilerplate/hello-world/
WebGLRenderer

WebGPURenderer (webgpu backend or webgl2 fallback with r173)

…ree/webgpu build

for me the only question is, what is the usecase for renderer.xr.setPoseTarget i never need this and its only used at

aframe/src/core/scene/a-scene.js

Line 553 in b9e4838

if (this.camera) { renderer.xr.setPoseTarget(this.camera.el.object3D); }

@dmarcos I think we could add a TSL example and then merge this, all the changes here are backward compatible with the WebGLRenderer aframe umd build. We can then make other changes if needed, mainly the text component that is completely broken by replacing it with troika-text if that one work with WebGPU backend.

As of three r173, VR mode is not working with WebGPU, we get the error:
THREE.XRManager: XR is currently not supported with a WebGPU backend. Use WebGL by passing "{ forceWebGL: true }" to the constructor of the renderer.

for me the only question is, what is the usecase for renderer.xr.setPoseTarget i never need this and its only used at

aframe/src/core/scene/a-scene.js

Line 553 in b9e4838

if (this.camera) { renderer.xr.setPoseTarget(this.camera.el.object3D); }

Good question, we have that patch supermedium/three.js@3bfa95d (in 0.173.4 branch)
not sure what it does, maybe keep the xz position of the aframe camera when entering VR? The y position is overridden in VR that I know. I usually set a xz position on camera rig, not the camera.

for me the only question is, what is the usecase for renderer.xr.setPoseTarget i never need this and its only used at

@arpu In Three.js the WebXRManager moves the "user" camera to match the xr camera. In A-Frame the camera is conceptually an <a-camera> or <a-entity camera>, which has the actual camera as a child. Just like how components such as look-controls and wasd-controls move the camera entity and not the camera itself, the setPoseTarget is there to let Three.js do the same.

So I guess we will need to do a similar patch to XRManager (used by WebGPU backend) to implement setPoseTarget when VR will be supported.

There is an alternative which I use in my fork, namely disabling XR camera updates in Three.js and doing it ourselves in the camera component. Not only does this avoid needing a modification to Three.js, it also ensures that the camera is up-to-date after the camera component. So any logic that is based on the users head position/orientation can use after: ['camera']. Currently these components tend to use 1 frame stale data.

for me the only question is, what is the usecase for renderer.xr.setPoseTarget i never need this and its only used at

@arpu In Three.js the WebXRManager moves the "user" camera to match the xr camera. In A-Frame the camera is conceptually an <a-camera> or <a-entity camera>, which has the actual camera as a child. Just like how components such as look-controls and wasd-controls move the camera entity and not the camera itself, the setPoseTarget is there to let Three.js do the same.

Correct. That method or similar existed in WebVRManager and WebXRManager and was removed at some point. Since the pose applies only to the camera and one cannot apply it to an arbitrary Object3D. I raised the concern to THREE but was ignored.

There is an alternative which I use in my fork, namely disabling XR camera updates in Three.js and doing it ourselves in the camera component. Not only does this avoid needing a modification to Three.js, it also ensures that the camera is up-to-date after the camera component. So any logic that is based on the users head position/orientation can use after: ['camera']. Currently these components tend to use 1 frame stale data.

How do you disable camera updates?

How do you disable camera updates?

By setting renderer.xr.cameraAutoUpdate to false (see https://github.com/mrdoob/three.js/blob/dev/src/renderers/WebGLRenderer.js#L1160). But as with anything, it isn't without its drawbacks. If the camera component handles the updating, then any camera movement afterwards (directly or indirectly through a camera rig) needs to be propagated to the world matrices of the sub cameras. This only needs to be done right before rendering, but it no longer happens automatically.

Basically the above flag is an all-or-nothing and in an ideal world the part that transfers the WebXR pose onto the camera is done in the camera component and the part that updates the world matrices of the underlying eye cameras is done right before rendering.

But I have no idea if the XRManager for WebGPURenderer is going to have the same behaviour, probably best to wait and see how that implementation will behave.

How do you disable camera updates?

By setting renderer.xr.cameraAutoUpdate to false (see https://github.com/mrdoob/three.js/blob/dev/src/renderers/WebGLRenderer.js#L1160). But as with anything, it isn't without its drawbacks. If the camera component handles the updating, then any camera movement afterwards (directly or indirectly through a camera rig) needs to be propagated to the world matrices of the sub cameras. This only needs to be done right before rendering, but it no longer happens automatically.

Basically the above flag is an all-or-nothing and in an ideal world the part that transfers the WebXR pose onto the camera is done in the camera component and the part that updates the world matrices of the underlying eye cameras is done right before rendering.

But I have no idea if the XRManager for WebGPURenderer is going to have the same behaviour, probably best to wait and see how that implementation will behave.

Thanks. I like the idea of having more control of the lower level parts that concern XR. So in your case the pose used for rendering is not the most current one since it’s done in the component?

I ported the galaxy example.
As far as I can tell, this is ready to be merged.

I ported the galaxy example. As far as I can tell, this is ready to be merged.

Thanks. Does it work in VR?

No, three r173 doesn't support WebXR with WebGPU.

Would probably name this example 'webgpu' instead of tsl?

The main reason I did this example is to play with TSL that is currently only available with WebGPURenderer, but that example may also run with WebGL2 for systems that doesn't support WebGPU, so I would like to keep TSL here in the title.
I renamed directory to webgpu and changed the title to "WebGPURenderer and TSL" so that matches what I put in the root index.html

Thanks!

vincentfretin mentioned this pull request Feb 4, 2025

Get rid of build step #5363

Open

dmarcos reviewed Feb 4, 2025

View reviewed changes

vincentfretin force-pushed the webgpu branch from ac1cf54 to e76924b Compare February 5, 2025 10:18

rename isNode to isANode

ffc72d0

vincentfretin force-pushed the webgpu branch from e76924b to 99acfa5 Compare February 5, 2025 10:19

vincentfretin force-pushed the webgpu branch from 80f9640 to 0f6b7c0 Compare February 6, 2025 10:43

vincentfretin added 4 commits February 6, 2025 11:52

add 'npm run start:webgpu' command to use WebGPURenderer

8a911a8

import UniformsUtils and UniformsLib, those are not exposed on the th…

9cb5bb3

…ree/webgpu build

renderer.info.programs is undefined with three/webgpu

95fd627

add empty tsl example

b0985c8

vincentfretin force-pushed the webgpu branch from 0f6b7c0 to b0985c8 Compare February 6, 2025 10:53

vincentfretin changed the title testing WebGPURenderer and TSL (do not merge) Add support for WebGPURenderer and TSL example Feb 6, 2025

vincentfretin marked this pull request as ready for review February 6, 2025 11:55

vincentfretin added 3 commits February 10, 2025 11:22

add galaxy example

d2563cd

add link to tsl example

368536b

remove mesh and gui in remove

f07cc12

dmarcos reviewed Feb 13, 2025

View reviewed changes

vincentfretin and others added 6 commits February 14, 2025 12:19

change title

8db2a9a

rename directory tsl to webgpu

714dcd3

reformat with 120 chars, single quote and no trailing comma

25481dd

reformat to 120 chars

30a7a65

Update index.html

af35cbd

Update index.html

3089799

dmarcos merged commit 536adc0 into aframevr:master Feb 14, 2025

vincentfretin deleted the webgpu branch February 15, 2025 08:45

mrxz mentioned this pull request Feb 19, 2025

Check for presence of UniformsLib/Utils instead of importing them from three/src #5668

Merged

+                <head>
+                  <meta charset="utf-8" />
+                  <title>TSL • A-Frame</title>
+                  <meta name="description" content="TSL • A-Frame" />

               function isNode (node) {
-                return node.tagName.toLowerCase() in knownTags || node.isNode;
+                return node.tagName.toLowerCase() in knownTags || node.isAframeNode;

Add support for WebGPURenderer and TSL example by vincentfretin · Pull Request #5655 · aframevr/aframe · GitHub

Uh oh!

Uh oh!

Add support for WebGPURenderer and TSL example - #5655

Add support for WebGPURenderer and TSL example#5655dmarcos merged 14 commits intoaframevr:masteraframevr/aframe:masterfrom vincentfretin:webgpuvincentfretin/aframe:webgpuCopy head branch name to clipboard

Conversation

vincentfretin commented Feb 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dmarcos Feb 4, 2025

Choose a reason for hiding this comment

Uh oh!

vincentfretin Feb 5, 2025

Choose a reason for hiding this comment

Uh oh!

vincentfretin Feb 5, 2025

Choose a reason for hiding this comment

Uh oh!

vincentfretin Feb 6, 2025

Choose a reason for hiding this comment

Uh oh!

dmarcos commented Feb 4, 2025

Uh oh!

vincentfretin commented Feb 6, 2025

Uh oh!

vincentfretin commented Feb 6, 2025

Uh oh!

vincentfretin commented Feb 6, 2025

Uh oh!

arpu commented Feb 6, 2025

Uh oh!

vincentfretin commented Feb 6, 2025

Uh oh!

vincentfretin commented Feb 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mrxz commented Feb 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vincentfretin commented Feb 6, 2025

Uh oh!

mrxz commented Feb 6, 2025

Uh oh!

dmarcos commented Feb 6, 2025

Uh oh!

dmarcos commented Feb 6, 2025

Uh oh!

mrxz commented Feb 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dmarcos commented Feb 6, 2025

Uh oh!

vincentfretin commented Feb 10, 2025

Uh oh!

dmarcos commented Feb 11, 2025

Uh oh!

vincentfretin commented Feb 12, 2025

Uh oh!

dmarcos Feb 13, 2025

Choose a reason for hiding this comment

Uh oh!

vincentfretin Feb 14, 2025

Choose a reason for hiding this comment

Uh oh!

dmarcos commented Feb 14, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Add support for WebGPURenderer and TSL example#5655
dmarcos merged 14 commits into
aframevr:masterfrom
vincentfretin:webgpu

vincentfretin commented Feb 4, 2025 •
edited
Loading

vincentfretin commented Feb 6, 2025 •
edited
Loading

mrxz commented Feb 6, 2025 •
edited
Loading

mrxz commented Feb 6, 2025 •
edited
Loading