Bug 208988 - [WebXR] Implement WebXR device API
Summary: [WebXR] Implement WebXR device API
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebXR (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Enhancement
Assignee: Nobody
URL: https://immersive-web.github.io/webxr
Keywords: InRadar
Depends on: 211130 222317 208702 208989 209859 210553 211129 211187 211506 211888 212099 212407 212470 212529 212826 213020 213022 213462 213509 213519 213555 213886 214187 215224 216925 217752 220732 220979 221225 221267 221551 221599 222173 222263 222270 222607 222737 222785 222796 223211 223257 223580 223635 224364 224365 224931 224935 225025 226074 226919 226923
Blocks: 222360
  Show dependency treegraph
 
Reported: 2020-03-12 03:34 PDT by Sergio Villar Senin
Modified: 2024-04-12 05:09 PDT (History)
32 users (show)

See Also:


Attachments
Android Chrome Native WebXR (847.42 KB, image/jpeg)
2023-05-30 08:59 PDT, Daniel Rossi
no flags Details
visionOS webxr session (154.79 KB, image/png)
2024-03-05 02:01 PST, Daniel Rossi
no flags Details
Chrome webXR Session (113.43 KB, image/png)
2024-03-05 02:02 PST, Daniel Rossi
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Sergio Villar Senin 2020-03-12 03:34:47 PDT
This is a metabug to track progress of WebXR device API implementation
Comment 1 Artem Batogovsky 2021-07-22 02:34:54 PDT
Hi!

Can we test it already? We want to test WebXR on iOS with out product as soon as possible to ensure that our users will have best possible experience
Comment 2 Will Walker 2021-07-30 10:59:24 PDT
Will the WebXR Device API be supported in Safari 15?
Comment 3 Shashank Agrawal 2021-10-08 07:42:06 PDT
Any further update here? I see that this now depends on only two tasks that are pending.
Comment 4 oladam 2021-10-13 02:29:37 PDT
When can you expect the implementation to be completed? I'm sure many developers out there are eager for this to happen
Comment 5 Sam 2021-11-17 08:06:03 PST
Please try to complete this ticket before 2050
Comment 6 Pierre Coulet 2021-12-01 03:13:47 PST
Hi, is there anyone with the knowledge to enlighten us on an estimated date for availability?
Comment 7 David Morehead 2021-12-08 10:06:52 PST
Yes, very interested in the progress of this issue as clients "can't believe" that webxr is not yet supported on IOS.
Comment 8 Frannie Brennan 2021-12-31 08:59:16 PST
Still very keen to see this :)
Comment 9 Al Murray 2021-12-31 10:33:37 PST
Not to clog the comments, but everyone and their mother is waiting eagerly for this. Whole new ballgame on the AR front.
Comment 11 Zach Rattner 2022-01-25 22:21:26 PST
Is there any update on an ETA for this feature, or anything an external party could do to help?
Comment 12 Nik Lever 2022-02-02 07:53:06 PST
Any news on this? Lots of developers waiting for WebXR on iOS. Fingers crossed it lands this year.
Comment 13 Sam Sneddon [:gsnedders] 2022-02-25 10:51:44 PST
<rdar://83079238>
Comment 14 Robert Farthing 2022-02-28 08:19:44 PST
Any ETA on this?
Comment 15 kyehuelin 2022-04-11 02:01:39 PDT
Any update / ETA?

I know there has been rumours of support for WebXR in iOS16 for June 2022

Can anyone in the know on this area confirm anything?
Comment 16 Tavius Koktavy 2022-08-24 13:33:30 PDT
Excited for the possibilities here!

I see the beta flags but they don't seem to enable WebXR on iPhone 12 Pro 15.6.1, has anyone had success?

Is there maybe something needed in JavaScript to tell WebXR that it should work on Safari?
Comment 17 Kevin Staunton-Lambert 2023-05-07 11:09:13 PDT
Hi there, just curious if I need a cert to be able to run immersive-ar session? I’d like to run the WebXR emulator too to support immersive-vr, is this possible with the flags set?
Comment 18 Daniel Rossi 2023-05-27 00:49:51 PDT
I've enabled all the WebXR flags in IOS 16.4 and these samples failed to detect for WebXR. should it display a cardboard mode like native WebXR in Android ? Do we have to wait to test until IOS 17 which makes some devices redundant and won't be able to get IOS 17. ?

I've had complaints to reinstate the WebXR polyfill but the project is vacant and its completely broken. Unrelated to IOS but I had to fix fullscreen support as iPhone has no native fullscreen support to show the canvas ! That is another oversight no html fullscreen make the canvas fullscreen. The fullscreen hack for iPhone is a fixed position 100wh 100vh css rule. 

Is there a way to check a build that actually has WebXR enabled ?

https://immersive-web.github.io/webxr-samples/

this check should work if WebXR is enabled. 

https://jsfiddle.net/danrossi303/ng7x4aw3/3/

More info about the polyfill broken still needed for IOS. 

https://discourse.threejs.org/t/webxr-polyfill-broken-with-latest-three-js/51812/6
Comment 19 Daniel Rossi 2023-05-29 18:44:14 PDT
I was able to check on an Ipad again. xr test is detected. But the next step is checking what type it supports. I believe the session support check may be failing. This should show if its supported.

I get a promise failure with "the operation is insecure". I'll keep testing. 

https://jsfiddle.net/danrossi303/ng7x4aw3/9/
Comment 20 Daniel Rossi 2023-05-29 19:40:34 PDT
Can't edit comments. It seems it fails within an iframe with a security error so jsfiddle wont work. So new check. so sessions are not supported. I tried both immersive-vr and inline. 

https://electroteque.org/dev/xr/xrcheck.html
Comment 21 Daniel Rossi 2023-05-30 08:59:13 PDT
Created attachment 466534 [details]
Android Chrome Native WebXR
Comment 22 Daniel Rossi 2023-05-30 08:59:52 PDT
Will sessions be supported without a headset like Android Chrome has native support for ?. Is that why session support checks are false in IOS 16.4 ?
Comment 23 Daniel Rossi 2023-05-30 19:55:46 PDT
I'm sorry I can't update comments. One thing of importance with WebXR. Landscape screen locking api. IOS doesn't have this, Android does. It requires a css orientation hack like iPhone html fullscreen. Will IOS be getting screen orientation lock api ? 

https://developer.mozilla.org/en-US/docs/Web/API/ScreenOrientation/lock
Comment 24 Daniel Rossi 2023-06-04 15:49:15 PDT
I've made a new ticket regarding screen orientation lock api which is needed for both fullscreen canvas/video content and WebXR. It's been available in Android for 6 years. 

https://bugs.webkit.org/show_bug.cgi?id=257695

There is already a ticket regarding to enabling html fullscreen support for iPhone, where Ipad has fullscreen support. It still won't support launching canvas elements fullscreen. Css fullscreen hacks just don't work well, and landscape lock css hacks work for the Iphone hack but not the best either. You can't manipulate orientation of Ipad fullscreen elements however, so in portrait mode it looks silly. 

https://bugs.webkit.org/show_bug.cgi?id=206854

I had to work on a hack to launch canvas rendering into a seperate video and make that fullscreen for iPhone. I'm surprised it actually worked and managed to hack it to autoplay, not making the video inline allows it to play and show properly. I think there is a bug with muted and inline video with MediaStream. I worked around the clock to pull it off. Device orientation of the original rendering is reflected on the second fullscreen video also. 

https://github.com/danrossi/canvas-pip-fullscreen/blob/main/src/CanvasFullscreen.js#L26

I think both are needed at the same time.
Comment 25 Daniel Rossi 2023-06-06 01:10:08 PDT
I've yet to upgrade to WebGPU myself. But 6 years later it's looking likely things are moving towards WebGPU from WebGL. It's looking like different methods are required for an XR session and layers. It might be good to have WebGPU support also with XR if not already. Along side html canvas fullscreen support and therefore landscape orientation lock with html fullscreen support. 

https://github.com/immersive-web/WebXR-WebGPU-Binding/blob/main/explainer.md
Comment 26 filipesnetopires 2023-08-16 07:39:21 PDT
I've been reading several unresolved threads regarding video playback failures with WebXR (example thread here: https://discourse.threejs.org/t/video-texture-no-longer-updating-after-entering-webxr-mode/43068).
Has this been addressed or is it on the radar and road mapped already? Is this an appropriate place to raise the issue?
Comment 27 Daniel Rossi 2023-08-16 21:54:35 PDT
Those are unrelated to webkit WebXR. I'm hoping there is a native view like Chrome has on Android or does it need a headset ? I have so many people asking about it and the polyfill is deprecated and broken. Has WebXR actually been integrated into IOS for testing yet. As it wouldn't accept sessions with my fiddle test code.
Comment 28 filipesnetopires 2023-08-21 07:30:36 PDT
WebXR has only been integrated into VisionOS, not iOS.
Comment 29 filipesnetopires 2023-08-21 10:22:21 PDT
Daniel, how are those unrelated to webkit WebXR?
The separation between WebXR Device API and Webkit is a bit blurry to me.
Comment 30 Kevin Staunton-Lambert 2023-08-21 11:01:29 PDT
Well for AR on phones/tablets I feel it’s important too - also video projectors are increasingly being used for real world interactive entertainment/exhibits which could easily be browser/WebGPU based, just other things WebXR in WebKit has value
Comment 31 Daniel Rossi 2024-03-01 02:38:45 PST
WebXR is not enabled by default in the VisionOS 1.0/1.1 Simulator. In 1.0 launching video to WebXR crashed and prevents video further playing back. In 1.1 with my implementation I am having issue getting WebGL video rendering and web inspection of 1.1 doesn't even work only in 1.0. I can't seem to find a solution for the broken web inspection in 1.1. 

Can try and launch this in 1.0 and see the crash. It goes to WebXR but further playback stops. Video playback stops. I think it's a browser crash but no error. Controls stop working also. Have to refresh the page. 

https://electroteque.org/plugins/videojs/vrvideo/
Comment 32 Daniel Rossi 2024-03-05 02:01:03 PST
Created attachment 470177 [details]
visionOS webxr session
Comment 33 Daniel Rossi 2024-03-05 02:02:02 PST
Created attachment 470178 [details]
Chrome webXR Session

displays "enabledFeatures"
Comment 34 Daniel Rossi 2024-03-05 02:09:42 PST
I lost a whole message adding screenshots. I am needing "enabledFeatures" property missing in the visionOS webXR session to determine hand-tracking support. Hand controls can be added after the session reports hand-tracking features is supported. 

visionOS 1.0/1.1 simulator has no input sources or hand-tracking support. So I am unable to check hand controls or other vr controllers in it. 

I am trying to figure out how to determine this right now due to that missing feature. Check for "MacIntel" / Ipad platform and if there is inputSources ? If vr controls as inputSources is enabled there should be a connected event also. So not sure how to determine this between the simulator and the headset. I don't have a headset to check.
Comment 35 Daniel Rossi 2024-03-05 03:29:46 PST
Due to the lack of the hand-tracking feature property and lack of inputSources with the simulator something like this I came up with and seems to be cross compatible. So nothing gets added for the simulator. Assuming the headset provides inputSources I can't check. 

https://discourse.threejs.org/t/detect-hand-tracking-xr-feature-support-to-add-hand-controllers/62229/4
Comment 36 Daniel Rossi 2024-03-28 22:44:59 PDT
Since Apple Vision OS 1.1 simulator, a mouse click acting as pinch turns on a "transient-pointer" input source. And the targetRaySpace is updated on movement. I can get raycasting of objects working. webinspect is broken so I had to build a logging proxy to see what was actually going on.

I wish a native WebXR was made available for IPhone like Android on top of native html fullscreen. I thought that is what was happening. I'm forced to try and implement a work around and is not easy.
Comment 37 Wladislav Artsimovich 2024-04-11 07:42:16 PDT
I'm deep in a WebXR project and quite sad to see iOS being unsupported.
Happy to support with any kind of testing.