Babylon.js and WebXR. or: How I Learned to Stop Worrying and Love XR
In the beginning, there was a piece of cardboard
VR for the web started (for me at least) with Google Cardboard and the device orientation support. Using the device orientation API you could query the device’s current orientation and change the camera’s transformation accordingly. For me it was a sign that we are going in the right direction — modern APIs are slowly arriving to the browsers, which will allow us to build wonderful websites. The possibilities are endless!
Or at least that was the feeling. This method had a lot of cavities (like — how do I walk forward or what is the right projection matrix for each eye) and was mainly used to display 3D movies or panoramas that didn’t require any internal movement or interaction with the scene. Every time you wanted to load a new website, you had to take the cardboard off, enter the address, press the “enter VR” button, and put it back on. Great fun!
The interfaces improved over time and allowed a bit more user interactions, especially once most frameworks integrated the gaze-mechanism, where you had to stare at a certain point for a second or two to select it. You just needed to be patient. And very very still.
But something was still missing…
Introducing WebVR
WebVR came out of the need for better interaction with the steadily-increasing types of VR devices that the market had to offer. The Vive was relatively new, Oculus Rift (what I know as CR1) was out as well, and all in all it seemed like many other are working on devices of their own. But there was no proper way to use them and their advanced gamepad-like controllers on the web. And thus, the WebVR standard was born.
WebVR was conceived in 2014 by Vladimir Vukicevic of Mozilla, and the version 1.0 was released in 2016. The WebVR interface allowed us to interact with the devices directly including their controllers, and all in all allowed us to develop much better VR experiences.
The introduction of the controllers as gamepads, and using the already-existing Gamepad API for the web (with the addition of their independent pose in world coordinates) allowed for much better user interactions and the real feeling that it is now possible to create amazing web-based VR experiments.
WebVR 1.1 (the latest version of WebVR) is now 2 years old. There was no major change between 1.0 and 1.1, and apart from small API modifications, the experience stayed the same. More and more browsers joined in the party and more devices were published, so it became relatively easy to use VR on the web without the need of any polyfill (an extra library that “adds” support for an unsupported feature ofa browser) or hacks.
But still, WebVR was not complete
There were many issues with the API:
First, it supported pose-based VR devices, but other devices started emerging, like the Hololens and Meta (which sadly passed away), which were not supported.
Second — there were no real standards. For example, any controller could identify with a self-declared ID. The oculus controllers were called “Oculus Touch” and added the handness (right or left) of the controller for both the Rift and the Quest, sadly (which had a different VR device id). The Microsoft controllers, however, were called “Spatial Controller (Spatial Interaction Source)”, and Vive had “OpenVR” in their IDs. Much like using the user agent, we had to sniff out the current device used, and hope that we knew it, otherwise we wouldn’t have mapped the buttons correctly. Each device also had different capabilities (which is totally fine), but we had to know the device in advance — does it have a trigger? What is the trigger position in the buttons array? And whenever a new device was released, we had to hope that the mapping was released along with it, and if not — we had to try it out and map it ourselves.
WebVR was also not ready for many new types of user inputs, with the main one missing was hands. Devices that detect hands (like the Leap motion or the Mio) were already on the market, Kinect was still a thing, but without hacking interfaces on top of them, they could not be integrated. Oh, and the logo was bad as well…
Babylon.js and WebVR
Babylon.js has supported the WebVR API using the WebVR camera since before 1.0 was officially released. We have updated the support to 1.1 when the API was released, and we also supported all major devices and their controllers — Vive, Rift (and eventually the Quest), Windows MR devices, and more. We have also introduced the VR experience helper, which was a simpler way to integrate VR into your scene. All that was needed was a few lines of code, and you got everything the user needed — pointer selection, teleportation, an enter/exit UI button and the entire communication with the native API. After all, what are 3D engines for, if not exactly for that.
I wrote that we supported WebVR, but we are actually still supporting it. Although we are not planning to remove WebVR support from the framework, WebVR has been deprecated for quite some time. And since Chrome has removed WebVR support, we can only assume the rest will soon follow and WebVR will be a thing of the past.
Introducing WebXR
Around two years ago, at the end of 2017, Mozilla announced a draft of what they called WebXR — a combined API for both AR and VR. The idea was to provide a unified API that will allow a developer to provide a single experience with a single codebase for all available devices — AR and VR native devices, VR using a mobile and AR using ARKit/ARCore on Android/iOS, while still supporting new emerging products that will be able to be used without the need to integrate them manually. The proposal had a few other neat ways to handle user input, rendering quality and more.
The specs are being written ever since, and are still a working draft, but were stable enough for Google to remove WebVR support in favor of WebXR on both Chromium mobile and desktop starting version 79 (current stable). There is also a wonderful WebXR polyfill that falls back to WebVR, if the browser doesn’t support WebXR yet.
The only major company who didn’t jump on the bandwagon as of now is Apple, which sadly has no public plan to support WebXR on either OSX or iOS. Yet. I hope.
WebXR integration in Babylon.js
The Babylon.js team has been working on integrating WebXR since the last stable release (4.0). The great work, started by Trevor Baron, is a complete revamp of the Babylon.js WebVR support and was implemented with scale in mind. The great work on the WebXR specs had helped us deliver a unified, modern and well structured implementation that will help you offer XR experiences in your scenes with as little effort as possible. All you need to do to get WebXR to work is to add one single line to your scene:
const xrHelper = await scene.createDefaultXRExperienceAsync( /* optional configuration */ );Babylon.js 4.1 will come with WebXR support that includes:
- An easy to use experience helper
- A dedicated session manager for the more advanced users
- One camera to rule them all
- Full device support. Any device that accepts WebXR sessions is supported. Mixed Reality? Check. Oculus Rift? Check. Android? Daydream? All working out of the box.
- Full input-source support, including everything we could dig out of https://github.com/immersive-web/webxr-input-profiles
- A plugin system (that I call “Features Manager”)
- Lots of vitamin B
- API for Experimental AR features
- Teleportation, scene interactions, physics, and lots more
And the best part is that it is all integrated in this already amazing framework, so you can use all of the framework’s features to make your XR scene even better.
Features, features, features
Since we are addicted to backwards compatibility, we couldn’t deliver WebXR without a sort of a plugin system that will adapt to future changes in the API. The WebXR features manager will always stay as it is, but will allow you to load the latest implementation of any feature, so you can support legacy users (that maybe use an older browser) while allowing the cutting-edge users to use the latest and greatest.
The best example are the AR features. The specs of these features are constantly changing, but we can’t keep on changing the implementation, since older browsers still support the old APIs and it is not the easiest task to always keep up with breaking changes.
After releasing Babylon 4.1 we will continuously write new features as the APIs change, but Babylon’s API to interact with those features will always stay the same.
Here is the simple interaction with the features manager:
What’s next
We have great plans for WebXR for 4.2. You can follow the WebXR tag on GitHub or hit us with questions at the Babylon.js forum if you have any question.
The documentation is being written as I write this blog post (multitasking on two screens). You can start with the introduction to WebXR and take it from there.
Happy Coding!
