WebVR/XR — The future is here

6 min readJan 24, 2019

Building an immersive application today is easy!

TLDR:

WebVR/WebXR lets websites talk to AR/VR devices with just a browser
WebXR is not ready yet, but WebVR is pretty stable in Firefox
It’s easy to get started right from the browser. See Getting Started below

What is WebVR/XR

WebVR and WebXR are both web API’s that allow web pages to interact with virtual or augmented reality devices to perform things such as:

Get the position/rotation of the device
Display images on the device
Handle events from VR/AR controllers (e.g. buttons or stick axis)
Get the position/rotation of the controllers
Query AR/VR capabilities of the device

This allows users to view and interact with immersive content without the need to download an app. They can just navigate to a website which talks to the WebXR/VR APIs.

Typically websites use assisting libraries (eg. Babylon.js, A-Frame or ThreeJS) to build XR/VR content as they simplify much of the boilerplate code. For example, Babylon.js provides a VRExperienceHelper class which can add VR support to a scene in just one line of code:

// Enable VR
var vrHelper = scene.createDefaultVRExperience();

View/modify the example online here: Playground

Why VR/AR on the web?

No downloads required, experiences are a click away
No large IDE required for development, just a browser and text editor
Cross platform
Use tools web developers know (js/npm/jQuery/typescript/etc)
Desktop/Mobile Progressive Web Apps (PWA)

Current status

WebVR was created initially to target the first wave of VR devices that were released (e.g. Vive, Oculus, Windows Mixed Reality, Daydream phones). It is available now when using a compatible device and browser. A working example can be found on the Babylon.js playground here. Current browser support is here. I’ve found Firefox to be the most stable for desktop headsets.

WebXR was created to target a wider range of immersive devices beyond just VR. These include augmented reality devices (Digital content on top of the real world) such as phones, Microsoft HoloLens and Magic Leap One. WebVR had a few issues making it hard to extend to all immersive devices, so it was decided to be deprecated in favor of the WebXR API. Existing WebVR apps won’t be out of luck thanks to the WebVR polyfill which provides WebVR as long as WebXR exists.

The WebXR spec is still under development but has early experimental implementations in some browsers (e.g. Chrome Canary). The current milestone dates are here. Unfortunately, it isn’t recommended to code against yet as it is still evolving.

Babylon.js has already started providing support for WebXR and the docs are available here. However, the docs are not final and will evolve as the spec changes.

Responsive

Web technology works on a wide range of devices and so do WebXR and WebVR. This makes it challenging to ensure applications work across a wide range of immersive hardware.

Some devices only detect rotation (3dof) and don’t let the user walk around the room. Some don’t have controllers. Some are handheld. Some you can see through. Some have controllers that track position and rotation (6dof). This can make application logic get pretty complicated.

WebVR display and gamepad objects report their capabilities as seen documented here.

display.capabilities.hasPositiongamepad.pose.hasPosition

WebXR provides the ability for applications to query supported spatial tracking modes and start an XR session for a specific mode. Applications can target the most limited devices first and add additional features given additional support to reach the most devices or target only a specific feature set that is required for the scenario.

More information on this can be found here.

Getting Started

The Babylon.js playground makes it pretty easy to try things out. There are a good number of example playgrounds that can be used as building blocks for more complex apps.

Playground Building Blocks

For any issues, feel free to ask questions on the Babylon forum.

What’s next

The WebXR API is a great first step, so hopefully it will be available in browsers sometime this year 🤞, but it is still missing a number of features to enable more compelling scenarios.

Displaying DOM content

Unfortunately, browsers don’t currently support turning existing html content into WebGL textures for numerous reasons including security worries; there are some proposals to solve this in the future but at present, apps need to do the rendering themselves.

Current workaround:

Babylon.js’s gui: Use the Babylon.js GUI — Babylon.js Documentation
html2canvas: https://html2canvas.hertzen.com/
Manually drawing text on a canvas and using Babylons dynamic texture to wrap the canvas: HTML Canvas Text

XR Workflow

Using headsets with WebVR/XR can be painful right now. For security reasons browsers don’t let websites interact with the headset until the user has explicitly performed an action to allow it so for now most use cases follow this pattern.

Navigate to website outside of VR/AR
Click a button on the site to enter immersive mode
Put on the headset
Exit immersive mode and take off headset
Repeat

It’s not much, but the extra effort makes staying out of the headset that much easier. The ability to navigate the web from within most headsets/browsers would make using them more compelling.

XR Multitasking

Currently WebVR/XR take full control of the headset, this means that only one application can be in view at a time. This is unfortunate because most people do more than one thing at a time. A shopping or mini game WebVR/XR experience is far less appealing if you can’t use it while watching a video or talking to friends. I created a proposal of how to add support for this on the WebXR proposals github page if you want to read more and I hope that WebXR doesn’t ship in a way that this can’t be supported in the future.

WebGPU

WebGPU is a new API draft that is aimed to give web 3D applications a lower level API than WebGL the provides more functionality and improved performance. This could help the web close the gap with native 3D engines.

Devices/Content

So far, the majority of the WebVR/XR content feels pretty gimmicky. The content most viewed daily (Social Networks, Videos, Games, News) generally still isn’t available on immersive devices in a way that provides more value than using a traditional screen. More immersive devices are being released each year. As these become more mainstream there will be a larger incentive for developers to target these devices with more engaging content.

Browsers

Although desktop browser innovation has seemed a bit limited recently, many experimental browsers have popped up around the immersive web. They provide great insight into new ideas that can make WebVR/AR much easier and enjoyable to consume.