Canvas Accessibility and GUI Animations with Babylon.js

Hello! Today, I’m writing about two very interesting new additions to the Babylon.js framework — GUI Animations and the Accessibility Twin Renderer. I built a little demo to test them out: an educational quiz! I named it Amy’s Quiz, in reference to the Mixamo Amy character who greets you on the title screen:

Isn’t she adorable?

You can view (and contribute to) the entire project here: carolhmj/babylon-accessibility-example: Demonstrate a small usage example of the Babylon accessibility package (github.com).

I started out by copying the files from this amazing template: RaananW/babylonjs-webpack-es6. It has everything configured to support a Typescript and Webpack build and it’s a great way to get a project going without having to worry about configuration — I can always go back later and customize as needed.

I deleted the scenes I didn’t need and started working on top of the loadModelAndEnv.ts file:

First part of the createScene function

The colors were taken from this great website that shows pairs of contrast-approved and aesthetically pleasing colors: Randoma11y — Accessible color combinations. As for the buildSceneObjects function, it creates the basic entities for the scene:

First part of the buildSceneObjects function

I use a lot of Babylon helpers to make scene creation easier, then import the models (one for Amy and another for a shark) and return all these objects seen earlier. You can see the full function code in the repo, and some useful references in the Babylon.js docs:

Scene | Babylon.js Documentation (babylonjs.com)

Importing Assets | Babylon.js Documentation (babylonjs.com)

After setting up the meshes, I start building our GUI with some specially designed functions:

Calling functions to build the main panels

Here’s an example of the question panel creation:

Creating the question panel

And the result:

Question panel

Our GUI is composed of three panels: Title, Question, and Answer. The return variable transitionButtons will hold an array of buttons that, when clicked, switches which panel is being displayed. This is done using the onPointerClickObservable of these buttons:

Creating the transitions between panels

More on Babylon Observables:

Observables | Babylon.js Documentation (babylonjs.com)

You can see other special functions here — swapModels and swapAnimations — these will change which model is displayed, and which animation is playing respectively. But since the focus today is GUI, let’s look at the swapContainers function:

swapContainers function

buildAnimation function

Allowing GUI to be the target of Babylon.js Animations allows us to do a lot of work with less code — in this case, letting us easily create a nice fade out — fade in effect with easing. The result can be seen here:

Animated transition between title and question screens

You can see there’s an alternate path controlled by the useAnimations variable — one where we don’t play the animation between panels and switch between them instantly. When adding animations to your GUI, it’s important to have reduced/no movement modes, as some movements can make users sick: Web Designers Grapple With Downside to Flashy Animation: Motion Sickness — WSJ.

Non-animated transition between title and question screens

And this is a perfect opportunity to start talking about the Accessibility Tree Renderer! This feature is available in a new package called @babylonjs/accessibility. It gives us a new field named accessibilityTag available in objects of type Node (Meshes, Transform Nodes, etc..) and Control (2D GUI). Any object that defines accessibilityTag.description will be added as a DOM node in a special div, named accessibility-host. Screen Readers can’t read the pixels inside a canvas, but they are very good at reading regular DOM.

The generated accessibility tree

Nodes don’t have a description by default, so the developer must provide them:

Defining accessibility descriptions for the Amy and shark models

Certain Controls set their accessibility description by default. This is the case of the TextBlock and Button. The Accessibility Tree Renderer will also know to automatically update the DOM when any of the accessible elements has its visibility turned off/on — that’s why I set the panels’ isVisible property on the transition functions.

The final piece for our accessible rocket to take flight is simply calling the HTMLTwinRenderer.Render function on createScene:

Creating the Twin renderer

And there we have! You can experiment by using tools like Windows Narrator and/or NVDA to navigate the scene.

A recording of navigating the experience with Windows Narrator

You can also check out another example I made from a previous Babylon demo — just a few lines of code to add and update accessbilityTags and we can render a previously created scene accessible! Weapons Accessibility Demo Test | Babylon.js Playground (babylonjs.com)

We also have the feature’s documentation on our official page: Accessibility Scene Tree for Screen Readers | Babylon.js Documentation (babylonjs.com)

This feature was our first step in the Babylon Accessibility journey, but it is most certainly not the last. Huge thank you to Sunny, who had the idea created the PR! We’re an open-source project, so we are always open to contributions, bugfixes and suggestions — just drop by our forum or our Github repository or send me a tweet!

Thanks for reading and hope you have a FANTASTIC day!

Carolina Herbster — Babylon.js team