glTF Extensions in Babylon.js

The mission for Babylon.js is to create one of the most powerful, beautiful, simple and open web rendering engines in the world. It is essential that the engine stays up-to-date with the latest advancements in real-time rendering technology to ensure that Babylon.js renders beautifully and consistently. This includes staying up-to-date and supporting the latest glTF extensions and proposals. You probably have seen that glTF has been gaining momentum in the 3D industry. As it progresses forward, both the Khronos 3D Formats Working Group and the glTF community are defining more and more extensions. The Babylon.js team is always keen on implementing the latest and greatest, both to support the working group and to give feedback on defining the extension specification, with the intention to support at least all ratified Khronos extensions. This article captures a snapshot of the current state of glTF extensions in Babylon.js.

The glTF Loader in Babylon.js

If you’ve read the previous article on Extending the glTF Loader in Babylon.js, you know that glTF loader extensions in Babylon.js are fully separated from the core glTF loader implementation. This is especially important as the number of glTF extensions grows. At the time of this writing, there are 17 glTF loader extensions implemented in Babylon.js, all in various states.

Stable glTF Extensions in Babylon.js

Before diving into some of the newer extensions, here are the extensions already ratified and supported in Babylon.js.

Click on the main links to find the Babylon.js implementation or the parenthetical link for the specification. Besides some small bug fixes, the code has not changed since their original implementation.

Experimental glTF Extensions in Babylon.js

Although the Babylon team recently shipped Babylon.js 4.1, some of the code are experimental. Many of the newer glTF extensions implementations fall into this category. They come with a warning that the specification and code is subject to change and may not be backwards compatible.

Like before, find out more by clicking on the main links for the implementation or the parenthetical link for the specification or pull request. Most of these are still in flux and the implementation may change to follow the changes to the proposed specifications.

KHR_mesh_instancing

Implemented recently by Drigax in Babylon.js, this extension enables the transforms of instances to be stored in a glTF as binary buffers which is more efficient to store and parse when compared with JSON.

teapots_galore sample from the pull request

There is some debate on whether this extension should be a Khronos (KHR) extension or a multi-vendor (EXT) extension, which affects the naming of the extension itself. See Promoting Extensions for an understanding of the different types of extensions.

KHR_mesh_quantization

This extension enables mesh data to be quantized, trading lower precision for reduced size, by expanding the allowed component types in a glTF.

This extension was proposed by zeux who also did the leg work in multiple implementations. The working group is planning on ratifying this extension soon.

KHR_materials_<clearcoat/sheen/specular>

These three extensions are some of the proposed extensions for what the working group calls PBR Next. KHR_materials_clearcoat is close to ratification. KHR_materials_sheen requires consensus on the equations. KHR_materials_specular is actively being discussed. All of these extensions expand on the existing physically-based material model in glTF core specification. The Babylon.js team is trying to stay on top of the implementation and issues to make sure the working group gets the right feedback.

ClearCoatTest from the glTF sample models (created by Ed Mackey, rendered in Babylon.js sandbox)

KHR_texture_basisu

This is an important extension that enables glTF to reference Basis Universal compressed textures inside a KTX2 container for faster download size and smaller GPU memory footprint.

Avocado with KTX2+BasisU processed with gltfpack (see thread)
Comparison of texture sizes in bytes
GPU memory with PNG or JPG
GPU memory using KTX2+Basis transcoded to BC1 on Windows 10

The state of the ecosystem is captured in this issue. One of the main issues is that BasisU does not always store non-color textures well, as noted in this issue. This isn’t a major issue for models like the Avocado, but it is an issue for models like the FlightHelmet. What is recommended still needs more research and discussion.

On the implementation side, there are two ways currently to consume KTX-Software for the web. One way is to build the libktx_js target which does most of the work on the native side, including uploading the data to the WebGL texture. While this is the most convenient way to read a KTX2 on the web, it comes with a bigger download and prevents transcoding on a worker thread. The alternative is to build the msc_basis_transcoder_js target which only does the transcoding; parsing the KTX2 container is done purely in JavaScript (see here for a example implementation). This enables running in a web worker with a smaller download. The implementation in Babylon.js currently uses the former method via libktx_js. This issue tracks changing the implementation to the alternative method.

Babylon Native

One of the main goals of Babylon Native is that the code written for Babylon.js just works in Babylon Native. As more implementations for various extensions are added to Babylon.js, these extensions also work automatically in Babylon Native. For example, the ClearCoatTest example from earlier works in Babylon Native and renders identically.

ClearCoatTest rendered in Babylon Native Win32

Conclusion

This is a quick snapshot of the many glTF extensions implemented in Babylon.js. More are coming. PBR Next has more proposed extensions, KTX2 and IBL has a proposed extension, and the 3D Commerce group will surely propose their own as well as the glTF community. As the industry progresses, the Babylon.js team will always be there to support and give feedback.

Babylon.js: Powerful, Beautiful, Simple, Open — Web-Based 3D At Its Best. https://www.babylonjs.com/