Inside the Evolution of glTF: Microsoft & Babylon.js Collaboration with the Khronos Group
The Khronos Group, glTF, Babylon.js, Microsoft. From implementing glTF extensions to joining working group meetings, members of the Babylon team have participated in the 3D Formats working group under Khronos to help standardize the 3D ecosystem we have today through glTF. This article tells the history behind how this came to be and what it is like to work on the glTF standard, collaborating with an open standards committee like the Khronos Group.
The Vision
3D for everyone! It all started with this vision. From creation to viewing to storage on a server, this vision required a 3D file format to support the different scenarios. A team within Microsoft already designed a 3D file format for this purpose, but we hit a roadblock when we wanted to integrate with Office. The Office team required that we either completely document this format in the open or use an existing open format. We consulted with Jean Paoli who helped the team navigate the situation. After some research, Jean suggested that we look at glTF which was working towards version 1.1 at the time. glTF 1.1 had just one aspect that was a dealbreaker: the usage of GLSL shaders in materials. Using GLSL shaders meant that the format was not easily portable to native applications. We needed a different way to represent the materials. Fortunately, glTF already had a proposed extension for PBR materials that was portable and served our needs well. After a few discussions with Khronos, we, as Microsoft, decided to join the 3D Formats working group, eventually culminating in this announcement.
The Early Days
In the beginning, we had little experience with many aspects of this work. The people involved initially, Forest Gouin (CTO), Saurabh Bhatia (PM), and I (dev), didn’t have much experience with open source, the web, or standards committees. For graphics, Forest had experience coming from a gaming company and I had been near people doing graphics for some time coming from the gaming division at Microsoft but never actually worked on it directly. None of us had much experience with open source or standards committees. While this may sound like a hinderance, it actually ended up being beneficial. We were helping define the glTF 2.0 specification. Not knowing the field very well meant we had to do our own research. Doing this research helped identify weaker parts in the specification so that we could improve them. Saurabh and I spent countless hours mulling over all the various aspects of the specification. Babylon.js was also vital to this effort. Both David Catuhe and Sebastien Vandenberghe helped tremendously with prototyping and implementing various features for glTF as well as discussed with Saurabh and me the details of the specification. They also had the right experience with web technologies to ensure we understood the browser side. Scott Nagy, who had a background in graphics, joined later to help with the specification and implementation. Patrick Ryan also joined to fill the gap from the perspective of a tech artist, contributing glTF assets to showcase PBR and to various discussion points.
The 3D Formats Working Group
From working group calls to conferences to in-person meetups, working in the Khronos Group was an educational experience. Early on, Saurabh and I spent hours preparing with code, slides, or research trying to solidify the specification for its 2.0 release. Some examples of these issues where we contributed heavily included determining what material model to use for PBR and how to deal with versioning. Once we had the data, we would discuss them during weekly working group calls or at F2F events for higher bandwidth. Doing this kind of work was not for everyone. It took a lot of effort and a great amount of attention to details. But, it also had great benefits. From one perspective, joining the 3D Formats working group gave us a glimpse of the future because the adoption of the standard will ripple through the industry. From another perspective, the 3D Formats working group brought the past into the present, as the standard surfaced and centralized the existing ideas. Attending the events, such as GDC, SIGGRAPH, meetups, gave us the opportunity to present Microsoft’s progress towards its vision as well as meet potential partners for the future. Doing this work gave us confidence that the technology we built contributed to a greater ecosystem that benefited Microsoft and others in the industry.
The Present
Sadly, all good things had to come to an end as it happens sometimes with a big corporation like Microsoft. Although the 3D for everyone vision did not fully pan out, the team that worked on Babylon.js and glTF goes on. After reorganizations internally within Microsoft as well as a global pandemic, the amount of effort we put into the 3D Format working group dropped. Instead of driving the discussions, we now listen and keep an eye on industry trends. We still will occasionally help with implementations to speed things up, but we are far more passive since the reorg. Joining Khronos events in person also took a hit due to the pandemic and budget cuts, but hopefully that will eventually come back.
Nowadays, we participate broadly but less deeply. Members of the Babylon team join various working groups within Khronos: PBR (me), interactivity (Raanan), procedural textures (Sergio), 3D Commerce (Thomas), 3D Formats (me again). We sync up internally every week to discuss topics from these meetings that may affect the Babylon team, other teams at Microsoft, or the general ecosystem. For example, we are actively working on the Flow Graph feature in support of the KHR_interactivity extension for glTF that hopefully will be coming soon. Sometimes, members of our community implement the glTF extension (thanks Mike!) and we discuss this also. We will typically only act on glTF extensions once they are ratified or almost ratified, interactivity being an exception.
The Future
This has been a history lesson on how Microsoft and the Babylon team joined the glTF effort from the Khronos Group. As the Khronos Group and the 3D Formats working group expand the discussions to new extensions, how glTF interfaces with USD, the metaverse, etc., there is no doubt we will see some interesting advancements in the future.
Gary Hsu — Babylon Native Tech Lead
