A Babylon Native Backstage Tour
Nice to meet you Babylon.js community!
I’m Sergio, I’m joining the Babylon team to work as a Software engineer on the Babylon Native project. I’ve graduated as a Biomedical Engineer in 2017 (I know, not what you would expect from a 3D graphics developer, right?), but during my undergrad I was able to work as a game developer for creating therapeutic games for kids with disabilities at the Holland Bloorview Kids Rehabilitation Hospital in Toronto, Canada. That experience showed me that working with 3D graphics was way more interesting to me than doing biomedical engineering work, so I decided to focus my career on that.
After working on multiple companies creating 3D experiences for medical training, engineering and entertainment, I ended up focusing more and more on creating frameworks and technologies to allow other teams to create amazing experiences.
So, in order to introduce myself, I would like to share with you guys how amazing the Babylon Native project is and the type of work we do there.
What is Babylon Native?
Babylon Native is a very exciting project and it fits perfectly with my love for learning. Its goal is to provide a way for native applications to utilize the powerful Babylon.js for 3D rendering in a cross platform way. For more information see:
Babylon React Native: Bringing 3D and XR to React Native Applications | by Babylon.js | Medium
Babylon Native: The Journey So Far | by Babylon.js | Medium
When executed on a web browser Babylon.js can make usage of APIs such as WebGL and WebGPU for creating and managing graphics resources on the GPU. However, when embedded on a native application, the JS runtime will not have access to those API. That’s why Babylon Native must provide all functionality required by Babylon.js to perform it’s rendering.
In order to properly handle all the different render API’s the Babylon Native project makes usage of BGFX rendering API as an abstraction layer, for more information about BGFX project see:
Work at the Babylon Native team
Among other things, our goal in the Babylon Native team is to bring all functionality currently supported by the Babylon.js framework to be supported when running within a native application, this way our users can just bring the code they already have running in the browser and get the exactly same experience when running in the native application.
Example — Adding support for morph targets:
One of my first tasks after joining Babylon Native team was to bring the support for morph targets. They way Babylon.js handles the differences in execution between the web browser (were it has access to the WebGL and WebGPU APIs) and Babylon Native (were graphic functionality will be provided by our C++ layer) is that it has two implementations of its Engine class, one that run in the browser and another (NativeEngine) that run on the native environment.
When adding support for a Babylon.js feature in Babylon Native, we have to first identify what engine capability is required for that feature and then make sure that NativeEngine class supports it.
For morph targets to work properly, NativeEngine was missing the support for the creating of GPU texture arrays. The engine method “createRawTexture2DArray” allows the creating of an array of Texture2D objects on the GPU. In the standard Babylon.js Engine this is implemented using WebGL or WebGPU. However, when running under BabylonNative the JS runtime will not have access to those API’s.
Adding support for 2D Texture Arrays:
First thing we do is to add an implementation of “createRawTexture2DArray” to the NativeEngine class. This implementation will not be able to use WebGL or WebGPU to create the texture array on the GPU.
In order to allow NativeEngine to access the GPU we create a C++ method that will take care of that functionality using bgfx. The “LoadRawTexture2Array” method will be exposed to Javascript using NAPI.
Finally, we are able to load an animated glb file with morph targets and see it been properly displayed in the Win32 application using the same code we would use on the browser.
Final thoughts
Babylon Native is a very unique project to work with. It gives you the opportunity to work with both high level languages such as Javascript as well as low level C++. More than that, it allow you to understand how Javascript engines work, how 3D graphics work and hundreds of other things!
I’m very excited to continue bringing all the amazing features from Babylon.js to the native work and to allow our users code to deliver amazing experiences everywhere!
Are you excited about this type of work?
The Babylon team has open positions for both the Babylon.js and Babylon Native! If you want to learn more about those positions please use the links below:
Babylon.js WebGL Software Engineer II in Other, Other, Brazil | Engineering at Microsoft
Babylon Software Engineer II in Redmond, Washington, United States | Engineering at Microsoft
References
Sergio R. Z. Masson — Babylon.js Team
