Building Babylon Native for the Apple Vision Pro

4 min readJun 29, 2023

Having a new platform to develop for is always exciting. The last one from Apple I remember is the introduction of arm64 for macOS.

Vision Pro coverage

With the Apple Vision Pro, we have a new OS, a new SDK, a new simulator, a new input and a totally new UX.

Getting the same experience with same rendering code on the Web and Native is the purpose of Babylon Native.

The Keynote and a few tweets had me wonder if the API would be so much high-level that it would be close to impossible to use something other than Swift.

Now Xcode 15 and vision SDK are available, it’s the moment to take a quick look at what is available.

Xcode beta 2 available at

Please note that the SDK is Beta. Things might change. This post is about exploration, testing, experimenting and getting more familiar with what Apple provided.

Toolchain and Xcode project

C Headers are available for XROS

First things first, let’s have a look at what is provided.

  • C/C++ Headers are there. The bare minimum should be available to build our C++ codebase.
  • is a toolchain file for ios. Basically, a set of variables used by CMake to properly generate Xcode project for the intended device.
  • Vision Pro is not yet supported and needs CMake 3.28.
  • As I’m writing this post, 3.27 is not yet released and there is approximately 4 months between 2 CMake major releases.

So, nothing before end of year to get a valid toolchain. This doesn’t prevent us to try with some manual steps.

Building Babylon Native

I generated the project using iOS toolchain. The iOS playground is already using Swift, with an Objective C++ bridge with C++. It’s not possible to call C++ from Swift directly.

Then inside Xcode, I changed the complete project to visionPro and verified the deployment target. There is more information on this page:

Make sure to manually set visionOS as the SDK and platform

Now, we are entering the compilation process, I faced 3 issues.

Sometimes, ignoring issues is the best to get things moving.
  • There is no preprocessor for visionOS and our base library ( ), so it gets a little bit mixed. I had a few #includes and commented some code I know is unused in Babylon Native.
  • Same for Metal renderer in bgfx ( ). The check is basically iOS or macOS. The change is tiny.
  • MTKView is not available! It’s part of the Metal Tool Kit (hence its name) and it’s a helper class that creates and handles the device. You just need to add delegates for rendering and resizing the view. I commented the code for now.

After a few more seconds of build, I had an App ready!

Running App on Simulator

I now have a nice icon in the dashboard! Victory! Well, not so fast.

It's loading!

App is there, I can launch it, but I get nothing except a black screen and no breakpoint reached in Xcode.

No luck, black screen.

It can be frustrating but I’m actually pretty happy. Now, it’s only a matter of getting the correct type of window and a valid Metal device. The road is still long but I can remove Xcode/toolchain/CMake from the equation. More to come!




Babylon.js: Powerful, Beautiful, Simple, Open — Web-Based 3D At Its Best.