With the Apple Vision Pro, we have a new OS, a new SDK, a new simulator, a new input and a totally new UX.
The Keynote and a few tweets had me wonder if the API would be so much high-level that it would be close to impossible to use something other than Swift.
Now Xcode 15 and vision SDK are available, it’s the moment to take a quick look at what is available.
Please note that the SDK is Beta. Things might change. This post is about exploration, testing, experimenting and getting more familiar with what Apple provided.
Toolchain and Xcode project
First things first, let’s have a look at what is provided.
- C/C++ Headers are there. The bare minimum should be available to build our C++ codebase.
- https://github.com/leetal/ios-cmake is a toolchain file for ios. Basically, a set of variables used by CMake to properly generate Xcode project for the intended device.
- Vision Pro is not yet supported and needs CMake 3.28. https://github.com/leetal/ios-cmake/pull/175
- As I’m writing this post, 3.27 is not yet released and there is approximately 4 months between 2 CMake major releases.
So, nothing before end of year to get a valid toolchain. This doesn’t prevent us to try with some manual steps.
Building Babylon Native
I generated the project using iOS toolchain. The iOS playground is already using Swift, with an Objective C++ bridge with C++. It’s not possible to call C++ from Swift directly.
Then inside Xcode, I changed the complete project to visionPro and verified the deployment target. There is more information on this page: https://developer.apple.com/documentation/visionos/bringing-your-app-to-visionos#Add-visionOS-as-a-supported-destination-for-your-app
Now, we are entering the compilation process, I faced 3 issues.
- There is no preprocessor for visionOS and our base library ( https://github.com/bkaradzic/bx ), so it gets a little bit mixed. I had a few #includes and commented some code I know is unused in Babylon Native.
- Same for Metal renderer in bgfx ( https://github.com/bkaradzic/bgfx ). The check is basically iOS or macOS. The change is tiny.
- MTKView is not available! It’s part of the Metal Tool Kit (hence its name) and it’s a helper class that creates and handles the device. You just need to add delegates for rendering and resizing the view. I commented the code for now.
After a few more seconds of build, I had an App ready!
Running App on Simulator
I now have a nice icon in the dashboard! Victory! Well, not so fast.
App is there, I can launch it, but I get nothing except a black screen and no breakpoint reached in Xcode.
It can be frustrating but I’m actually pretty happy. Now, it’s only a matter of getting the correct type of window and a valid Metal device. The road is still long but I can remove Xcode/toolchain/CMake from the equation. More to come!