Babylon Native in a Headless Environment
When we say rendering, we are often talking about rendering at 60fps for an application, whether it be a game or other application that utilize the GPU. There are, however, other scenarios where we might want to use the GPU to run processes that never display anything at all, such as processing a video, manipulating images, or rendering 3D assets, perhaps all running on a server. In this article, I will describe how such scenarios can be achieved with Babylon Native. Specifically, I will show an example of how we can use Babylon Native to capture screenshots of 3D assets using DirectX 11 on Windows.
If you haven’t already read some of our other stories regarding Babylon Native, especially if you haven’t heard of Babylon Native before, it might be worth some time to read them to gain some context before continuing.
Disclaimer: The API contracts used in this example is subject to change as the core team is still working on the correct API contract shape.
The Graphics Device
First, we need to create a standalone DirectX device.
This code is nothing unusual, but it can be tweaked to use WARP, for example, if the environment doesn’t have a GPU.
Next, we will create a Babylon Native graphics device using this DirectX device.
We must specify the width and height (1024x1024 in this example) as the Babylon Native device is not associated with a window or view as it typically is.
Using Chakra is convenient with Visual Studio since we can add a
The Output Texture
We also must create an output render target texture for the
outputRenderTarget of the Babylon.js camera. First, we create a DirectX render target texture.
Note that because this a not a normal rendering application, we are explicitly rendering individual frames and thus we also need synchronization constructs (
std::promise in this case) to ensure correct order. As noted in the documentation for ExternalTexture, the
ExternalTexture::AddToContextAsync function requires that the graphics device renders one frame before it will complete. The
addToContext future will wait until
AddToContextAsync is called and
FinishRenderingCurrentFrame will render a frame to allow
AddToContextAsync to finish.
Next, we will review the first part (
nativeTexture which is the texture from the result of
AddToContextAsync. This argument is then wrapped using
wrapNativeTexture and added as the color attachment of a Babylon.js render target texture. We will see how this is used shortly.
The glTF Assets
Back to the native side, we are now ready to load the glTF assets and capture screenshots.
loadAndRenderAssetAsync, waiting for it to complete, and saving a PNG to disk.
The output render target of the camera is assigned the output render target texture from earlier so that the scene will render to this output texture instead of the default back buffer which, of course, doesn’t exist in this context. This, in turn, will render directly to the native DirectX render target texture we set up earlier.
Building and running the ConsoleApp example looks like this.
Along with three PNG files.
There is one more thing! Notice the helper function calls to
RenderDoc::StopFrameCapture? These will tell RenderDoc to start and stop capturing a frame since RenderDoc won’t know when a frame starts or stops since we are not in the typical rendering case. We can turn on RenderDoc capture by uncommenting one line in
RenderDoc.h. Using RenderDoc is incredibly useful for debugging issues with the GPU.
I hope this gives you an idea of how Babylon Native can be used in headless environment. It is not the typical scenario, but it is a scenario that is more difficult or more expensive to achieve using other technologies. We will continue to strive to make Babylon Native useful in as many scenarios as possible.
Gary Hsu — Babylon Native Team Lead