How to simulate out-of-bounds viewports when using WebGPU or BabylonNative

Since joining the Babylon team last year I’ve been learning a lot about 3D graphics and a couple of months ago I got introduced to the concept of viewports. I’ve never worked with viewports before, so this new knowledge was a revelation to me, even though viewports have been around for a long time. So long, in fact, that they’re included in the very first OpenGL 1.0 spec all the way back in the 1990s! I loved becoming one of the lucky 10,000 with viewports, so in this post I’ll be talking a bit about what I’ve learned and dive into how we solved a limitation we ran into when trying to use so-called “out-of-bounds” viewports, which aren’t supported on some graphics devices.

What even is a viewport?

In Babylon.js terms, a viewport is a rectangular area on the screen that a camera will draw to. It has the values x, y, width, and height, with x and y starting at the bottom left of the screen, and width and height normalized to the screen width and height. The default viewport covers the entire screen, so it's x and y are both set to 0, and its width and height are both set to 1. Like this ...

The default viewport, depicted in purple, covers the entire screen.

Most of the time, the default viewport is all that’s needed, and you don’t even have to think about it because it lets you draw to the whole screen, but what if you only want to draw to one particular section of the screen? This is where the humble viewport is your friend. For example, to draw everything in the lower left corner of the screen you can change the camera’s viewport width and height values to 0.5. Like this ...

Changing the viewport’s width and height to 0.5 shrinks it to half the screen’s width and half the screen’s height.

To move this smaller viewport to the upper right corner of the screen instead of the lower left, you can change the camera’s viewport x and y values to 0.5. Like this ...

Changing the viewport’s x to 0.5 moves it half-way across the screen’s width, and changing the viewport’s y to 0.5 moves it half-way up the screen’s height.

So far so good, but what if the viewport rectangle is defined in a way that makes it goes off the edges of the screen? For example, what if we leave the viewport’s x and y set to 0.5 but change its width and height from 0.5 to 0.6? Like this ...

Changing the viewport’s width and height to 0.6 makes it go off the edges of the screen.

Notice that the rendered scene still fills the same amount of the viewport as before, but the upper right corner of the viewport is outside the bounds of the screen, now, so if you’re looking at just the screen, then the scene appears to be slightly more zoomed in and closer to the upper right corner than before. This can be shown more clearly by increasing the viewport’s width and height to 0.75 ...

Changing the viewport’s width and height to 0.75 now clearly shows the scene is zoomed in and has moved closer to the upper right corner of the screen.

Taking this effect a bit further we can set the viewport’s x and y values to negative values and make its width and height larger than screen size. This will make the scene appear to be zoomed in while still filling the entire screen. Like this ...

Note that only the camera’s viewport values are changing here, so we’re effectively zooming in on the scene without moving the camera in 3D space or changing its field of view, while still rendering the scene at the screen’s full pixel resolution!

Sounds easy enough, right? Well, not quite because …

The problem

Some graphics devices don’t let you set the viewport’s values to be out-of-bounds. This is not something that is well-defined in the OpenGL viewport spec, so some graphics devices support it, and some don’t. For those that don’t support it, the viewport’s values are clamped to the screen-size, which means the viewport values can’t be defined in a way that makes the viewport go off the screen. As a result of this ambiguity in the spec, WebGPU and the bgfx graphics library used in BabylonNative both clamp the viewport values to keep things consistent across all available hardware. This means the out-of-bounds viewport zooming effect I showed earlier does not work when targeting WebGPU and/or BabylonNative.

Unfortunately, some pre-existing codebases were not designed with these limitations in mind, and we ran into this issue while replacing an application’s older 3D engine with BabylonNative. Changing the application’s architecture would be difficult, so we decided to try solving it a different way. We used a Babylon.js material plugin.

The solution

Viewports essentially just move and scale the scene’s geometry to fit a defined rectangle, so when I asked our graphics guru Alexis how we might work around the out-of-bounds limitation we ran into, he responded by showing me how to use the Babylon.js material plugin API to modify the vertex shader of each material in the scene to move and scale the geometry the same way a viewport does. (Take a look at the material plugin docs here if you’re not familiar with them. They do a good job explaining how to use them).

Here’s a playground that shows a vertex shader material plugin in action when the UI’s “Viewport material plugin” enabled option is checked. If you scroll down to the bottom of the playground code, you'll see this shader code that gets inserted into every material used by the scene ...

gl_Position.x = gl_Position.x * viewport_w * viewport_h / viewport_w + (viewport_x + viewport_w - 1.0 + viewport_x) * gl_Position.w;
gl_Position.y = gl_Position.y * viewport_h + (viewport_y + viewport_h - 1.0 + viewport_y) * gl_Position.w;

This is the code responsible for simulating the viewport, and it allows the viewport values to be as far off the screen as you want. And since it isn’t using the viewport graphics API, it works the same with WebGL, WebGPU and BabylonNative! The code itself is fairly easy to understand. It just scales and moves the given gl_Position by the values you would normally give to the viewport API. The only thing that is a little mysterious is the last multiplication by gl_Position.w on both lines. What is that w? The answer to that question can look complicated, but all it's doing is adjusting the final x and y coordinates according to their depth in the viewing frustrum. As part of the normal rendering pipeline, the GPU will apply the perspective divide by doing gl_Position.xyz / gl_Position.w. Since we want to apply a translation that “survives” this division (because it must occur in screen space), we premultiply the translation by gl_Position.w.

Another noteworthy piece of code is the use of a scissor rectangle to prevent any drawing outside of the defined viewport. It’s not obvious from the examples I’ve shown so far, but if you look at this playground that uses a skybox with the scissor code commented out, you can see the background is drawn outside of the defined viewport. When changing the camera’s viewport values without using the material plugin, the GPU clips anything outside the viewport, but we need to do it manually when the material plugin is enabled. This is why we turn the scissor on and off for each render call when the material plugin is enabled. Try uncommenting the code starting at line 113 to see what effect it has on the skybox. The result should look like this: https://playground.babylonjs.com/#EXCRS4#8

Wrapping it up

So, there you have it! Viewports can be a really handy tool in some situations, and if you find that using out-of-bounds viewports is something you need then you’ll probably want to use the material plugin method I’ve shown here, especially when targeting WebGPU or BabylonNative.

Cheers! Thanks for reading and stay tuned for more to come!

Andy Fillebrown — BabylonNative Team

https://twitter.com/docEdub