Amping up your Babylon.js GUI with the power of the grid
The grid is one of the foundations of all design practice and is apparent in almost everything we see from user interfaces to print layouts. Babylon.js is growing our graphical user interface (GUI) controls by adding grids into the system, which is one of the most impactful features of the 3.3 release. To help demonstrate the control we now have in the GUI system I decided to tackle an expansion to one of our existing GUI controls, the color picker.
First, a little background about me. I have been a visual designer for decades and more recently a 3D artist and technical artist. I have some background in writing code for the web (HTML, CSS, and PHP) and for game engines (C#) but I am far from an expert. I exist somewhere between artist and engineer and have very little experience with JavaScript which makes this project a perfect test case for the ease of building for and contributing to Babylon.js.
We already have a color picker GUI control in the engine, but it is only the graphical color wheel that will return a Color3. What if I want to invoke a color picker to assign a specific color? And what about saving a color to use later, how can we do that? We have all seen color picker windows in the software we use daily and they all have similarities — the ability to manually input RGB or hexadecimal values, a clickable graphical representation of a color wheel, and a palette of saved colors to be reused. This was a good test case for grids and to understand how complex the code to create a fully featured color picker window would be.
Designing the window
The first thing I did was to sketch out what I thought we needed in terms of controls and these are the inputs I decided upon.
- Color wheel picker which I would be using our current control, BABYLON.GUI.ColorPicker()
- Swatches for the current and last selected colors to help the user choose a new color based on the last one picked.
- Text input for RGB values in 8-bit integers (0–255)
- Text input for RGB values in floating point numbers (0–1) which is the default color input for Babylon.js
- Text input for hexadecimal color values (#000000 — #ffffff) which is the default for GUI controls
- Live update of all values in the window as any individual value is changed by user input
- A save function that will save the currently selected color into a drawer for later use
- Error checking to prevent the user from entering invalid values or characters and avoid failing to return a color
With those guidelines laid out I sketched out a sample window with some grid lines so I could get some base measurements. I used Adobe Illustrator for this step, but it can be done with any tool you are comfortable with. I find that using a workflow that most artists employ, sketching out your composition in low fidelity to help make choices and plan your final composition, allows me to know what I need to build before I write my first line of code. I am more prepared for the structure I need to build and can refer to my sketch to see how the grids need to nest if I get lost while writing the code.
The power of the grid is that I can easily create rows and columns to align to my layout and assign controls to a specific cell. I could certainly lay out the whole window with a single cell and style each control’s position manually using an offset to the top and left of the control, but that method does not work well for responsive design or quickly editing the placement of groups of controls. To help understand the complexity of the nested grids, I created the diagram below to illustrate the individual grid controls, how many rows and columns each has, and where each is placed in the hierarchy.
Building the system
Now that I have a plan for the window, I set out to write the functions that would invoke the window on demand. For the prototype, I simply added an event listener to call the function on a key press. As I was adding the grid controls into the window I assigned a random color background to each control to determine that I had the correct layout before adding any of the input or button controls. At the last level of the grid I needed to ensure I had the blocking done correctly for the buttons and input fields. For this, I inserted a function to create a BABYLON.GUI.Rectangle() with a width and height of 100% and a random background color and called that function in each cell that would have an input.
After setting up the grid as I needed, I simply added a control into each of the cells as needed: BABYLON.GUI.ColorPicker(), BABYLON.GUI.InputText(), BABYLON.GUI.CreateSimpleButton(), and BABYLON.GUI.Text(). Once the controls are incorporated, having access to the new debug display features in the revamped inspector helps to ensure everything is aligned as shown below. The greatest part of the complexity came not from the layout of the window, but rather from wanting to update all the controls live as the user made changes through click or keyboard input.
Checks and balances
To offer the greatest flexibility, I leveraged the name property of controls giving each control a unique name. This serves two purposes. The first is that your control will appear in the tree structure of the inspector with a descriptive name allowing you to easily navigate to the control you need. The second is that I can test against the name of the control to know if I should update its value or not. I created a function that would update all controls at once and passes the value of the currently focused control’s name to the function. Each control name is compared with the name of the focused control and only those who don’t match are updated.
The result is that when the user types a number the rest of the values for other controls are updated and a new color evaluated, but the input the user is manipulating does not change. This becomes especially important when editing the float values for a channel because the values “0”, “0.”, and “.” should all evaluate to 0 for updating the color, but we don’t want to change the field the user is actively editing or they won’t be able to type the value they are intending.
There was also some complexity with entering a hexadecimal string as we also wanted to update that on the fly without the need to hit enter to confirm. To do this I simply check the hexadecimal string to see if it contains fewer than 6 characters. If it does, I add one or more “0” characters to the start of the string until it is 6 characters long. Again, the focused control isn’t updated, so the user does not see the leading “0” characters unless they change focus away from the hexadecimal control in which case we do update the string with the correct value with leading “0” characters. This allows us to pass valid color data to the other controls so they will update correctly as the hexadecimal is edited.
The other small feature I added for hexadecimal values is to automatically understand shorthand representation. A shorthand representation of a hexadecimal looks like “#abc” which evaluates to “#aabbcc”. To accomplish this I added a test in the hexadecimal functions to check if at any time the string was exactly three characters. If so, instead of adding leading “0” characters, I instead evaluate the string as an expanded hexadecimal generated from the shorthand string. This means that if you want to use a shorthand color, you only need to type the shorthand value and the color will evaluate correctly. Once the control loses focus, the shorthand string is converted to the full hexadecimal, but I wanted to allow the user the ability to speed up their input for common colors.
The last bit of work in the input controls was to add regular expression checking to make sure that each control would only receive the correct characters and any invalid characters would revert the focused control to the last valid entry. This prevents the user from entering a character that would not evaluate correctly and filling all controls with “NAN”, or not-a-number, warnings. This is especially important because the GUI controls all use strings for input so there is a lot of converting between strings and numbers to get everything to work.
Saving progress
The final step in creating the window was to add the saved color drawer. The window takes, as part of its constructor, a string array for saved colors. I chose to go with a string array instead of a color array because all the GUI controls require a hexadecimal string for color. Passing the color back to the application from the color picker window would require one conversion to a Color3, but bringing in a Color3 array would require many conversions to hexadecimal to set colors on controls.
When creating the color picker window, if the saved colors array is not empty, a new control is added below the main window and a color swatch represented by a button is created for each saved color with that color as the background. Clicking on a color swatch button will set the active color in the window to that saved color allowing users to reuse a color without entering the values manually. The size of the color swatch and the number of swatches per row is configurable and will auto wrap to another row if enough colors are saved.
There is also a configuration for the total number of colors that can be saved to prevent the drawer from getting too large. Once the limit is hit the Save Color button is disabled until some of the saved colors are deleted. Clicking on the Edit Saved Colors button changes the color swatch buttons to have a delete icon in the center of them as converts their behavior from changing the active color to removing that color from the saved colors array.
Due to the complexity of the interactions, error checking, and layout of the color picker window, we have decided to push it as a class into Babylon.js and can be called as needed in its complete state. Currently I am working on converting the prototype to TypesScript and making the window size configurable so that the contained controls automatically reflow based on the overall size. This will be my first contribution to the code base of Babylon.js as my role on the team has been as a creative lead, not an engineer. However, I will say that the process of contributing is very straight forward and if you are interested in contributing as well, you can find everything you need at https://doc.babylonjs.com/how_to/how_to_start
I hope this gives you some insight into how you can leverage grids in Babylon.js as well as a peek into how a feature in the engine can come from a simple exploration to make a task easier. The project was a fun exploration of the GUI system and I have some thoughts for how to expand upon this feature. But that will be the topic for a future post.
Patrick Ryan — Babylon.js creative lead
