In this post, I show how to display text into frame boxes. I also propose a solution to get more dynamism in the layer. For instance, it allows adding/removing faces in the mesh when the program is running.
Objective
The objective is quite simple: display a text in a frame box, for instance, when characters are speaking:
The main idea is to create a new layer class named UILayer
that renders different decorations, starting with frame boxes. It is hard to predict the exact number of required faces/quads in the mesh for this kind of layer. As a result, I propose to update the current facade to allow layers with a face count that can change during the program execution.
Dynamic meshes
Design

There is still the OpenGLMesh
class that creates the mesh on the OpenGL side: the vaoId
and vboIds
attributes contain the OpenGL objects’ identifiers. It has many useful sizes, like the size of a pixel in OpenGL coordinates, which ease the computation of locations on the screen or texture. It also handles the texture/tileset and stores related sizes. The novelty is the behavior of setData()
and updateData()
: both can deal with face counts different from the previous one.
In the OpenGLLayer
class, there is now mesh data stored in Numpy arrays. Previously, only some child classes got these arrays, and there were thus duplicate code. This new design simplifies several classes and allows all of them to change the number of their faces. Those who do not need this feature can tell it, thus avoiding the update of GPU data at each rendering. In such a case, the dynamic
attribute is False
.
Note that the OpenGLLayer
class is no more a child class of OpenGLMesh
and now contains a reference to an instance of this class in the mesh
attribute. Furthermore, we repeat many methods from one class to the other, for example, the draw()
method:
def draw(self): if self.__dynamic: self.__mesh.updateData(self.__vertices, self.__faces, self.__uvMap) self.__mesh.draw()
In each case, we call the same method in the OpenGLMesh
class, and sometimes we add functionality. In the draw()
method case, we automatically update the mesh data if the layer is dynamic.
The mesh
attribute in the OpenGLLayer
class allows us to delete and recreate the OpenGL mesh data if needed.
The last methods of OpenGLLayer
are commodity functions that create, add, and set mesh faces. When this data is ready, we can send data to the GPU using the sendData()
method if the layer is static or let the class send it automatically if the layer is dynamic.
Create and add faces
The new addFaces()
method adds faces to the current mesh. It also works if the existing mesh is empty:
def addFaces(self, faceCount: int) -> int: previousFaceCount = self.faceCount newFaceCount = previousFaceCount + faceCount if self.__vertices is None: self.__vertices = np.zeros([faceCount, 4, 2], dtype=np.float32) self.__uvMap = np.zeros([faceCount, 4, 2], dtype=np.float32) self.__faces = np.zeros([faceCount, 4], dtype=np.uint) else: self.__vertices.resize([newFaceCount, 4, 2]) self.__uvMap.resize([newFaceCount, 4, 2]) self.__faces.resize([newFaceCount, 4]) for faceIndex in range(previousFaceCount, newFaceCount): self.setFacePixelLocation(faceIndex, 0, 0, self.__mesh.tileWidth, self.__mesh.tileHeight) self.setFacePixelTexture( faceIndex, self.__transparentTile[0], self.__transparentTile[1], self.__transparentTile[2], self.__transparentTile[3] ) self.__faces[faceIndex, 0] = faceIndex * 4 self.__faces[faceIndex, 1] = faceIndex * 4 + 1 self.__faces[faceIndex, 2] = faceIndex * 4 + 2 self.__faces[faceIndex, 3] = faceIndex * 4 + 3 return previousFaceCount
Lines 5-12 create or resize the Numpy arrays. If the new face count is larger than the previous one, the resize()
Numpy method keeps the existing data.
Lines 14-25 initialize the new faces with a transparent tile.
The createFaces()
method is straightforward: reset the mesh and add the requested faces:
def createFaces(self, faceCount: int) -> int: self.__vertices = None return self.addFaces(faceCount)
Remove faces
The removeFaces()
is more tricky:
def removeFaces(self, firstIndex: int, lastIndex: int): assert self.__vertices is not None assert 0 <= firstIndex < self.__vertices.shape[0] assert 0 < lastIndex <= self.__vertices.shape[0] assert firstIndex < lastIndex # Delete all data related to the faces to remove self.__vertices = np.delete(self.__vertices, range(firstIndex, lastIndex), axis=0) self.__faces = np.delete(self.__faces, range(firstIndex, lastIndex), axis=0) self.__uvMap = np.delete(self.__uvMap, range(firstIndex, lastIndex), axis=0) # Update the face indexes faceCount = self.__faces.shape[0] for faceIndex in range(firstIndex, faceCount): self.__faces[faceIndex, 0] = faceIndex * 4 self.__faces[faceIndex, 1] = faceIndex * 4 + 1 self.__faces[faceIndex, 2] = faceIndex * 4 + 2 self.__faces[faceIndex, 3] = faceIndex * 4 + 3
This method aims to remove all data related to faces between firstIndex
(inclusive) and lastIndex
(exclusive).
Lines 8-10 use the delete()
Numpy function that removes an axis’s values in a range. In our case, we want to remove all values of the first axis between the first and last face index.
Lines 13-18 recompute the face indexes for all the faces after the ones we removed.
User Interface Layer
Design

We create a new type of layer with the UILayer
class. It contains four methods: createFrameBox()
to create a new frame box, showFrameBox()
and hideFrameBox()
to show and hide, and deleteFrameBox()
to remove the frame box.
The OpenGLUILayer
class implements these methods. It stores all frame box data in a dictionary of FrameBox
instances. Each frame has a unique integer value to identify it. The FrameBox
instances directly use the functionalities of an OpenGLLayer
instance to show and hide (actually, the OpenGLUILayer
instance). These tasks do not require any knowledge specific to a UI layer. We could have put these tasks in the OpenGLUILayer
class, but as usual, the more we split code, the better it is.
Draw a frame box
To draw a frame box, we use nine tiles from the following tileset:

We draw each corner at tile size (32 per 32 pixels). For the others, we render them on faces larger than a tile. OpenGL automatically stretches them, so if the tile supports it, the rendering is correct.
Create a new frame box
The createFrameBox()
method of the OpenGLUILayer
class allocates the data required for a new frame box:
def createFrameBox(self, x1: float, y1: float, x2: float, y2: float) -> int: # Create id for new frame box self.__lastId += 1 frameId = self.__lastId # Allocate faces faceIndex = self.addFaces(9) frame = FrameBox(faceIndex, x1, y1, x2, y2, self.tileWidth, self.tileHeight) self.__frameBoxes[frameId] = frame # Defines the faces frame.show(self) return frameId
Lines 3-4 creates a new unique id. It is the simplest algorithm for that and is reliable as long as we create less than 2**63 – 1 boxes!
Lines 7-9 do the allocations. For a frame box, we need nine new faces (line 7). The addFaces()
method returns the index of the first new face, and the index of the other faces are the following values, from faceIndex+1
to faceIndex+8
. Then, we create a new instance of FrameBox
with this first face index, as well as the size of the frame and tiles (line 8). Finally, we add the frame to the dictionary (line 9).
Line 12 shows the frame. It defines the location and tiles of its faces.
Show and hide a frame box
We let the frame show and hide: these procedures do not require any data outside the frame data, so there is no reason to do it in the OpenGLUILayer
class:
def showFrameBox(self, frameId: int): self.__frameBoxes[frameId].show(self) def hideFrameBox(self, frameId: int): self.__frameBoxes[frameId].hide(self)
Delete a frame box
This method is also a bit tricky:
def deleteFrameBox(self, frameId: int): # Remove faces used by the frame box firstIndex = self.__frameBoxes[frameId].faceIndex lastIndex = firstIndex + 9 self.removeFaces(firstIndex, lastIndex) # Remove the frame box from the dict del self.__frameBoxes[frameId] # Update face indexes for frameBox in self.__frameBoxes.values(): if frameBox.faceIndex >= lastIndex: frameBox.faceIndex -= 9
Lines 4-6 compute the index range of the faces used by the frame. Then, it calls the removeFaces()
method to remove then. After this call, the face indexes larger than the last index of the frame have changed.
Line 9 removes the frame from the dictionary.
Lines 12-14 update the face indexes larger than the last face index of the frame.
Show a frame box
The show()
method of the FrameBox
class sets all faces’ location and tiles:
def show(self, layer: OpenGLLayer): # Local variables (for readability) faceIndex = self.faceIndex x1 = self.x1 y1 = self.y1 x2 = self.x2 y2 = self.y2 tileWidth = self.tileWidth tileHeight = self.tileHeight # Top left layer.setFacePixelLocation(faceIndex, x1, y1, x1 + tileWidth, y1 + tileHeight) layer.setFacePixelTexture(faceIndex, 0, 0, tileWidth, tileHeight) faceIndex += 1 # Top layer.setFacePixelLocation(faceIndex, x1 + tileWidth, y1, x2 - tileWidth, y1 + tileHeight) layer.setFacePixelTexture(faceIndex, tileWidth, 0, 2 * tileWidth, tileHeight) faceIndex += 1 # Top right layer.setFacePixelLocation(faceIndex, x2 - tileWidth, y1, x2, y1 + tileHeight) layer.setFacePixelTexture(faceIndex, 2 * tileWidth, 0, 3 * tileWidth, tileHeight) faceIndex += 1 # Right layer.setFacePixelLocation(faceIndex, x2 - tileWidth, y1 + tileHeight, x2, y2 - tileHeight) layer.setFacePixelTexture(faceIndex, 2 * tileWidth, tileHeight, 3 * tileWidth, 2 * tileHeight) faceIndex += 1 # Bottom right layer.setFacePixelLocation(faceIndex, x2 - tileWidth, y2 - tileHeight, x2, y2) layer.setFacePixelTexture(faceIndex, 2 * tileWidth, 2 * tileHeight, 3 * tileWidth, 3 * tileHeight) faceIndex += 1 # Bottom layer.setFacePixelLocation(faceIndex, x1 + tileWidth, y2 - tileHeight, x2 - tileWidth, y2) layer.setFacePixelTexture(faceIndex, tileWidth, 2 * tileHeight, 2 * tileWidth, 3 * tileHeight) faceIndex += 1 # Bottom left layer.setFacePixelLocation(faceIndex, x1, y2 - tileHeight, x1 + tileWidth, y2) layer.setFacePixelTexture(faceIndex, 0, 2 * tileHeight, tileWidth, 3 * tileHeight) faceIndex += 1 # Left layer.setFacePixelLocation(faceIndex, x1, y1 + tileHeight, x1 + tileWidth, y2 - tileHeight) layer.setFacePixelTexture(faceIndex, 0, tileHeight, tileWidth, 2 * tileHeight) faceIndex += 1 # Center layer.setFacePixelLocation(faceIndex, x1 + tileWidth, y1 + tileHeight, x2 - tileWidth, y2 - tileHeight) layer.setFacePixelTexture(faceIndex, tileWidth, tileHeight, 2 * tileWidth, 2 * tileHeight) faceIndex += 1
Hide a frame box
To hide a frame box, we only need to replace the tile of each face with a transparent one, which does the hideFace()
method:
def hide(self, layer: OpenGLLayer): for faceIndex in range(9): layer.hideFace(self.faceIndex + faceIndex)