In this post, I show how to display text into frame boxes. I also propose a solution to get more dynamism in the layer. For instance, it allows adding/removing faces in the mesh when the program is running.

Objective

The objective is quite simple: display a text in a frame box, for instance, when characters are speaking:

The main idea is to create a new layer class named UILayer that renders different decorations, starting with frame boxes. It is hard to predict the exact number of required faces/quads in the mesh for this kind of layer. As a result, I propose to update the current facade to allow layers with a face count that can change during the program execution.

Dynamic meshes

Design

There is still the OpenGLMesh class that creates the mesh on the OpenGL side: the vaoId and vboIds attributes contain the OpenGL objects' identifiers. It has many useful sizes, like the size of a pixel in OpenGL coordinates, which ease the computation of locations on the screen or texture. It also handles the texture/tileset and stores related sizes. The novelty is the behavior of setData() and updateData(): both can deal with face counts different from the previous one.

In the OpenGLLayer class, there is now mesh data stored in Numpy arrays. Previously, only some child classes got these arrays, and there were thus duplicate code. This new design simplifies several classes and allows all of them to change the number of their faces. Those who do not need this feature can tell it, thus avoiding the update of GPU data at each rendering. In such a case, the dynamic attribute is False.

Note that the OpenGLLayer class is no more a child class of OpenGLMesh and now contains a reference to an instance of this class in the mesh attribute. Furthermore, we repeat many methods from one class to the other, for example, the draw() method:

def draw(self):
    if self.__dynamic:
        self.__mesh.updateData(self.__vertices, self.__faces, self.__uvMap)
    self.__mesh.draw()

In each case, we call the same method in the OpenGLMesh class, and sometimes we add functionality. In the draw() method case, we automatically update the mesh data if the layer is dynamic.

The mesh attribute in the OpenGLLayer class allows us to delete and recreate the OpenGL mesh data if needed.

The last methods of OpenGLLayer are commodity functions that create, add, and set mesh faces. When this data is ready, we can send data to the GPU using the sendData() method if the layer is static or let the class send it automatically if the layer is dynamic.

Create and add faces

The new addFaces() method adds faces to the current mesh. It also works if the existing mesh is empty:

def addFaces(self, faceCount: int) -> int:
    previousFaceCount = self.faceCount
    newFaceCount = previousFaceCount + faceCount

    if self.__vertices is None:
        self.__vertices = np.zeros([faceCount, 4, 2], dtype=np.float32)
        self.__uvMap = np.zeros([faceCount, 4, 2], dtype=np.float32)
        self.__faces = np.zeros([faceCount, 4], dtype=np.uint)
    else:
        self.__vertices.resize([newFaceCount, 4, 2])
        self.__uvMap.resize([newFaceCount, 4, 2])
        self.__faces.resize([newFaceCount, 4])

    for faceIndex in range(previousFaceCount, newFaceCount):
        self.setFacePixelLocation(faceIndex, 0, 0, self.__mesh.tileWidth, self.__mesh.tileHeight)
        self.setFacePixelTexture(
            faceIndex,
            self.__transparentTile[0], self.__transparentTile[1],
            self.__transparentTile[2], self.__transparentTile[3]
        )

        self.__faces[faceIndex, 0] = faceIndex * 4
        self.__faces[faceIndex, 1] = faceIndex * 4 + 1
        self.__faces[faceIndex, 2] = faceIndex * 4 + 2
        self.__faces[faceIndex, 3] = faceIndex * 4 + 3

    return previousFaceCount

Lines 5-12 create or resize the Numpy arrays. If the new face count is larger than the previous one, the resize() Numpy method keeps the existing data.

Lines 14-25 initialize the new faces with a transparent tile.

The createFaces() method is straightforward: reset the mesh and add the requested faces:

def createFaces(self, faceCount: int) -> int:
    self.__vertices = None
    return self.addFaces(faceCount)

Remove faces

The removeFaces() is more tricky:

def removeFaces(self, firstIndex: int, lastIndex: int):
    assert self.__vertices is not None
    assert 0 <= firstIndex < self.__vertices.shape[0]
    assert 0 < lastIndex <= self.__vertices.shape[0]
    assert firstIndex < lastIndex

    # Delete all data related to the faces to remove
    self.__vertices = np.delete(self.__vertices, range(firstIndex, lastIndex), axis=0)
    self.__faces = np.delete(self.__faces, range(firstIndex, lastIndex), axis=0)
    self.__uvMap = np.delete(self.__uvMap, range(firstIndex, lastIndex), axis=0)

    # Update the face indexes
    faceCount = self.__faces.shape[0]
    for faceIndex in range(firstIndex, faceCount):
        self.__faces[faceIndex, 0] = faceIndex * 4
        self.__faces[faceIndex, 1] = faceIndex * 4 + 1
        self.__faces[faceIndex, 2] = faceIndex * 4 + 2
        self.__faces[faceIndex, 3] = faceIndex * 4 + 3

This method aims to remove all data related to faces between firstIndex (inclusive) and lastIndex (exclusive).

Lines 8-10 use the delete() Numpy function that removes an axis's values in a range. In our case, we want to remove all values of the first axis between the first and last face index.

Lines 13-18 recompute the face indexes for all the faces after the ones we removed.

User Interface Layer

Design

We create a new type of layer with the UILayer class. It contains four methods: createFrameBox() to create a new frame box, showFrameBox() and hideFrameBox() to show and hide, and deleteFrameBox() to remove the frame box.

The OpenGLUILayer class implements these methods. It stores all frame box data in a dictionary of FrameBox instances. Each frame has a unique integer value to identify it. The FrameBox instances directly use the functionalities of an OpenGLLayer instance to show and hide (actually, the OpenGLUILayer instance). These tasks do not require any knowledge specific to a UI layer. We could have put these tasks in the OpenGLUILayer class, but as usual, the more we split code, the better it is.

Draw a frame box

To draw a frame box, we use nine tiles from the following tileset:

We draw each corner at tile size (32 per 32 pixels). For the others, we render them on faces larger than a tile. OpenGL automatically stretches them, so if the tile supports it, the rendering is correct.

Create a new frame box

The createFrameBox() method of the OpenGLUILayer class allocates the data required for a new frame box:

def createFrameBox(self, x1: float, y1: float, x2: float, y2: float) -> int:
    # Create id for new frame box
    self.__lastId += 1
    frameId = self.__lastId

    # Allocate faces
    faceIndex = self.addFaces(9)
    frame = FrameBox(faceIndex, x1, y1, x2, y2, self.tileWidth, self.tileHeight)
    self.__frameBoxes[frameId] = frame

    # Defines the faces
    frame.show(self)

    return frameId

Lines 3-4 creates a new unique id. It is the simplest algorithm for that and is reliable as long as we create less than 2**63 - 1 boxes!

Lines 7-9 do the allocations. For a frame box, we need nine new faces (line 7). The addFaces() method returns the index of the first new face, and the index of the other faces are the following values, from faceIndex+1 to faceIndex+8. Then, we create a new instance of FrameBox with this first face index, as well as the size of the frame and tiles (line 8). Finally, we add the frame to the dictionary (line 9).

Line 12 shows the frame. It defines the location and tiles of its faces.

Show and hide a frame box

We let the frame show and hide: these procedures do not require any data outside the frame data, so there is no reason to do it in the OpenGLUILayer class:

def showFrameBox(self, frameId: int):
    self.__frameBoxes[frameId].show(self)

def hideFrameBox(self, frameId: int):
    self.__frameBoxes[frameId].hide(self)

Delete a frame box

This method is also a bit tricky:

def deleteFrameBox(self, frameId: int):

    # Remove faces used by the frame box
    firstIndex = self.__frameBoxes[frameId].faceIndex
    lastIndex = firstIndex + 9
    self.removeFaces(firstIndex, lastIndex)

    # Remove the frame box from the dict
    del self.__frameBoxes[frameId]

    # Update face indexes
    for frameBox in self.__frameBoxes.values():
        if frameBox.faceIndex >= lastIndex:
            frameBox.faceIndex -= 9

Lines 4-6 compute the index range of the faces used by the frame. Then, it calls the removeFaces() method to remove then. After this call, the face indexes larger than the last index of the frame have changed.

Line 9 removes the frame from the dictionary.

Lines 12-14 update the face indexes larger than the last face index of the frame.

Show a frame box

The show() method of the FrameBox class sets all faces' location and tiles:

def show(self, layer: OpenGLLayer):

    # Local variables (for readability)
    faceIndex = self.faceIndex
    x1 = self.x1
    y1 = self.y1
    x2 = self.x2
    y2 = self.y2
    tileWidth = self.tileWidth
    tileHeight = self.tileHeight

    # Top left
    layer.setFacePixelLocation(faceIndex, x1, y1, x1 + tileWidth, y1 + tileHeight)
    layer.setFacePixelTexture(faceIndex, 0, 0, tileWidth, tileHeight)
    faceIndex += 1

    # Top
    layer.setFacePixelLocation(faceIndex, x1 + tileWidth, y1, x2 - tileWidth, y1 + tileHeight)
    layer.setFacePixelTexture(faceIndex, tileWidth, 0, 2 * tileWidth, tileHeight)
    faceIndex += 1

    # Top right
    layer.setFacePixelLocation(faceIndex, x2 - tileWidth, y1, x2, y1 + tileHeight)
    layer.setFacePixelTexture(faceIndex, 2 * tileWidth, 0, 3 * tileWidth, tileHeight)
    faceIndex += 1

    # Right
    layer.setFacePixelLocation(faceIndex, x2 - tileWidth, y1 + tileHeight, x2, y2 - tileHeight)
    layer.setFacePixelTexture(faceIndex, 2 * tileWidth, tileHeight, 3 * tileWidth, 2 * tileHeight)
    faceIndex += 1

    # Bottom right
    layer.setFacePixelLocation(faceIndex, x2 - tileWidth, y2 - tileHeight, x2, y2)
    layer.setFacePixelTexture(faceIndex, 2 * tileWidth, 2 * tileHeight, 3 * tileWidth, 3 * tileHeight)
    faceIndex += 1

    # Bottom
    layer.setFacePixelLocation(faceIndex, x1 + tileWidth, y2 - tileHeight, x2 - tileWidth, y2)
    layer.setFacePixelTexture(faceIndex, tileWidth, 2 * tileHeight, 2 * tileWidth, 3 * tileHeight)
    faceIndex += 1

    # Bottom left
    layer.setFacePixelLocation(faceIndex, x1, y2 - tileHeight, x1 + tileWidth, y2)
    layer.setFacePixelTexture(faceIndex, 0, 2 * tileHeight, tileWidth, 3 * tileHeight)
    faceIndex += 1

    # Left
    layer.setFacePixelLocation(faceIndex, x1, y1 + tileHeight, x1 + tileWidth, y2 - tileHeight)
    layer.setFacePixelTexture(faceIndex, 0, tileHeight, tileWidth, 2 * tileHeight)
    faceIndex += 1

    # Center
    layer.setFacePixelLocation(faceIndex, x1 + tileWidth, y1 + tileHeight, x2 - tileWidth, y2 - tileHeight)
    layer.setFacePixelTexture(faceIndex, tileWidth, tileHeight, 2 * tileWidth, 2 * tileHeight)
    faceIndex += 1

Hide a frame box

To hide a frame box, we only need to replace the tile of each face with a transparent one, which does the hideFace()method:

def hide(self, layer: OpenGLLayer):
    for faceIndex in range(9):
        layer.hideFace(self.faceIndex + faceIndex)

Final program

Download code & assets

In the next post, I start to create some game data to test the facade.