Close

Log #25: nicegui

A project log for lalelu_drums

Gesture controlled percussion based on markerless human-pose estimation with an AI network from a live video.

lars-friedrichLars Friedrich 3 hours ago0 Comments

A key aspect of lalelu_drums' architecture is the separation into a backend and a frontend, making it a very suitable use case for a web UI (see figure 1 in the project details). Up to now, a dedicated software is required to run on the frontend device to communicate with the backend and provide a graphical user interface. If instead, the backend served a web UI, only a browser would be required on the frontend device. This way, the frontend device could be exchanged more easily. Moreover, an architecture built around a web UI should support the operation of multiple frontend devices at the same time, which could be a helpful feature.

nicegui is a python based UI framework, that seems very suitable for creating a web UI for lalelu_drums. While nicegui is easy to use and a lot of examples are provided, there are still a few aspects that were not immediately clear to me and I want to discuss my findings in this log entry.

1) Multiple clients
When designing a web UI, one must consider what should happen if the UI is opened multiple times, e. g. in multiple tabs of the same browser or on different devices. Depending on the application the desired behaviour can be different. In the case of lalelu_drums, there is a single hardware ressource and accordingly, the state of that hardware needs to be synchronized between the different instances of the UI. Following the nicegui example on device control, this can be done by separating the hardware model from the UI and using events to signalize changes in the hardware state to the UI.

2) Hardware state vs. UI state
Once it is clear that there is a single hardware state that needs to be synchronized, one has to realize that the different instances of the UI can still be in different states even though being connected to the same hardware. For example, one browser tab could show a live video while another tab could show a dashboard on performance statistics.

3) Object oriented programming
The modularization class_example.py of nicegui gives an idea of how to achieve a separation of different UI pages into classes. One has to understand that the function decorated with @ui.page is called every time a client opens the respective page. It is worth noting that in class_example.py the @ui.page decorator is used inside the constructor so there is only a single instance of that class and the self object is shared among all clients. If, in contrast, the goal is to have one object of a UI-class per client, the constructor needs to be called in the @ui.page decorated function, as it is done in my example below.

The following script provides an architecture that takes the aforementioned aspects into account. The hardware state is shared via the Model class using events. The MyUi class provides a parameterized way to setup a page. It is instantiated in the functions decorated with @ui.page.

from nicegui import ui, Event, app


class Model:
    def __init__(self):
        self.state = False

        self.evtStateChange = Event()
    
    def toggle(self):
        self.state = not self.state
        self.evtStateChange.emit()


class MyUi:
    def __init__(self, model: Model, name, target):
        self.model = model
        self.name = name

        if name == 'main page':
            ui.colors(primary='#CF3F23')
        elif name == 'sub page':
            ui.colors(primary='#23CF76')

        with ui.header():
            ui.label(name.upper())
        self.checkBox = ui.checkbox('state', value=self.model.state)
        self.checkBox.disable()
        ui.button('toggle', on_click=self.model.toggle)
        ui.link('link', target)

        model.evtStateChange.subscribe(self.onStateChange)
    
    def onStateChange(self):
        self.checkBox.value = self.model.state
        print(f'{self.name} {id(self)}, {self.model.state}')


model = Model()

@ui.page('/')
def buildPage():
    myUi = MyUi(model, 'main page', '/subpage')
    print(f'opening / {id(myUi)}')
    
@ui.page('/subpage')
def my_page():
    myUi = MyUi(model, 'sub page', '/')
    print(f'opening /subpage {id(myUi)}')


def onDisconnect(client):
    print(f'disconnect: {id(client)}')

def onDelete(client):
    print(f'delete: {id(client)}')

app.on_disconnect(onDisconnect)
app.on_delete(onDelete)

ui.run(reload=False)

The following screen cast shows the example code in action.

Discussions