The simulator, which uses the PC variant of the firmware, works great but has one drawback for quick testing: interacting with the device is manual. This means that each time, I have to click on the buttons (which represent real buttons) to navigate through the menus and observe how various layouts are rendered.

I started thinking about whether this process could be automated and came up with a solution: using pytest, which is likely the most popular test framework for Python.

It works, and a sample HTML report is shown below:

There are some glitches in the images, and I think the images appear a bit blurred. Since I don't have reference images to verify how it should look, the test only displays the images; it doesn't determine pass or fail.

Here is what the test looks like for the report above:

def test_iterate_list(assert_display_content):
    """Test to validate iteration through a list using display content."""
    logger.info("Starting test: test_iterate_list")

    dut: Simulation = Simulation()
    dut.start_firmware()
    logger.info("Firmware started for the simulation.")

    logger.info("Capturing initial display state.")
    assert_display_content(dut, "List Iteration Test 1", "test_iterate_list_1.png")

    logger.info("Simulating DOWN key press and release.")
    push_key(dut, SimulationKey.DOWN)
    time.sleep(0.1)

    logger.info("Capturing updated display state.")
    assert_display_content(dut, "List Iteration Test 2", "test_iterate_list_2.png")

    logger.info("Simulating DOWN key press and release.")
    push_key(dut, SimulationKey.DOWN)
    time.sleep(0.1)

    logger.info("Capturing updated display state.")
    assert_display_content(dut, "List Iteration Test 3", "test_iterate_list_3.png")

    logger.info("Simulating DOWN key press and release.")
    push_key(dut, SimulationKey.DOWN)
    time.sleep(0.1)

    logger.info("Capturing updated display state.")
    assert_display_content(dut, "List Iteration Test 4", "test_iterate_list_4.png")

    dut.stop_firmware()
    logger.info("Firmware stopped for the simulation.")

This feature was surprisingly easy to implement. I used the simulation.py class (which I developed for simulation) to wrap the .so library in Python, allowing for seamless communication with the firmware, and then connected it to pytest using its fixtures.

One tricky aspect was discovering that pytest looks for its fixtures in conftest.py. Fixtures won’t work if they’re located in the same file as the tests. It took me a while to figure this out.

Now, I can run this lengthy one-liner in the command line (inside Docker) to compile and test the firmware:

cd /workspace/build/ && cmake .. && make -j24 && cd /workspace/Test/SystemTests && pytest test_display.py -s  --html=report.html

 Then I refresh the page with the test results in Google, and I have all the results displayed in an easy and predictable way.

Link to commit.