The bug was in the button checking code and was causing the debounce filter to work every cycle of infinite loop.

bool button_pressed(button_t *btn)
{
    bool pressed = FALSE;
    BitStatus current_status = !!GPIO_ReadInputPin(btn->gport, btn->gpin);

    if (btn->_last_status != current_status) {
        delay_ms(5);
        current_status = GPIO_ReadInputPin(btn->gport, btn->gpin);
    }
    ...

There is a typo here that gets compiled and causes an increase in the response time of the remote by 30 ms (6 buttons, 5 ms for debouncing). The GPIO_ReadInputPin() function returns not a bool value, but a bitmask, i.e. 0 if logic level on the pin is low and (1 << pin) if the logic level on the pin is high.

So the right code looks like that:

bool button_pressed(button_t *btn)
{
    bool pressed = FALSE;
    BitStatus current_status = !!GPIO_ReadInputPin(btn->gport, btn->gpin);

    if (btn->_last_status != current_status) {
        delay_ms(5);
        // Forgot to normalize to 0..1
        current_status = !!GPIO_ReadInputPin(btn->gport, btn->gpin);
    }
    ...