So I have found some spare time, and willing to continue on this project, and now trying to make it finally work. For the display to work as intended, I have to add yet another board, which would do things properly instead of buggy parts that were assembled previously. One thing with old boards which prevents re-use is epoxy coating I did on their underside, to protect from shorting.

For greatly reducing ghosting I found that reducing refresh rate works really well.

Below is the photo of display, as it is looks like now (it shows sequence 1234):

You can discern digits, if keeping in mind that active segments are dark.

The board in the lower left corner is b&w PAL TV signal generator, acting as interim frequency generator (based on 16MHz quartz and 74hc163 counters), while the original one is unconnected.

On the right is the small board with 4 2NAND gates wired as one XOR, connected to D flip-flops, so together they make synchronous counter.