I wrote an Arduino sketch to transmit FSQ messages and have the serial port tell me what it's doing.

It didn't work.

The program would hang on first initialising the SI5351.

Remove the "Serial" lines from the code. It works. WTF.

I wonder if this is a bug with this old Arduino Nano or something worse.

Might try to use SoftwareSerial on some other pins.