Managed to scale down the PCB from 50mm x 50mm to 40mm x 40mm, this is the smallest I can make it.

Also, thanks to @flashcactus's suggestion that some people might want a ISP header I added one to the board. It can be soldered if needed but the MCUs will come with a bootlaoder flashed on to them so they can be reprogrammed over USB.

The updated files are also available on GitHub.