Lack of updates hasn't meant lack of progress here. Since I last wrote, two things have happened:

I've built a mock-up for one potential solution to my battery holder problem.

First, I laser-cut a shape into some acrylic:

Then, I cut and sanded an angle into an acrylic tube:

It seems to slide onto the cutout pretty easily, and then rotating the tube so that the long end is wedged into the side cuts, and locks into place.

It seems to work decently well. The tube has an inner diametre of 12mm, just over the LR44 battery's 11.7mm, so they fit with only a little bit of rattle.

The other thing I've done is sent off a PCB for manufacturing. Not actually OSHPark, unfortunately, so I don't expect to see it until the end of the month at the earliest.