I borrowed a cheap dial gauge, marked in 0.01mm (10um) increments, and fitted it to the printer so I could watch the height of the gantry while experimenting.

Repeated Z-homings got a surprisingly consistent result: good to within about +/-2-3um...right at the limit of what the gauge could indicate...but only when the microswitch worked. Three out of the 30 homings that I tried were way off, like 100um or more off - this exactly matches my suspicions.

Turning the screw slowly highlighted two things:

• the Z stepper is indeed capable of incredibly fine control/small steps - no noticeable 'stepping' on the gauge, suggesting steps are down at the '5um or less' scale.
• the point at which the microswitch clicks is very consistent, and 100% reliable. So this explanation for why it's not working seems spot-on. The NC contact is breaking slightly early, before the 'tipping point' where the spring slams the armature over to hit the NO contact. Leaving the NC contact happens slowly, so is a bit uncontrolled, compared to hitting the NO contact which happens at great speed ensuring a repeatable make. Designing the end-stop input to use the NC contact is Ender's mistake here IMO.

Also, interestingly, there's about 50um of backlash in Z. So I've modified my bed-levelling script to always move to the target height in an upwards direction - like in normal printing - rather than downwards. That should take care of the backlash effect.