This thing is normally made for pressure.  Switching the valve around to make it do vacuum ends up putting a different kind of load on the arm that it wasn't designed for.

As the arm is pulled in by the diaphragm contracting from the increasing vacuum, it actually gets pulled out of the strongest area of the swinging magnetic field and vacuum performance then drops off.

So I realigned the magnet to correct for this by bending the arm out away from the diaphragm.

Now under maximum vacuum, we are still within the stronger area of the field.  The arm can generate more force, and the maximum attainable vacuum increases.

If I was starting over, this is probably the only change I would make now.  I'll take the increased pumping volume the Neodymium magnet gave me, but a higher vacuum is more useful in the end.  Just bending the arm is a lot simpler too..