I was building a new one for a friend. In the process some updates were made:

Software:

To change network you press the button when the needle is performing the initial test scan after powerup. The needle will position it self in the middle and WiFi network "MCViser" will be active. Connect to the network to connect it to your WiFi and to give it the address to the minecraft server.

This is how it was ment to work before, but now it actually works every time.

Hardware:

I found that the esp01 antenna was to close to the servo and cables. With a riser in the socket for the esp it became much more stable. (I did only have 1x6 high female pin header, but two did the trick.)