After much thinking, I decided to use the D0 and D1 pins to select the anodes of the first two digits. This will  get me a working calculator, but the built in serial input / output will be compromised. Since the code is working and further debugging will be simple, this is a tradeoff I am willing to accept at the moment. 

Since the A6 and A7 pins are inputs, they do not need to be disconnected. The quickest way to fix this is to bridge A0 to A6 and A1 to A7 

In the following development prototype, this fix is done very visibly. The wires can be concealed on the back of the unit. 

Green Wires soldered directly on the Arduino and wires routed underneath:

And finally wire wrap, the cleanest solution. 

Here is the first video showing that all the digits work!

Time for a quick instagram post:

https://www.instagram.com/p/Bd6ygN4nJ5-/

https://www.instagram.com/p/Bd6xcynn9ka/