Hmm, you're making me wonder: maybe this could be done also with TTL logic, without too much hassle?  Seems like it could be an interesting project.

Of couse, the basic arithmetic element would just be an accumulator and some multiplexers or xor gates. What is a bit more complex, is how to get all the data in. But you could start small and use a microcontroller to do the shuffling.

Also, analog neurons could be made the same way.

Well, I am working on a TTL + uPD7220 video card, and getting all the data in is a big task there. For sure, there are good ways to load the inputs or strobe the weights - it is a similar issue, at least if you care about speed. What may be really interesting is making it both forward propagate and learn - although I'm sure this is also much more difficult. Anyway - thank you for sharing your project, it is really cool, as well as inspiring.