I've designed a new version that I haven't built yet, but it's looking pretty cool. The changes are:

• No need for batteries to power the amplifier. I have some doubts about whether this will work, as the first stage is a bit tricky (with tens of gigaohms as a feedback resistor), but I will give it a try. Now, I use a small DC/DC converter to create a negative voltage for the amplifier. Everything is powered by a 9V wall adapter. There are still batteries to polarize the ionization chamber, but since it draws almost no current, it's fine.
• Made it more compact. I just want something I can put on my shelf, connect to a data logger, and monitor background radiation. I have made a data logger and I want to use this device as a source of data for it, you can check this project on Hackaday, I have 112 followers here! :)) 

The device now consists of four parts:

• Ionization chamber (I was considering ordering it to be cut out on a CNC, but it's too expensive).
• Polarization for the ionization chamber (three 12V batteries instead of just two). I hope this will increase sensitivity.
• Amplifier PCB (it has spaces for the mentioned batteries). The metal chassis is the same.
• PCB with data processing and power supply handling (same ADC, same microcontroller).

To summarize, the new version will be just more robust (if it works)!

I will share the diagrams I made below. If you have any remarks, I would appreciate them.

Below is the amplifier PCB and its rendering.

Below is the acquisition module and its rendering.

I hope to have time soon to build this version and share the results with you.