I didn't have time yet to lay it out properly with static diagram of forces but that's the problem I have. @Edward Mallon advised to use washers as a convenient way to add proper amount of weight to the buoy. That solution looks very elegant to me. At the same time my buoy is quite elongated and I think I cannot just add enough weight to the bottom of the buoy. If center of buoyancy and center of gravity are far enough there may appear rotational force that will try to keep the buoy upright.

Somehow I feel it's best to keep buoy axis aligned with tilt line.

Going forward I'll try to make the buoy more compact and that should allow me to use less weight and have the buoy weighted better.

For now, that's my very crude solution which, I hope, distributes weight good enough so my center of buoyancy and center of gravity are close enough.

I don't have any solid proof to my considerations (as to if my concerns above are valid at all) so if you have any thoughts, please kindly share with me.