I have been trying to find a proper framework that can work as I want it to.

The service that comes to mind is Jitsi. It seems it has a webRTC based system that I can use : Github Link here.

This now brings me several challenges. I am still studying Jitsi.