Cassapa - Augmented Reality Pool Game

Finally revamped/cleaned the code and uploaded it to github:

https://github.com/aporto/cassapa

Since this is a glassless augmented reality system, I'm facing some lighting problems related to the interaction between the camera and the projector.

The first problem is simple: The camera needs a lot of ambient light to detect the image. However, the projector works better on darker environments. Finding the best compromise between these may be tricky. And, with my first set, I didn't have much choice, because of my cheap projector. Now I'm moving to a better one.

A possible solution for that is to use a non-visible source of light. For example, a torch made out of IR leds. That would not interfere with the projector image, but would still be "visible" by the camera sensors. The problem here is that we would loose any color information on the image. You can see, by the simulation of the next image, that it would be harder to identify the balls against the cloth without the contrast of their colors

As pointed by the red arrow, we also have some visual elements that are easily confused with a ball (The light reflecting on my hand, at this example). Another detection that is harder to do without color information

Another problem is that the projector image is projector over the image captured by the camera. So, the captured image will include all drawings made by the projector, and them may interfere with the computer vision. For example, the software may confuse a projected aiming line with the cue. I'm dealing with this by adjusting the brightness of the projected image, so it will only create "translucent" drawing over the table cloth that I can easily filter out via software.

It seems that the common-sense for Augmented Reality is something related to what we see. I found out that I was struck with this "vision" (tum-tum-tss) when I realized that what I wanted is a system to help me aiming the cue, and that doesn't mean this help shall be visual.

So, since I'm still having problems to attach the projector permanently on the ceiling, I've added an audio output to Cassapa, to help the aiming when the projector is not available. It plays an continuous sound when the cue is aiming a ball, and the changes the pitch according to the trajectory forecast. Pitch increases as the probability of the ball hitting the pocket increases.

Currently I'm doing this with FMOD library, which has a non-commercial license, but I'm still looking for a fully open alternative. Since this is functionality is not mandatory for Cassapa, it can be completely removed if you don't agree with free-as-beer-but-not-free-as-speech software.

To get rig of those ugly wires and equipments hanging on the ceiling of my living room, I'm preparing a cover that will also work as a lamp for the pool table. 

I made this box with a 1mm sheet of aluminum. It's just a long metal rectangle, bended to make a square,

So, I've got a mini-PC installed on the ceiling for the first prototype.  This is a Zotac box, with an Intel D525 processor. Its performance for the computer vision software is good (Sometimes 17 frames/sec). The camera is a Logitech C525, which has a very nice articulated base, and can be easily adjusted to point the correct direction.

Unfortunately, there's no room for the projector on the ceiling. All projectors I found need at lease 3 meters from the projection screen, and I have less than 2 m from the ceiling to the table surface. There are some low-distance projectors out there, but they are expensive. I'll try to put the projector at another position, and then fix the image distortion via software or via the projector.

Wife didn't get very much happy, so I'm already planning a lamp or something else to cover it. :)