A Smart Fridge With Brains

My speed of progress on this project has been a little slower than anticipated, mostly due to school (BLERGH), but I have made a little bit of progress. To me, the most important part of this concept for a smart fridge is the way (or should I say weigh) I am envisioning it using weight sensors to make intelligent assumptions without user input. Example: you take a carton of milk out of the fridge, pour yourself a glass, and then put the carton back on the fridge shelf. Because the fridge has weight sensors, it knows the weight of the carton before and after you consumed some, so combined with publicly available nutrition information, it can tell you exactly how many calories of milk you consumed, without requiring a single button to be pressed.

Here is the thing though, weight sensing seems to a lot more complex than you might think. There are so many sensors out there that can be easily integrated into MCU projects (simply hook sensor output up to ADC), but I had no idea where to even start with weight. I started researching and it seems like the most commonly used sensor is a strain gauge. This is essentially a flexible film that has an embedded resistor. As the gauge flexes (or is strained), the resistance changes. However, this change in resistance is miniscule, so you need to use the strain gauge in a Wheatstone Bridge configuration in order to actually measure the change in resistance. After that, you need a special kind of amplifier, an instrumentation amplifier, to amplify the signal before it can be fed into a microcontroller, if you want a wide range of values to be read. This video is an excellent overview of the process:

All of this is certainly feasible to do, but the costs start adding up really quickly. For some reason it is really hard to source strain gauges (also often sold in the form of load cells) for cheap. When you add in the cost of an instrumentation amplifier (hard to find under $3 per IC), things really start getting pricey. I think this comment thread (on this HAD article) best sums up the dilemma:

At this point, I think the best option for this project might actually be to just head over to Goodwill (or some other thriftstore) and purchase some already made kitchen scales, then disassemble them and see if there is any easy way to tap into the output. Even on Amazon, I can get a digital kitchen scale for around $7, coming from inside the US, which is LESS than a single order of strain gauges.

In the meantime, while I was thinking about all of this, I was trying to come up with alternative ways to measure weight, other than the go-to method of strain gauges. I happened to have a few Hall Effect sensors laying around, so I came up with a concept for using them to measure the change in distance between a platform (holding the object to be weighed) and a base platform, separated by some sort of spring. Based on price, this might be a more cost-effective method, considering I probably only paid about a penny for each sensor, but much more labor-intensive to implement. I'm not actually going to use this method in my project, but it was something fun to play around with. Here is short video of me goofing around with this concept:

I should also note that I am far from the first person to experiment with using hall effect sensors to measure weight. Hall effect sensors are commonly used in automotive applications for measuring force/displacement:

The best example I could find of a hall-effect sensor scale was actually a project featured on Hackaday previously: http://hackaday.com/2011/01/21/magnetic-digital-scale/.

I am just getting started with this project, which means all I have at the moment is a concept and a list of desired features. However, I have started researching what I would need to bring something like this to life, and have discovered some things worth noting:

The first thing to note is that I will likely not be using OpenCV. When I first thought of attempting a smart fridge with object recognition, my very first thought was that I would have to use OpenCV, which I believed was the industry standard. However, after actually looking at the website and documentation, I think OpenCV, although powerful, is too complex and non-user-friendly for my needs.

Instead of OpenCV, I think I will try using Caffee, which appears to be very powerful, easy to use, and is also covered under a more generous license (BSD). The demo on their site is absolutely amazing and makes it seem as though the software would be a perfect match for my needs.

The other thing I realized is that my original plan, to use a raspberry pi, is probably not a good idea. The computational requirements for near real-time object recognition exceed what the raspberry pi was designed for, and although there have been developments in optimizing Caffee for the device, the fastest it appears to run is at 3 seconds of processing per frame, which is too slow for my needs. I think what I will use instead is an actual laptop or desktop computer. I have an old laptop that is not in use that would be great for running a lightweight linux distro and Caffee, so I will probably try that before spending money on any new hardware.