Power Monitor

Update shortly...

At the current rate I am posting data from one input at the rate of every 2-3s, and it looks like this will equate to ~3.5GB/yr.

(http://blog.trifork.com/2014/07/29/creating-charts-with-c3-js/)

(http://mbostock.github.io/d3/talk/20111018/tree.html)

The data is static right now, but hopefully I will have it loading dynamically soon. I am going with a different node concept than what emoncms uses. Here, all nodes are virtual devices that provide an output based on processed information from a real device, and after creation are treated as a real device.

Other than handling inputs this is all still 'smoke and mirrors' after this gets further along I will create it's own project page.

It doesn't look like much, but this involved setting up a new router (remember - Backup early and often), and since I was at it, I set up asterisk again (I had two old computers bite the dust over the summer). The instructions on this project page still work http://hackaday.io/project/1486-freepbx-google-voice, with the addition of a few notes.

I had run into a problem with the electric imp where I could post data to my server over the lan from any computer in the house, but the electric imp could not. I had this problem when I had first set up phant, and I had fixed it, but I don't remember how. This is what prompted setting up asterisk again. To get around the problem I set apache up on the new computer with http://freedns.afraid.org just as I had done for the pbx and forwarded the traffic to my flask server.

If anyone knows the solution to use the electric imp on the lan please post it. THANKS!

So, a full day for two lines on a web page and the beginning of finding out how many data points I can hold on my computer.

I am a little less than half complete with porting emoncms ( http://emoncms.org )to python. This was an exercise in just seeing how the cms worked, changing somethings that I didn't like (I particularly dislike php) and extending the capabilities. I also wanted to be able to rapidly test and add data processing filters. I think that I am a day or two from logging data with the cms, and a week from applying filters (in an ideal world). The primary usage is that it will capture any data sent to it so long as a unique identifier is provided (this can be anything so long as it is unique to the user write key), once the system has begun capturing data a node id is mapped to the data stream, and then data processing can be done on a node. While the data processing that comes with emoncms is mostly power related, I wanted something more universal some ideas are: kalman filter, greatest integer, and gradient descent, and some prediction processing. The processing functions are to be modular so that they can be programmed and dropped into the system and run.

I am pulling various rss feeds into the system, because it saves me from clicking all around the internets, and also hopefully at some point I will find the time to play with NLTK.

This is a port of bleextop ( https://github.com/crysfel/Bleextop ) to python for administration (The port was successful and complete, but extending it has met with disastrous results. There is a learning curve with extjs.)

And while there isn't much to see, this is the look of the py-emoncms.

In the end each bit of functionality in the main program should be modular as well, so that features can be add or removed as desired for space considerations and usage needs.

Eventually I would like to add financial capabilities with the yahoo finance api ( https://pypi.python.org/pypi/yahoo-finance/1.1.4 ) and Icarra ( http://www.icarra.com ).

The FreePBX was redirecting to my server, but I will eventually port it as well.

As is usual for me, I have set my goals far beyond what my capabilities and time allow for, but it should keep me busy for a while.

I have been going through a number of sources covering topics from Bayesian statistics, genetic algorithms, machine learning, binomial regression, Bayesian binomial regression, gradient descent, and on and on and on. Right now I am thinking it is going to take me years to wade through the muddy waters of machine learning, understand each branch of mathematics, reread the material, apply it, and I would probably have better luck getting this thing to listen to enlgish and speak in french than I would interpreting my data.

However, I have had a book sitting on my shelf for a couple of years 'Computation: Finite and Infinite Machines' by Marvin L. Minsky. After finally picking up the book and reading the first two chapters I began to understand the first part of the problem I am having. I completely over looked the 'machine' part of machine learning, and the reality of this device. I have been looking at this circuit as some abstract of an idea, where in reality it is a finite machine.

Just as a mechanical adding machine has a finite number of parts and a finite number of outputs based on inputs, so too is the nature of this device.

So from section 2.2 'For a given machine M at a given time t, we can imagine an infinite variety of possible histories. The one that has actually occurred will determine the machine's response to the next stimulus. Now it may be that some events from the very remote past may contribute to determining this response function. If this is the case, one can say that the machine shows some "trace", or "memory", of those remote events. If every ancient event left a separate, independent trace, the machine would need to have infinite capacity, in some sense, to store them.'

I had gotten lost in the infinity because I smoothly skipped over the fundamentals of 'what is a machine'. Now since infinite storage capacity is not practical, nor necessarily useful I hope that I can break the problem down and focus on the finite number of histories that need to be considered. And, hopefully the rest of the book continues to be as helpful as the beginning (and hopefully I am not just fooling myself into thinking that I am understanding).

Over the next couple of months I wil try to replace the electric imp with an esp8266 and make some other revisions.

This isn't really an update. I am just listing links I have come across while trying to figure out how I am going to deal with the data.

http://greg.czerniak.info/guides/kalman1/

http://www.cs.unc.edu/~welch/media/pdf/kalman_intro.pdf
http://www.cs.unc.edu/~welch/kalman/media/pdf/maybeck_ch1.pdf

http://www.cs.unc.edu/~welch/kalman/

http://www.math.ucla.edu/~tom/papers/LiarsDice.pdf

http://www.csee.wvu.edu/~xinl/library/books/stochastic_programming.pdf

Spent some time playing with this in various configurations, but it is inconsistent at best due to the weak magnetic signal from the meter.  I want to order a few different Hall Effect sensors and log the data, but I will also try a photo reflective sensor.

Source:

http://www.instructables.com/id/Monitoring-residential-water-usage-by-reading-muni/?ALLSTEPS