In the previous log we looked at how I attempted to simulate the ADC readings from an LDR by taking a year of solar radiation data from a photovoltaic study, converting coarsely from W/m2 to Lux, characterising an LDR against the TSL2591 and then using that to convert to the expected ADC readings under the sun on those days recorded in the photovoltaic study.

In this log, I will show you how I started to apply filters to try and pick out data phenomenon that would be easy for a microcontroller to discern as a particular phase point in the day.

For our first graph, we're looking at W/m2 values on the left side - we are still in Excel here, not on the microcontroller. This is a plot of a week in Jan 2014 in London, with the raw readings in blue and then increasing windows over which the readings were averaged, for 3 hours, 6 hours and 12 hours. 

As might be expected, averaging smooths out the double peak seen during the first day on the left side of the graph, although it introduces a phase shift and reduces the magnitude of the peaks. 

You can even apply a basic threshold, such as checking the previous reading was below the last 24 hours median and this reading is over it:

But as you can see, there will be times when a dull day does not get a trigger if the day before was bright and some days apparently double trigger (probably due to bright then dark then bright again spells).

So averaging on its own is of limited use.

Next I looked at measuring the steepness and direction of the slope - is it getting brighter or dimmer? After all we are looking for things like dusk/dawn as markers of the time of day. This is effectively a measure of the curve's derivative but for a time unit of 1, we don't need to divide anything:

This is the same first week's data from the first graph - including 3, 6, 12hr averaged values -  in derivative form. Nice - looks like some zero crossing we could exploit. But there is some double zero crossing for 3hr filtered values and we still have lag. Not saying something else will fix lag but we should not forget it is there.

I need to pause this log here and will return to add details and graphs of some more promising looking filters, as well show how some filters look good at one point in the year but seem not to work the whole year round.