Finally returned from a work hiatus and got back into our Uneekor-based test environment.  That environment is set up to make comparisons between the DIY LM and a Uneekor LM.  This post details an example comparison using some new logging facilities that were designed for just this purpose.  

For the impatient, the short summary for the first sample is:

Now onto the details.  First, it’s been difficult to get the two strobe based LM systems not to interfere with each other, but with the help of some more image processing, we can frequently get some decent comparisons now.  More details on the comparison methodology later.  We’re lucky to have a Uneekor as a measuring stick for the DIY LM, but it does come with its challenges.  Primarily, it’s a measuring stick that–when used–often changes the length of the thing being measured! :/  That said, it does work well enough with the DIY LM to use it to confirm the basic workings of the LM and to hopefully provide a wide worst-case accuracy analysis.

For this first example, here’s a 7-iron shot (summary above), starting with teed-up ball and its position identification:

This first shot is not strobed, so doesn’t have the same problems as imaging the ball in flight.  

The in-flight images are shown below, with the identifications of the ball positions in the second image.

You can see that the circle identification for Ball 4 is definitely off, and even Ball 3 isn’t perfect.  This seems to happen most of the time when the Uneekor’s strobes are blinking at the same time as the DIY LM.  Fortunately, when the DIY LM is running in its usual way (by itself), the circle identification is usually spot-on.  For example, here’s a typical identification when the Uneekor isn’t interfering:

Moving on to spin calculation, in our present example the 3D ball rotation rates are calculated using the following images of the balls.  The first two images are the images used to determine an angular displacement between the balls from one point in time to another a couple of milliseconds later.  The third image is a visual sanity-check.  It shows the result of the first image rotated in 3D using the estimated angles that are used to calculate the DIY LM’s spin values.  As shown, it looks fairly close, but maybe just a little under-rotated in terms of back spin.

The spin calculation is even better, of course, when the circle identification is closer to perfect when the Uneekor isn’t in use.  That completes the review of the artifacts the system stores for each shot.

Ultimately, the question is what the real-world (well, real-world simulator) results are.  In other words, how do the differences listed below for the DIY LM data actually affect where the (simulated) ball ends up? 

To recap, those numeric element-by-element errors are on the last line of the spreadsheet below:  

It is fairly easy to determine how the above differences affect where the simulator projects the ball will land on the virtual golf course.  To do so, we injected the original Uneekor-based shot into an E6 simulator, and then followed that by injecting the same shot, but modified by using the DIY LM values instead of the Uneekor values.  Just one value (HLA, speed, etc.) is changed at a time except for the last injected ball.  The JSON-driven data is:

"kInterShotInjectionPauseSeconds": 20,
            "test_shots_to_inject": {
                "1": {            ⇐=================== This is the Uneekor result with all of its data
                    "Speed": 71.9,
                    "BackSpin": 4153,
                    "SideSpin": -871,
                    "HLA": -2.0,
                    "VLA": 21.1
                },
                "2": {            ⇐=================== This one is mostly the Uneekor data, but with the ball speed from the DIY LM
                    "Speed": 73.4,
                    "BackSpin": 4153,
                    "SideSpin": -871,
                    "HLA": -2.0,
                    "VLA": 21.1
                },
                "3": {
                    "Speed": 71.9,
                    "BackSpin": 3968,
                    "SideSpin": -871,
                    "HLA": -2.0,
                    "VLA": 21.1
                },
                "4": {
                    "Speed": 71.9,
                    "BackSpin": 4153,
                    "SideSpin": -871,
                    "HLA": -2.0,
                    "VLA": 20.9
                },
                "5": {
                    "Speed": 71.9,
                    "BackSpin": 4153,
                    "SideSpin": -871,
                    "HLA": -0.3,
                    "VLA": 21.1
                },
                "6": {
                    "Speed": 71.9,
                    "BackSpin": 4153,
                    "SideSpin": -873,
                    "HLA": -2.0,
                    "VLA": 21.1
                },
                "7": {         ⇐=================== This last one is the actual DIY LM result with ALL of its data
                    "Speed": 73.4,
                    "BackSpin": 3968,
                    "SideSpin": -873,
                    "HLA": -0.3,
                    "VLA": 20.9
                }
            }

A cool little video showing the shot injection in real time is here.

For example, for the different DIY LM ball speed (which I assumed would create one of the largest simulated differences), we have a total ball distance of 88 yards (Uneekor) versus 91.5 yards (DIY LM).  The video shows the other differences.  Ultimately, the DIY LM put the ball about 4 yards from where the Uneekor did.  That’s still a farther distance than is desirable.  

Here’s another example shot that was a little closer:

The difference between the two LMs is shown below, again in E6.  In this case, some of the differences sort of canceled the others, resulting in a pretty similar lie:

However, on the bright side, the DIY LM has much better ball/circle identification performance when it’s operating by itself.  For that reason, I’d consider the behavior in the (very artificial) two-LM situation in which this comparison was made to be a worse-case scenario.  I rarely see the ball outlines so poorly approximated when there’s only one strobe light working.  Also, a major contributor to the errors in this comparison appears to be due to under-estimating the radii of the balls.  The lighting is harder to calibrate when the Uneekor overhead strobe is flashing, so the balls are a little under-exposed.  This leads to the ball edge being less sharp, and ultimately the LM is less accurate particularly for the HLA (horizontal launch angle).  And HLA was a major contributor to the differences between the two LM in the experiments so far.