Background:

Several folks have asked about accuracy and precision.  That’s a big topic, so I decided to start on only a single aspect – spin analysis – and just provide a couple of examples.  I haven’t worked out a formal, full, math-based error analysis yet, and am not certain I can. Instead, this log entry will just focus on providing anecdotal descriptions of the errors in what I believe are ‘typical’ scenarios.  I’m not even sure that a math-only analysis is a reasonable goal – there are an awful lot of case-specific potential places for errors to creep in, and it may not be possible to enumerate them all. I’d also probably have to go back to my grad-school books on error analysis and remember all of that. Ugh.  Of course, it would be great if this was all open source someday and some smarter folks could dig into the math and tear it all apart!

Another important background point is that a real “ground-truth” against which to compare the system is hard to find.  You’d literally have to go to a real golf course, setup the LM(would have to be a very cloudy day to prevent the camera from being flooded with IR), and then measure the real-world distance, angle, etc. on the course and compare it to the results of the physics model in use by the LM/Simulator.  I don’t have those facilities yet. :/  As an alternative, the analysis here will be focused on comparing a best-estimate of the ‘’correct’ ball spin using visual techniques against the ball spin determined by the LM.  We’ll also make a comparison of how those difference(s) manifest themselves in pseudo-real-world coordinates in a simulator (here, GSPro).  The carry distance will also, of course, be determined by the determined launch angles and velocity, but we’re focused on spin here.  I’ll examine those measurements and the related accuracy in a later log entry.  So, let’s start…

Analysis:

Spin speed accuracy is primarily governed by the accuracy of the angular delta measurement between the ball exposures and the accuracy of the determined time delta between the exposures.  The former is currently most important.  Precision is limited by the fact the system only deals with degrees of spin as an integer, so 1 degree is the upper limit on precision.

Let’s start with the following first example, where the strobed ball images are shown below.  I chose these examples because they have a ball with strong registration markings that ended up facing the camera.  That makes for easier visual analysis of the spin.  These were also images where the Z-rotation was fairly substantive (these shots were using a 7-iron combined with crappy golf skills), and the X and Y rotations were near-zero, making it easier to focus on the back-spin.

The system chose two ball exposures from the above image to use for spin analysis.  Those are the first two images, below, and labeled ‘1’ and ‘2’ above.

From these images, the system determined XYZ Angles (in degrees) of -5, 0, and 37.  The -5 is side-spin, and the 37 is backspin. The 0 y-axis is “rifle-bore” spin, which is usually near zero.  The system determined the time between the two images to be 3500uS (3.5 ms).  The dark penumbra around the balls is a result of de-rotating the balls to a common frame of reference.  That de-rotation (reversed later in the processing) creates some dark edges because some of the back of the ball (to the camera) has no information when rotated around to the front in 3D.  The small dots (especially on the third image) are a result of the 3D/2D projection/de-projection operations.

In order to provide a visual sanity-check for the player, the system reproduces the first ball image rotated by the determined angular deltas, and shows that image as the third image above.  In a perfect world, the result of rotating the first image by the delta angles should produce an image that looks exactly like the second image.  In order to do a more precise analysis, an on-screen protractor was used to estimate the angles of the second ball relative to the first one.  This is just a visual approximation, as we’re not measuring from the true axis of rotation.  However, given the primarily-z-axis spin, it should be pretty close.

The first angle is 64.31 and the second is 27.82.  Thus, the approximate Z-angle of rotation (ignoring the two other axes) is: 64.31-27.82 = 36.49 degrees.  You get about the same number if you measure from the center dot in the middle of the ball image.  That makes the error about 0.5 degrees.  

The X (side spin) rotation error is larger, especially relative to the very small actual side spin.  It is a little hard to measure this angular error very accurately, but you can tell from the third (projected) picture that the “X” is a little closer to the center-dot than the actual (2nd) picture.  But, just as an estimate, let’s say 2mm as the radius of each dimple, and with a 134mm circumference (all from the intergoogle).  The error looks like about a 1mm difference compared to the 2nd image.  That’s 1/134 * 360, about 2.7 degrees difference in the X axis.

In conclusion, the back-spin error is 0.5/37.0 =  1.4% too slow.   Side-spin, 2.5/5 = a 100% over-estimate.  Pretty significant in this instance! :/  That said, if a more serious, say, slice is something like 30 degrees, and if the side-spin error stays at two or three degrees, that’s maybe not too bad.

So how does this affect the spin RPM calculations?  Well, 37 degrees in 3500us -> 1762 RPM.  The best-guess for the actual RPM was 1738 RPM.  A 24 RPM difference.

Calculated side spin was 428 RPM, but (potentially) should have been 214 RPM.

Now, let’s look at the potential error in the strobe timing.  I have test-points on the system PCB to which I attached my oscilloscope.  Some of the earlier-in-time pulses for both examples here are shown in blue, below:

The question is, when the system determines that a certain amount of time has passed between two ball images, how accurate is that timing?  Turns out, pretty darn accurate.  For example, looking at what should be a 1 ms pulse interval at the largest magnification at which my scope will show two pulses (to make the example a little more convincing), there appears to be NO apparent error at all:

In the above picture, each small horizontal tick is 40uS.  I can’t visually tell if there’s any error, but since I can’t see below ~4uS, let’s use 4uS as the possible error, just to be conservative.  With an error that small, I just ignored the effect (which is pretty small compared to the angular errors).

So— what does this mean in terms of your (simulated) game score? To determine the ‘real-world’ results of the above differences, I injected two shots into the GSPro simulator that differed only by the spin values.  (Thanks to the #GSPro folks for their support!).  The first two shots were, in order, the “correct” and “as-measured” data discussed above, except at 50mph.  And the next pair was the same except for 70mph.

At 50 mph, the resulting carry difference is less than 3.6 inches (0.1 yards) of carry, and the total length difference is ~3.6 inches.  Visually, I’d say the LM-determined ball is about a foot to the right of the “correct” ball.  That’s due to the considerable 200RPM difference in side spins.  (BTW, I screen-printed the second image about a quarter-second too soon, but wanted to catch the view before it returned to the original spot.  You can see the second ball from an earlier run just ahead of the ball-in-flight.)

At 70 mph, the carry difference is again only a few inches, but the side distance is more like 3 feet.  More work to be done on that!

Conclusions:

All in all, considering the use of the less expensive cameras and such, I think the DIY LM is making good progress in terms of spin accuracy.  I believe that the spin accuracy can be increased using several techniques.  The main tradeoff is between longer processing time for spin (I’d like it to be no more than a couple seconds).  More time allows for a larger increase in the angular resolution of the algorithm. There should be some other ways to improve side-spin accuracy without costing more processing time as well. 

In a later post, we’ll look at VLA, HLA, and velocity.