DIY Golf Launch Monitor

This is the first (and hopefully last ever) ASMR video from the DIY Launch Monitor Project:

Back story is that the sound of pulling 3D printed parts off the print bed, as well as pulling off the printed supports, is surprisingly cool.  And apparently kinda popular on youTube.  See, e.g., here and here.

So, why not make a goofy video of our own to exhibit those sounds? Turns out, there are lots of reasons.  Like because, we're software and hardware developers, not videographers.  And also the fact that recording the sounds with any fidelity is harder than it might seem.  Especially using rudimentary recording equipment.

Fortunately, an ancient Blue USB microphone was located, and that helped a lot. :)

_________________________

On a serious note, we've been doing some work to try to ensure that folks who eventually build their own version of the DIY LM will be successful doing so.  One of the more difficult parts of the build so far has  been the varied, inconsistent success we've had with printing the enclosure parts.  It's harder than it looks to get consistent results, and we want to ensure that it goes as easily as possible.  

To that end, we've been printing on different printers and with different types and brands of plastic filament to see what works best.  That's where the ASMR video came from.  Hopefully some re-designs that are in progress will make this easier for folks.  

Some other areas of concern are the calibration process (which could use some more support tools) and just the process of getting a couple of Raspberry Pi's set up as a development and runtime environment for the rest of the software.  Hacking the Pi Global Shutter camera to support external shutter triggering isn't easy either, especially if you don't have the steady hands of a surgeon. :/  An upcoming log post will go into some of the things we're learning about getting an open source project ready to release into the wild.

Anyway, we're continuing to knock off some of the rough edges of the system so that others will hopefully one day be able to replicate it on their own.  On that note, we'll soo be looking for a couple of volunteers to try out an alpha version of the system design to see how hard it will be to replicate.  More on that later...

The two schematics below show the initial version of the connector board (on the top/left) and the new version (on the bottom/right).  They both do the same basic thing -- opto-isolation and high voltage switching for the strobe board and the camera trigger signals.

So, why the change?

Well, now that we’re starting to think about what needs to be done before releasing the project into the wild, there are a lot of considerations.  They include things like proper attribution of open-source, IP management, ease of building and testing, and … safety!

One issue that came up in the safety review was the fact that the original connection board design would allow the LED strobe board to be full ‘on’ continually if the power from the Raspberry Pi failed, or if the Pi otherwise became disconnected from the board.  The problem was that if a 100 watt, densely-packed LED array is on for any more than a few seconds, it starts to get really hot.  Like, hot enough to potentially even melt its #d printed plastic mounting hardware!

This was a good (if painful) lesson in identifying and considering failure modes earlier in the design process, where things are easier to accommodate and/or protect against.  The circuit in the connector board is pretty simple, requiring an non-inverting opto-isolator among other things.  One easy way to do so (other than switching from a pull-up to a pull-down transistor on certain opto-isolation ICs), was just to add an inverter to the input signal.  That’s the red-colored path in the top/left schematic.  

But, we hadn’t thought of the consequences of inverting the input, as opposed to adding the inverter after the isolator as in the purple path in the bottom/right schematic.  The right-most circuit has no other material downsides, but it also has the benefit that the strobe output to the 12 V MOSFET switch will go to low in the event of most failure modes, like a loss of power from the Sys1_Conn from the first Raspberry Pi or anything else that would cause power to the 7404 hex inverter chip to fail.  The opposite is true of the original circuit, where a failure causes the input to the isolator to go low, which causes the output to go high and switch on the strobe.  So, the new circuit inverts the output, not the input.  This should vastly decrease the chances of the LED array ever being on for more than a few microseconds at a time.

Easy fix, but required another run of PCB boards. :/  More great lessons from this project…

Apparently the best way to learn what designs DO NOT work reliably is just to make every 3D printing mistake in the book as we go. Which we are doing quite well.  For example, the image below shows the result of printing the enclosure design with a different filament type (PLA in this case) to see how it would work.  The result indicates insufficient print-bed adhesion as well as (possibly) over-aggressive cooling tuning resulting in pull-away and part warping.  New design will have a more filled-in base layer.

Things always look wonderful in the CAD software, but then you hit print and ... <doh> ...

The work on the enclosure is starting to pay off.  After some investments in equipment, CAD software and the (sometimes painful) learning curves, we've got the DIY LM in its first, prototype enclosure.  A less hacked-together version is being built presently.  

Anyway, here's a few pictures of the first stab at it...

Finally returned from a work hiatus and got back into our Uneekor-based test environment.  That environment is set up to make comparisons between the DIY LM and a Uneekor LM.  This post details an example comparison using some new logging facilities that were designed for just this purpose.  

For the impatient, the short summary for the first sample is:

Now onto the details.  First, it’s been difficult to get the two strobe based LM systems not to interfere with each other, but with the help of some more image processing, we can frequently get some decent comparisons now.  More details on the comparison methodology later.  We’re lucky to have a Uneekor as a measuring stick for the DIY LM, but it does come with its challenges.  Primarily, it’s a measuring stick that–when used–often changes the length of the thing being measured! :/  That said, it does work well enough with the DIY LM to use it to confirm the basic workings of the LM and to hopefully provide a wide worst-case accuracy analysis.

For this first example, here’s a 7-iron shot (summary above), starting with teed-up ball and its position identification:

This first shot is not strobed, so doesn’t have the same problems as imaging the ball in flight.  

The in-flight images are shown below, with the identifications of the ball positions in the second image.

You can see that the circle identification for Ball 4 is definitely off, and even Ball 3 isn’t perfect.  This seems to happen most of the time when the Uneekor’s strobes are blinking at the same time as the DIY LM.  Fortunately, when the DIY LM is running in its usual way (by itself), the circle identification is usually spot-on.  For example, here’s a typical identification when the Uneekor isn’t interfering:

Moving on to spin calculation, in our present example the 3D ball rotation rates are calculated using the following images of the balls.  The first two images are the images used to determine an angular displacement between the balls from one point in time to another a couple of milliseconds later.  The third image is a visual sanity-check.  It shows the result of the first image rotated in 3D using the estimated angles that are used to calculate the DIY LM’s spin values.  As shown, it looks fairly close, but maybe just a little under-rotated in terms of back spin.

The spin calculation is even better, of course, when the circle identification is closer to perfect when the Uneekor isn’t in use.  That completes the review of the artifacts the system stores for each shot.

Ultimately, the question is what the real-world (well, real-world simulator) results are.  In other words, how do the differences listed below for the DIY LM data actually affect where the (simulated) ball ends up? 

To recap, those numeric element-by-element errors are on the last line of the spreadsheet below:  

It is fairly easy to determine how the above differences affect where the simulator projects the ball will land on the virtual golf course.  To do so, we injected the original Uneekor-based shot into an E6 simulator, and then followed that by injecting the same shot, but modified by using the DIY LM values instead of the Uneekor values.  Just one value (HLA, speed, etc.) is changed at a time except for the last injected ball.  The JSON-driven data is:

"kInterShotInjectionPauseSeconds": 20,
            "test_shots_to_inject": {
                "1": {            ⇐=================== This is the Uneekor result with all of its data
                    "Speed": 71.9,
                    "BackSpin": 4153,
                    "SideSpin": -871,
                    "HLA": -2.0,
                    "VLA": 21.1
                },
                "2": {            ⇐=================== This one is mostly the Uneekor data, but with the ball speed from the DIY LM
                    "Speed": 73.4,
                    "BackSpin": 4153,
                    "SideSpin": -871,
                    "HLA": -2.0,
                    "VLA": 21.1
                },
                "3": {
                    "Speed": 71.9,
                    "BackSpin": 3968,
                    "SideSpin": -871,
                    "HLA": -2.0,
                    "VLA": 21.1
                },
                "4": {
                    "Speed": 71.9,
                    "BackSpin": 4153,
                    "SideSpin": -871,
                    "HLA": -2.0,
                    "VLA": 20.9
                },
                "5": {
                    "Speed": 71.9,
                    "BackSpin": 4153,
                    "SideSpin": -871,
                    "HLA": -0.3,
                    "VLA": 21.1
                },
                "6": {
                    "Speed": 71.9,
                    "BackSpin": 4153,
                    "SideSpin": -873,
                    "HLA": -2.0,
                    "VLA": 21.1
                },
                "7": {         ⇐=================== This last one is the actual DIY LM result with ALL of its data
                    "Speed": 73.4,
                    "BackSpin": 3968,
                    "SideSpin": -873,
                    "HLA": -0.3,
                    "VLA": 20.9
                }
            }

A cool little video showing the shot injection in real time is here.

For example, for the different DIY LM ball speed (which I assumed would create one of the largest simulated differences), we have a total ball distance of 88 yards (Uneekor) versus 91.5 yards (DIY LM).  The video shows the other differences.  Ultimately, the DIY LM put the ball about 4 yards from where the Uneekor did.  That’s still a farther distance than is desirable.  

Here’s another example shot that was a little closer:

The difference between the two LMs is shown below, again in E6.  In this case, some of the differences sort of canceled the others, resulting in a pretty similar lie:

However, on the bright side, the DIY LM has much better ball/circle identification performance when it’s operating by itself.  For that reason, I’d consider the behavior in the (very artificial) two-LM situation in which this comparison was made to be a worse-case scenario.  I rarely see the ball outlines so poorly approximated when there’s only one strobe light working.  Also, a major contributor to the errors in this comparison appears to be due to under-estimating the radii of the balls.  The lighting is harder to calibrate when the Uneekor overhead strobe is flashing, so the balls are a little under-exposed.  This leads to the ball edge being less sharp, and ultimately the LM is less accurate particularly for the HLA (horizontal launch angle).  And HLA was a major contributor to the differences between the two LM in the experiments so far.

Our first U.S. patent application was published by the USPTO today.  

Here's the fastest recorded shot processed by the DIY LM so far.  136mph (61 m/s).  Still hoping to get something a lot closer to 100m/s.  I think it's the fastest back-spin so far as well.  Thanks to Dave L. for the help!

Ok, so that's not very fast, but when testing something today, I took a club and just tried to really wack the ball.  And I noticed that for the first time that I can remember, the system recorded a shot (barely!) over 100 mph (45 m/s).  Sadly, I can't hit the ball a whole lot faster than that, but it appeared to be recorded correctly.  There was also plenty of additional room on the screen for the strobe pulses to be pushed further apart and further to the right of the screen.  That means that higher speeds should not be a problem.

Unfortunately, I had the "on" strobe set to an unusually long time (>30uS), so that may have contributed to the blurring that is visible in the ball-spin analysis images.  Which in turn resulted in an inaccurate spin measurement as seen in the mis-match between the second ball image and the as-calculated-spin ball (third ball image).  Still, kind of satisfying.

The LM is currently down to about 5 seconds delay between hitting the ball and seeing the shot in the golf simulator.  This includes full processing, including 3D spin down to 1 degree in each axis.  See video here.

This speed is not as fast as it needs to be, but is getting closer.  The recent speed up is due primarily to utilizing more processing cores, turning down debugging, and moving up to a Pi 5 from a Pi 4.  Additional speed increases should be available by decreasing the network traffic that is currently going on after a shot.  I still think the LM should be able to get to less than two seconds.

Today for the first time, we were able to make a few comparisons of the DIY Launch Monitor's outputs to a commercial LM.  It’s also the first time to test in a more professional simulator environment.  The bed-sheet-over-PVC-piping-in-the-basement is still an option, but was pretty limiting.  And funny.  Instead, we now have access to a large simulator bay with the Uneekor overhead and running TGC 2019 (*).

The results?  Well, pretty decent for a first try.  See the example video here.  Lots of work to do, obviously, and WTH is going on with the side spin (at least in this one test case)?  We also need to stop truncating the output to integers in order to have a more meaningful comparison.  Hopefully these aren’t difficult fixes.  I’m not too unhappy right now in any case, given that the material costs of the DIY LM are around 1/20th of the cost of a Uneekor.  However, I’m certain the DIY LM can do a lot better than it's current performance. 

BUT, the main problem with the comparison environment was something I should’ve foreseen, but didn’t.  The Uneekor also appears to use an infrared strobe.  I never knew how it worked until I started this comparison, and it appears to run a high-speed IR strobe at all times once the ball is teed up.  This is wreaking havoc on the DIY LM!  See the bright-orange mess on the LM’s user interface in the video.  Current ideas include trying to notch-filter whatever wavelength the Uneekor is centered on, as well as some type of filter to remove the linear ghost images of the golf shaft as it moves.  I'm curious how other folks do comparisons of this sort (to the extent anyone has).

___________

(*) I’d really like to complete an interface to TGC 2019.  The DIY LM already works with GSPro and E6.  But no one at #2k Games (pr@2k.com) answers our emails, and I can’t locate a business contact there.  Anyone who knows someone, please DM me!