Luxonis DepthAI

Hi everyone,

So the first prototypes of the DepthAI for Raspberry Pi just finished population late last night, and shipped to us this morning!

Some images below:

So we'll be getting these back this week.  It's going to be a busy week of testing, as we're getting these, and also the Myriad X modules one day apart.

On this board our test plan is to exercise all the standard Raspberry Pi features and IO (as it's supposed to act as a standard Raspberry Pi, minus WiFi for this revision), take notes on any errors/etc. for fixing in Altium, and then if all is OK-enough, connect the Myriad X module to it, and see if the whole thing runs together (with the code we have working so far).

Any comments or questions?  Feel free to drop them here or shoot an email to brandon at luxonis dot com.

Best,

The Luxonis Team

Hi everyone!

So we've been continuing to 'churn the butter' on the Myriad X + i.MX 8M module we're making.  Think of this as like the 'pro' version of DepthAI for Raspberry Pi.  It has all the power of the Myriad X and the convenient (Ubuntu-18.04) Linux Support of the low-power i.MX 8M.  (Speaking of low power, check out all it can do, here, at incredibly-low DC power use; all 4 cores maxed w/ Dhrystone is 1.5W - so low!).

Anyways, here's some more Altium-fodder on the layout.  It's starting to look clean, although we still have a long ways to go:

So you can see a lot of the differential routing nearing completion, which is a bit of the slower, more tedious effort:

And the interface to the main board is over 3x 100-pin connectors (DF40C-100DP-0.4V), which are really nice and popular for this sort of thing:

On those connectors... we've discovered, after having selected them, that they're used on everything!  They're even on the Google Coral SoM (here), the gyrfalcon SoM (here), and a bunch of NXP i.MX series SoMs.  One of the NXP dev boards even uses them.

And it's also the same connector we're using on the Myriad X SoM we've made (and which arrives tomorrow!).

Thoughts?

Thanks,

The Luxonis Team

Hey everyone,

So the next step on the path to CommuteGuardian is a SoM for the Myriad X and the i.MX 8M.  This will allow us to drop in a full system running a modern Linux, and with the VPU power of the Myriad X, into our Commute Guardian prototype (and hopefully, final product).  The data path is shown below:

And it will allow you to do the same for your application!  Think of this as like the 'Pro' version of DepthAI for Raspberry Pi.  As it allows it to drop directly into your prototype or product, with the wicked power of the i.MX 8M backing up the visual processing prowess of the Myriad X.

So we just started about a week ago, and here are some layout views to show progress so far:

Thoughts?  Questions/ideas/comments welcome!

Cheers,

The Luxonis Team!

Hi everyone,

Quick update that the Bare PCBs for the DepthAI for Raspberry Pi (see here for details) are done.  We are now waiting on components to arrive to do population.

Pics or it didn't happen:

Cheers,

The Luxonis Team

Hi Hackaday!

So our first prototype of DepthAI for Raspberry Pi was ordered last week (catching up on posts!).  So it's bigger than the final version, and we'll probably clean up some routing during the later shrinking of the layout, but we wanted to get it ordered sooner rather than later to be able to discover what we've messed up sooner than later.  :-)  And also having stuff in-hand always makes you realize what you should have done instead.  ;-)

First, the Altium 3D views:

And for the full specs:

• HDMI Connector
• CM3 Connector
• LAN9513 for 3x USB 2.0 HUB & Ethernet (10/100)
  • 2x USB 
  • 1x USB slave input to accommodate USB boot config from CMIO (might remove later)
  • 1x hard-wired USB 2.0 lane to Myriad X module
  • 1x Ethernet
• Standard Raspberry Pi header
• Display connector (same as RPI 3 B+)
• Rpi Camera connector (same as RPI 3 B+)
  • This is so you could use the board w/out our image sensors if you want… i.e. using it like an ‘embedded NCS2’ version.
• Sd-card slot
• Standard 3.5mm audio jack
• Three onboard cameras:
  • 1 x color; 12 MP: IMX378
  • 2 x grayscale for depth; 1MP each: OV09282
• BW1099 Connector (this is our Myriad X module, the BW1099)
• Raspberry Pi Compute Module connector.

Many here may be unfamiliar with the Raspberry Pi Compute Module.  It allows you to 'integrate the Raspberry Pi into your design'.  So it's a Raspberry Pi in a SO-DIMM form-factor:

So this is what pops into the board above to allow it to be used just like a Raspberry Pi, but harnessing the crazy power of the Myriad X (4 trillion operations per second!) to process video and run neural networks.

Any/all comments/questions/suggestions/etc. are welcome!  And feel free to chat in our Hackaday.io chat room (it's a bit lonely right now!).

Thanks,

The Luxonis Team

Hi everyone,

So far we've been developing everything on Linux 16.04 so far, as that's where all the toolsets properly run, and have been working a bit in the background to get things to compile and run on the Raspberry Pi.  Starting w/ the underlying components and then working up to more complicated things, like our existing depth + feature tracking app (which will hopefully soon also do neural inference with these).

So we got it working today!  It's still super finicky, and randomly refuses to connect, but at least it compiles and when it does run, it runs correctly.  Or as my room-mate Bill Bixler back in college would like to say: 50% of the time it works 100% of the time!

So here it is in all its glory, actually running on the Raspberry Pi:

So we still have work to do to stabilize it.  Funnily enough, it's just the connecting that has an issue, once it successfully connects it will run properly indefinitely.  We think there's something amiss w/ how we're deciding which USB device to talk to, so perhaps we're randomly trying to talk to our mouse like it's a Myriad X, for example.

We'll keep everyone updated!

Cheers!

The Luxonis Team

Hi everyone,

So all the hardware updates so far have been the work of our lead hardware engineer, Brian.  Today we'd like to introduce a new addition to our team, Martin, who's expertise is in firmware (and software) and is taking on depth work on the Myriad X to start with (and then will be tackling all sorts of other Myriad X firmware and software).

So Martin's made some short work on getting the depth performance to be better, finding and solving a couple bugs, and experimenting with some depth processing techniques to see about improving the depth results (which will eventually, if they work well, will be implemented onto the SHAVES of the Myriad X).

Here's the depth prior to calibration:

It turns out we had been using non-calibrated depth as result of a bug in loading the calibration that was found and fixed.

And here is with the bug fix and calibration applied:

And here's with some edge detection on the mono and smoothing like the Intel D435 does:

And these are just initial guesstimates at the coefficients to use for smoothing, so they're probably not ideal for our setup.  

We quickly tried a bit less smoothing and that seems even better:

So we're still tweaking and experimenting, but we figured we'd update as these are starting to look about as good as the results from the D435 from our CommuteGuardian prototype (here).

And for comparison with that video, below are some results from median filtering (first video) and also mono camera-based edge detection and smoothing (second video).  The edge detection isn't synced properly yet, so you can see ghosting as the mono (grayscale) video feed gets out of sync w/ the depth feed when there's motion.

It's worth noting that this is depth on sensors that are pretty close-in, so that's why it quickly progresses to being towards the edge of the perceivable range of the disparity depth.  Here's the board this is running on, which has the global-shutter (disparity depth) cameras spaced at ~34mm.

And on our DepthAI for Raspberry Pi the spacing will be much larger, which will a lot further-out depth sensing (likely around 12-15 meters), with spacing of 110mm between the cameras instead.

More updates to come!

Cheers,

The Luxonis team!

Hi everyone!

So I want to start out this post that on every other project with MacroFab we've been quite pleased with MacroFab - so we do think this is an exception and a one-time sort of thing with them.

So we got bad news yesterday that our Myriad X module, which we ordered back on June 26th, had gotten lost in their system.   Worse, it was lost in such a way that the automation was telling all their management and engineers (and us) that everything was in-progress and on-time to ship on July 15th.  So when we kept asking (as we're super excited to get these back), they (truthfully) kept telling us that all was good - as that's what their system was telling them.  (Such a thing actually belies an impressive amount of automation on the part of MacroFab, actually, that there's such trust in the automation, and that such extensive automation exists.)

So anyways, on July 10th when we checked in once again, the following anomaly popped up this time:

Notice that the start date is set for 8 days after the ship date.  Doh!

This prompted MacroFab to dig in more... it was the first sign that something was amiss.  And in digging in, they discovered that there was a corner-case bug with their recent upgrade of their automation software, and our order and 2 orders had fallen into that corner.  :-)

So what does this mean for us?  We thought the 3-week countdown to having the Myriad X module started on June 26th.  Nope.  Because of this bug, it actually starts today!  So we're looking at August 8th for our first modules to come in.

A bummer, for sure, but this happens in the world of doing things, making things.  And, the good news, is the little breakout board for the module did arrive on time (we ordered them so that they'd arrive same day, which was yesterday):

So we tested as far as we could on these, but real testing will happen when the Myriad X module that this is designed for gets in around August 8th.  Anyways, they look good so far (they're pretty simple, mainly just a breakout board):

1. No solder joints looked problematic
2. Application of 5V was nominal, and the 3v3 regulator turned on and generated 3.3V
3. The tiny FB chokes on the differential pairs all showed continuity from one side to the other, but also showed isolation from + to - lines of the same diff pair, indicating proper soldering
4. 5V is present at the input of the 1099 connector and also at both camera connectors, as expected. 

We also have some stereo depth firmware updates we're going to share soon!  Some promising results!

Best,

The Luxonis Team

Hi everyone!

So we're now in progress laying out our first revision of DepthAI for Raspberry Pi.  This is the one that has a slot for a Raspberry Pi Compute Module (3B+).

First, pictures of the layout as it's progressing now, and then the brief specs:

So in short our Myriad X module (the BW1099) pops on the front of the board, the Raspberry Pi Compute Module pops on the back, and the two global-shutter cameras are on front for depth, along with the 12MP color sensor which supports 4K video output (H.264 encoded) to the Raspberry Pi as well.

And for the full specs:

• HDMI Connector
• CM3 Connector
• LAN9513 for 3x USB 2.0 HUB & Ethernet (10/100)
  • 2x USB 
  • 1x USB slave input to accommodate USB boot config from CMIO (might remove later)
  • 1x hard-wired USB 2.0 lane to Myriad X module
  • 1x Ethernet
• Standard Raspberry Pi header
• Display connector (same as RPI 3 B+)
• Rpi Camera connector (same as RPI 3 B+)
  • This is so you could use the board w/out our image sensors if they want… i.e. this is like ‘embedded NCS2’ version.
• Sd-card slot
• Standard 3.5mm audio jack
• Three onboard cameras:
  • 1 x color; 12 MP: IMX378
  • 2 x grayscale for depth; 1MP each: OV09282
• BW1099 Connector (this is our Myriad X module, the BW1099)

Thoughts?  

We're hoping to sell the whole package in late September for $199 and will be launching a CrowdSupply page soon (to help get the necessary volume order to drive the cost down).

Cheers,

The Luxonis Team

luxonis.com

Hi everyone!

So we ordered the Modular design for the Movidius Myriad X.  As below it breaks out MIPI, and will be the core of our products.

BUT!  It also allows you to integrate this into your prototypes and/or products.  Without the hassle of dealing w/ a super-fine-pitch tiny ~400 pin BGA!

We do have a question:

Should we make one of these with a built-in host controller, say an i.MX 8M?

And oh by the way here's the breakout board, which allows connection to the IMX378 modular sensor and also 2x OV09282 (for stereo disparity depth):

Cheers,

The Luxonis Team