Luxonis DepthAI

Hi everyone,

We're making a modular design - and it's almost done.  We'll probably order our first round next week.

Why?

• Allows integration of the Myriad X directly into your design.
• Doesn't require your host have PCIE (USB instead)
• Allows plug/play Raspberry Pi + Myriad X at 30FPS instead of 8FPS of NCS2, including stereo depth
• Allows _you_ to make whatever board you want, without having to muck with dealing with the complexity and cost of making a high-density BGA board.

How?

So we'll make two carrier boards for this to start, both of which will be open source:

1. A carrier board for this module with an output USB interface
2. A carrier board for this module and also for the Raspberry Pi Compute Module

What?

• Myriad X with all supporting hardware.  All the carrier board needs to do is provide power and connectivity.
• All connectivity through single 100-pin connector (DF40C-100DP-0.4V(51))
• 1 x 4-lane MIPI for high-res image sensor (e.g. 12MP)
• 2 x 2-lane MIPI for standard-res image sensors (e.g. 1MP)
• USB interface
• 40mm x 30mm (~ 1.57" x 1.18")

Hey everyone!

So the first boards we designed were an internal development board of sorts.  Initial specifications:

• Breaks out connections for 5 cameras (including 2 Raspberry Pi connectors)
• Has SD-Card, emmc, SPI-flash and a bunch of other things we probably don't need, but we figured we'd break out in case we want to develop for them on a future product/functionality (without having to order new boards)
• Has connectors for our dual-camera module (for disparity-depth)

The renderings of these are in the previous post... but now we have working boards, so for some pictures!

So the first thing we did was run hello world via JTAG... and it worked!

I studied German at one point, so now I like to do 'Hello World' in German instead. ^

Then, we went ahead (gasp!) and tried out to see if the image sensors all worked properly:

And they totally did.  Above you can see (uncalibrated) stereo disparity depth and also feature tracking.

So the next thing we're working on is making a module for the Myriad X... as we (re)discovered how pricey boards of this caliber are when you do low quantities.

So the module will allow us to do a variety of different form-factors, hosts (e.g. Raspberry Pi Compute Module, Toradex NXP i.MX8 Computer on Module, etc.) that the Myriad X can pop into.

Cheers!

 - The Luxonis Team

Hey everyone!

So we did it.  We ORDERED our first boards.  Man, low-volume pricing on tight-tolerance boards is so painful.

Fortunately we think our kids are crafty, so they won't need a college fund, right?

We're actually going with a modular approach from what we learned from this, so that we can make various hardware incarnations, which will all leverage the same Myriad X module.

That way we don't have to give up our kids' college funds every time we decide to change features/form-factor/etc.

So that's what we're currently working on.  The current plan for the module are 2 x MIPI 4-lane which are usable as 4 x MIPI 2-lane and of course USB and all that jazz.

The idea is that this allows up to 4x 2-lane cameras, 1x 4-lane camera and 2x 2-lane cameras, or 2x 4-lane cameras.

Which we think should cover any/all permutations of the boards that this module will go on.  If there's interest, we'll sell the module alone as well.

Thoughts?

And as a bonus, here's one of the camera modules, which is a greyscale stereo pair:

Hey guys,

So we got hardware depth and video tracking working.  It's not calibrated depth yet (so that's why it doesn't look so great - it's using a unity 3x3 homography matrix).  But it's working!  (Caveat on that, it's still buggy and crashes on startup 9/10 times, but the 1/10 is so satisfying!)

But to re-iterate, all the calculation shown in the video is being done on the Myriad X (depth calculation and feature tracking).  The host is doing nothing (other than just displaying the data that the Myriad X is streaming, which is optional).  

The nice part is the Myriad X doesn't even get warm doing this.  And that's with zero heatsink.  Just the chip exposed to ambient air.

And for more info as to our end goals, check out:
aipi.io - a Raspberry Pi depth vision + AI carrier board, which is itself a product we thought would be useful to the world, and is an internal stepping stone to:
commuteguardian.com - the AI bike light to save lives

So we've been chasing down parts for our first board run (of 10 prototype/dev.) units, and in the meantime we now have a logo, thanks for our fancy graphic designer:

Hey Embedded AI Enthusiasts,

We're excited to share that we just finished component placement and initial routing of our first version of the board.  This one is for initial development, debugging, etc. - and actually doesn't even have a Raspberry Pi slot yet.  It'll primarily be programmed by JTAG and prodded and debugged.

Anyways, here's a 3D view of it:

It is, however, the same size as a Raspberry Pi 3.  For the later versions, we'll remove a TON of extra stuff that's on this one - so there'll be more room for the Raspberry Pi CM3B+ module.

Thoughts?

Best,

AiPi Team!

To help take this product further, please could people fill in the short survey - should take no longer than 60 seconds:

https://goo.gl/forms/8QloGu9o5GxKYtgG3

Hey Machine Learners,

Exciting news!  We ordered our first set of Myriad X parts today, for our initial round of internal development/verification boards.

Only 10 units so far, as their just for internal use.

And if all goes well, we’ll be able to order the boards for manufacture/population in a couple weeks.

Best,

Brandon

This demo is with an Intel RealSense D435 + Raspberry Pi 3B + NCS1.

It's doing MobileNet-SSD Object Detection and depth-data projection to give XYZ position of every pixel.  And we're printing the XYZ of the middle-pixel of bounding boxes in the bounding box label (hence with the chair, it changes when I walk behind it, because the center-pixel is actually the wall behind the chair in its initial orientation).  All other pixels' XYZ are available per frame, so you can use the ones most pertinent, average over an area, etc.  And in the case of the Commute Guardian, the XYZ location of the edge of the vehicle is used for impact prediction.

We're working to make a board which leverages the Myriad X to do the depth calculation (and de-warp/etc.) directly while also doing the neural network side (the object detection).  This should take the whole system from ~3FPS to ~30FPS, while reducing cost.

And if you want to give input on what the design should be, or other designs you'd want instead (or generally just to find out more options for embedded machine learning), head over to:
https://discuss.luxonis.com/

And if you want to know the background/why of us making this stuff, the end goal is to save bike commuters' lives:
https://commuteguardian.com/

We're simply releasing our work, before the final bike product is out because we realized that the board itself (particularly with the Raspberry Pi as the brain) would be super useful for a bunch of engineers across a variety of project types.

Hey guys,

So in our efforts we found it actually pretty hard to know all the options out there for neural processors/etc.  So we figured having a forum to discuss these would be great.

And even greater, a place to drop your needs/etc. which aren't met by current solutions (which is super common as embedded machine learning is just really starting).

So without further ado, here is our forum for that:

https://discuss.luxonis.com/d/4-neural-processors-and-hosts-we-ve-found-so-far

And I figured I'd start everyone off with the summary of parts of interest we've found so far, and links to some other useful resources which cover even more (WAY too many) parts.

Cheers,

Brandon