0%

0%

Machine Learning with Microcontrollers Hack Chat

Arduinos and Pis and AI, oh my

Hack Chat

Following Follow event

Become a Hackaday.io member

OR

Just one more thing

To make the experience fit your profile, pick a username and tell us what interests you.

OK, I'm done! Skip

Similar projects worth following

2.2k views

Event Details

Limor "Ladyada" Fried and Phillip Torrone will host the Hack Chat on Wednesday, September 11, 2019 at noon PDT.

Time zones got you down? Here's a handy time converter!

Join Hack Chat

We've gotten to the point where a $35 Raspberry Pi can be a reasonable alternative to a traditional desktop or laptop, and microcontrollers in the Arduino ecosystem are getting powerful enough to handle some remarkably demanding computational jobs. But there's still one area where microcontrollers seem to be lagging a bit: machine learning. Sure, there are purpose-built edge-computing SBCs, but wouldn't it be great to be able to run AI models on versatile and ubiquitous MCUs that you can pick up for a couple of bucks?

We're moving in that direction, and our friends at Adafruit Industries want to stop by the Hack Chat and tell us all about what they're working on. In addition to Ladyada and PT, we'll be joined by Meghna Natraj, Daniel Situnayake, and Pete Warden, all from the Google TensorFlow team. If you've got any interest in edge computing on small form-factor computers, you won't want to miss this chat. Join us, ask your questions about TensorFlow Lite and TensorFlow Lite for Microcontrollers, and see what's possible in machine learning way out on the edge.

Event Logs

Collapse

Hack Chat Transcript, Part 4
Lutetium • 09/11/2019 at 20:20 • 0 comments

Daniel Situnayake12:52 PM
@13r1ckz I recommended a couple of books earlier in the chat:

https://www.manning.com/books/deep-learning-with-python

https://www.amazon.com/Hands-Machine-Learning-Scikit-Learn-TensorFlow/dp/1491962291

as.instrumedglobal12:53 PM
@Daniel Situnayake yeah actually, working on an algorithm to detect various open surgical tools identified so that it can help us out categorizing them after being jumbled in a surgery. any resources I can jump start from, typically a dataset, or do you advise me to build my own and work my way up?

Don Gabriel12:53 PM
Is there an example(s) with YOLO + ML combo implementation with a MCU?

Max-Felix Müller12:53 PM
Thanks @Daniel Situnayake and @Pete Warden

cet12:53 PM
Thank you for that link, and I second 'Deep learning with python' for a beginner.

13r1ckz12:55 PM
@Daniel Situnayake wow great ill get those books asap

Daniel Situnayake12:56 PM
@as.instrumedglobal that's very interesting! I think you'd likely need to find or build a dataset first, then you could use transfer learning to re-train an existing model to recognize the surgical tools

Daniel Situnayake12:57 PM
@Don Gabriel not that I know of!

Daniel Situnayake12:58 PM
@as.instrumedglobal this blog post I just found shows how you can do transfer learning with an object detection model (which can identify multiple instances of multiple classes of object in an image): https://medium.com/practical-deep-learning/a-complete-transfer-learning-toolchain-for-semantic-segmentation-3892d722b604

Don Gabriel12:59 PM
Also, RISC-V seems to more becoming popular. Any ML / DL ports for RISC-V in the future?

Daniel Situnayake12:59 PM
OK, I better head out and finish off writing this book :D Ping me on Twitter if you have any more questions!! https://twitter.com/dansitu

Daniel Situnayake12:59 PM
Thank you everyone!!

Max-Felix Müller12:59 PM
We talked a lot about Corrtex CPUs here. Are they strictly neccessary? What about ATmegas for example? I guess it depends on the size of the model and what the plan is with the processed dat

Don Gabriel1:00 PM
I am comfortable with PICs since I have used them for many years

de∫hipu1:01 PM
if only they had an open source toolchain...

de∫hipu1:02 PM
@Max-Felix Müller atmegas are on the smaller side of the microcontroller families, might be enough to do a simple perceptron, but 2kB of ram doesn't work well with bigger nets

Don Gabriel1:02 PM
open source toolchain for PICs? RISC-V does

de∫hipu1:03 PM
with an open source programmer as well?

Don Gabriel1:03 PM
I just started experimenting with RISC-V and seems promising

Max-Felix Müller1:03 PM
@de∫hipu Yeah, I think you will need a bit more than the atmega has to offer

as.instrumedglobal1:03 PM
@Daniel Situnayake hmmm yeah I figured I'd have to do that since I couldn't find any. well, I think it can be an interesting project to work on, and I have access to plenty surgical sets from work so I know I can get decent info at least for the common tools. Thanks for the help and to all the Tensorflow Dev team, keep up the great work! To the peeps from Adafruit, your hardware makes me drool and everytime I check you Insta feed there's one more board that goes on the bucket list :D

de∫hipu1:04 PM
@Max-Felix Müller you can always add external memory, but it would be sloooooow

gh787311:04 PM
Microchip now has a bunch of ARM processor cores, including the M7 Core, that they program using the same toolchain, and have hung the same old peripherals on them, so not a big transition.

Don Gabriel1:04 PM
cool thx gh78731

Max-Felix Müller1:05 PM
@de∫hipu I think I will have to run some tests... What a great project idea for my holidays :D

de∫hipu1:08 PM
@Max-Felix Müller you know, someone...
Read more »
Hack Chat Transcript, Part 3
Lutetium • 09/11/2019 at 20:18 • 0 comments

Pete Warden12:35 PM
@happyday.mjohnson the general rule is to do as little preprocessing as possible. We do run FFTs on audio to get features, but we take images and raw accelerometer data

Daniel Situnayake12:35 PM
@Sébastien Vézina keep reading :) You won't need to know much of the underlying theory to build some amazing stuff

13r1ckz12:35 PM
How well does the square identifier model detects the backs of a human as a human? (have some problems with that)

Pete Warden12:35 PM
@Dan Maloney Our Dan had a similar idea for recognizing pigeons! :)

Arsenijs12:36 PM
@mtfurlan sorry, I've been having infrastructure troubles, will look into it tonight

Daniel Situnayake12:36 PM
@Pete Warden giving away my million dollar idea!! haha

happyday.mjohnson12:36 PM
@pete - in my way minimum exploration of accel data, there seems to be a benefit on some level of filtering, But I must admit I'm a bit clueless. I guess to me it makes intuitive sense. Since modeling is such a "Dark Art"

Meghna Natraj12:36 PM
@happyday.mjohnson yes colab is awesome -- it's free and convenient -- and you get GPUs too! A lot of folks think you need a lot of data and resources to get started -- but you actually don't. :)

pt12:37 PM
from youtube "PTS

any boards planned with MCUs that include tensor accelerators like the MAIX?'

mtfurlan12:37 PM
@Arsenijs No worries I completely understand. Maybe have it echo your message back to IRC so it's clear if it doesn't work?

Andres Manjarres12:37 PM
@Pete Warden @Daniel Situnayake Incredible book, a lot thanks

happyday.mjohnson12:37 PM
@Meghan i just wish I understood the path from colab -> model on CP.

Pete Warden12:37 PM
@happyday.mjohnson I'm a newbie to accelerometer data too, but our Beijing team are working on the demo, and seem to have made a lot of progress without much preprocessing. The networks themselves are good at learning features

Daniel Situnayake12:37 PM
Thank you @Andres Manjarres !!

happyday.mjohnson12:37 PM
@pete - do you have a timeframe when we can learn from your accel work?

jdaf_1997 joined the room.12:38 PM

happyday.mjohnson12:38 PM
@pete - it would be so cool to get this figured out before my parents pass away :-)

Foxmjay12:38 PM
@Max-Felix Müller not really , do you have a link for it ?

Daniel Situnayake12:38 PM
@happyday.mjohnson we'll have example code available by the end of the month!

Arsenijs12:38 PM
@mtfurlan as soon as I have the "it doesn't forward" detection coded in, it will just re-get the HaD token and re-send it, in short, it will fix itself instead of just signalling "broken" =)

happyday.mjohnson12:39 PM
@Daniel - thank you. Where is the best place to follow for new postings?

Max-Felix Müller12:39 PM
@Foxmjay no, I was just curious

John Loeffler12:39 PM
Limor, PT I am working on the Hackaday Finals could I DM you about your M4 and 240x240 Displays (mechanical33486@icloud.com)

pt12:39 PM
@John Loeffler post here! happy to answer anything!

Daniel Situnayake12:40 PM
@happyday.mjohnson Twitter is useful! I'd follow https://twitter.com/tensorflow for big announcements, and you can follow me (https://twitter.com/dansitu) for more fine-grained stuff

happyday.mjohnson12:40 PM
@Daniel ...sigh...ok...i stay off of THE twitter and THE Facebook...but for TF/Keras....

pt12:40 PM
@happyday.mjohnson we have a bunch of news, guides, projects, etc. that we post up here too : https://blog.adafruit.com/category/machine-learning/

pt12:41 PM
and we're adding more things here too: https://www.adafruit.com/index.php?main_page=category&cPath=1006

Don Gabriel joined the room.12:41 PM

happyday.mjohnson12:41 PM
@pt thank you. Your learning resources are incredible. Thank you for such an amazing company.

Daniel Situnayake12:41 PM
@happyday.mjohnson Twitter is surprisingly amazing for ML stuff, you can follow a lot...
Read more »
Hack Chat Transcript, Part 2
Lutetium • 09/11/2019 at 20:17 • 0 comments

happyday.mjohnson12:21 PM
@Daniel thanks for the github link.

riley.august joined the room.12:21 PM

Daniel Konhauser12:21 PM
are you leveraging transfer learning in these demos?

pt12:21 PM
@Tara the demos we are showing are pi 4, nordic nrf52840, and samd 51

Daniel Situnayake12:21 PM
@Duncan our speech model that can detect "yes" or "no" is ~20kb, our vision model that can detect the presence or absence of a person is ~250kb. I'm not sure

Pete Warden12:21 PM
@Tara we're actually working on a book that has a chapter about setting up TF Lite for new devices. TF Lite Micro should be a lot easier to port than regular TF Lite too - http://shop.oreilly.com/product/0636920254508.do

Matteo Borri12:21 PM
that looks amazing for when you're far out @Evan Juras thank you

Meghna Natraj12:22 PM
@Andres Manjarres Unfortunately no. 1) we train regular tensor flow models (and factor in micro controller constraints if possible- small model size, low latency) 2) convert it to a tensorflow lite model 3) run it on a microcontroller that we support.

Daniel Situnayake12:22 PM
*I'm not sure of the exact latency, but it's pretty low!

pt12:22 PM
@Duncan we'll show the yes no demo on video right now here too ---

Tara12:22 PM
Thanks @pt and @Pete Warden !! I'll definitely check it out.

Pete Warden12:22 PM
@Tara drop an email to petewarden@google.com and we might be able to send you a draft copy

Evan Juras12:22 PM
Thanks @Matteo Borri :D

Andres Manjarres12:22 PM
@Meghna Natraj Okey, thanks

limor12:22 PM
@Dan Maloney yes! we would have to take ~100+ photos of me and pt and train a new model to detect us

John Loeffler12:23 PM
Limor and PT any Cortex M7 Boards in the works?

Dick Brooks12:23 PM
Question: Do you plan to build a Deep Learning demo also?

happyday.mjohnson12:23 PM
I use google colab for deep learning/keras/tensorflow , mu for CP...what is/will be the ideal tool chain for this experience?

jcradford12:23 PM
@Pete Warden I'd be interested in that for a high school class...

limor12:23 PM
we haven't written a guide on that yet - hopefully soon, we're learning a lot every weekend!

@limor Cool, thanks. Thought it would be something like that.

Duncan12:23 PM
@Daniel Situnayake 20kB is impressive! Nice work.

Pete Warden12:23 PM
@jcradford please drop me an email too

somenice12:23 PM
Thanks @Daniel Situnayake

Prof. Fartsparkle12:23 PM
@limor do some models made for desktop/server run with tensorflow lite on the SAMD51? Like the more popular image recognition networks like mobilnet or tiny-yolo?

riley.august12:23 PM
Very cool. Also, Matteo, you misremembered what we generated. It was not a learning capable program, it was a fixed program. We used genetic programming and simulation to develop task specific programs.

I was planning on scraping family photos from Google Photos to train a model for a security system I'm working on

pt12:24 PM
ok! all the demos worked, yay

Daniel Situnayake12:24 PM
@Dick Brooks if you want to see the absolute basics of training a deep learning model and deploying it to device, check out this sample: http://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/experimental/micro/examples/hello_world

pt12:24 PM
one tip for folks who want to play around, check out https://runwayml.com

Tara12:24 PM
@pt Is the demo code you are showing for the pi4, Nordic nrf52840, and samd 51 available?

pt12:24 PM
@tara yep!

Gabriel D'Espindula joined the room.12:24 PM

Everyone, if you miss something, rest assured that there will be a full transcript posted after the chat, as well as links to the demos and livestream.

happyday.mjohnson12:25 PM
@Daniel - thank you for the github on learning tensorflow...besides Adafruit learning (most awesome) where else should we go to explore?

limor12:25 PM
@Evan Juras yay i used your awsome...
Read more »
Hack Chat Transcript, Part 1
Lutetium • 09/11/2019 at 20:15 • 0 comments

OK, big crowd today, let's get started. Welcome to the Hack Chat everyone, thanks for coming along for a tour of what's possible with machine learning and microcontrollers. We've got @pt and @limor from Adafruit, along with @Meghna Natraj, @Daniel Situnayake, and Pete Warden fromt he Google TensorFlow team.

We've also got a livestream for demos -

Also, we hear that a class of sixth-graders is tuned in. Let's make sure we make them feel welcome - we always support STEM!

limor12:00 PM
heyyyy everybooooody

Daniel Situnayake12:01 PM
Hey everyone!! I am so jealous of that sixth-grade class; they have an awesome teacher

Can everyone on the Adafruit and Google side just introduce themselves briefly?

C.R. Myers joined the room.12:01 PM

Hotmspam joined the room.12:01 PM

Pete Warden joined the room.12:01 PM

nu2life joined the room.12:01 PM

Cormac Harkin joined the room.12:01 PM

limor12:02 PM
hi everyone itsa me- ladyada

Pete Warden12:02 PM
Hi! This is Pete from the TensorFlow team at Google

Juan Manuel joined the room.12:02 PM

Alex Colello joined the room.12:02 PM

limor12:02 PM
check out the youtube video for the live demoooos

limor12:02 PM
and more chitchat :)

Meghna Natraj12:03 PM
Hi everyone! I'm Meghna from the Google Tensorflow Lite team. Excited to be a part of this event! :)

omarmung joined the room.12:03 PM

pt12:03 PM
i'm pt, work with limor at adafruit, and founded hackaday 15 years ago which is a coincidence

limor12:04 PM
lets drop some LINKS

Evan Juras12:04 PM
Hey @Pete Warden and google folks ! When is TensorFlow 2.0 officially releasing?? :D

limor12:04 PM
while y'all think of the questions and ya wanna ask

Larry Bank joined the room.12:04 PM

A happy coincidence ;-)

Thomas Marsh joined the room.12:04 PM

Daniel Situnayake12:04 PM
Hey!! I'm Dan, and I work on the TensorFlow Lite team at Google. TF Lite is a set of tools for deploying machine learning inference to devices, from mobile phones all the way down to microcontrollers. Before I was at Google, I worked on an insect farming startup called Tiny Farms :)

Boian Mitov12:04 PM
Hello everyone... sorry for the late apperance...

happyday.mjohnson12:05 PM
Hey Dan, so it's a Bug's Life?

Christopher Bero12:05 PM
Can we get a brief overview of ML on uCs for those of us unfamiliar with the space? What problems is it solving, and on generic microcontrollers or special parts?

SimonAllen12:05 PM
So when does it start?

Andres Manjarres12:05 PM
@pt How neuronal network do yo use in this process?

Dave joined the room.12:05 PM

Daniel Situnayake12:05 PM
OK so, if you're not familiar with ML on MCUs, I can share a few thoughts!

happyday.mjohnson12:06 PM
Can we make our own models (e.g.: using an LSTM for timeseries accelerometer readings)?

Arthur Lorencini Bergamaschi12:06 PM
@pt What problems this Tiny ML can solve?

Tegwyn☠Twmffat12:07 PM
Who invented the term 'confusion matrix'?

Duncan12:07 PM
@Daniel Situnayake What is the size of a typical TensorFlow Lite network (in kB) and what is the inference latency?

Paco joined the room.12:07 PM

3gmarquardt joined the room.12:07 PM

limor12:07 PM
@andres for raspberry pi we're using the mobilenet v2 models that are optimized for mobile device use - its really a tone of work they put into it to optimize them and we wouldn't be able to do a better job!

Pete Warden12:07 PM
Re: TensorFlow 2.0 - I honestly don't know, but we did just publish a release candidate I believe, so it won't be long

pt12:08 PM
@Christopher Bero while @Daniel Situnayake is answering, here are 2 example, one for the pi 4 and one for a samd https://learn.adafruit.com/running-tensorflow-lite-on-the-raspberry-pi-4?view=all & https://learn.adafruit.com/tensorflow-lite-for-microcontrollers-kit-quickstart?view=all

happyday.mjohnson...
Read more »

View all 4 event logs

Discussions