Open Personal Assistant Robotic Platform

It's been a while since i last posted an update. A combination of having a lot to do at work, starting my own side business and moving to a new city have made the progress a lot slower. But a few things have happened since the last update that i thought i would share for now.

So in the last video I installed the Jetson nano to the robot and now I got it pretty much up and running so the robot can again recognize objects and follow me around. I would say that I got the desired results, the machine vision is much faster on the Jetson nano and is much more suitable for the robot. 

The second thing i did was to replace the speakers, the old ones were really to big, didn't look good and didn't even sound that good. So i got a hold of a pair of smaller speakers with much better sound. I printed new cases for them and attached them lower to the ground so they don't stick out as much as the old ones. 

I got some video material on all of this and hope to get it out as soon as i have the time!

Sooo... i desided to try something a bit new, thought it would be a fun experience to start doing a build diary in video format and in the same time learn more about filming, video editing and so on. So i did this first (really simple) video on some background on the project. So please subscribe to my youtube channel to be sure to catch all the videos. But of course i will continue to post everything i do here and post all the videos here as well.

I have continued working on more features for the voice assistant. Also i have been doing some work on improving the follow mode which i tried combining in this test. Turned out pretty cool! You can see it's much faster than the first video on the follow mode in the projects description. I can still do some improvements on the speed though, but i think i need to improve the hardware to be able to run the predictions for the object detection faster. So i think i will do this either by switching out the Raspberry Pi with a Nvidia Jetson Nano computer or using an accelerator like Coral. I would like to try and experiment a bit with a Jetson Nano, but im leaning more towards the accelerator right now since it would be cheaper and i don't have to transfer everything to the Jetson Nano which could possibly be alot of work.

Been working on the custom app for Mycroft AI to be able to control the robot by voice. I just got some basic commands down so far, but it's pretty cool to see it in action.

To give the robot a little more of a personal feel i also started training a new Wakeword/Name for the robot. So i named it David, seemed like a good name since two of my favorite robots from movies is named David :)

To fully integrate the new voice assistent i've been working on adding speakers to the robot, so used some speakers i had laying around from an old bluetooth speaker project and printed a holder for them. It's not the prettiest. But as you can see on my wiring in the robot, i try to get it functional at first and make it pretty later :) 

The weekend was spent on testing out different voice assistants. The setup i had before was to simply have my own text-to-speech where i just checked for certain keywords. Not the best, but it worked. But i wanted to add something like Alexa so i instantly could get way much more functionality without having to program everything on my own. 

So i started out with Alexa, was not that hard to implement a test version from their examples so i could play podcasts and radio and so on and it worked great. It was fast and accurate. However i really wanted Spotify support so i could play my music and podcasts. But it turns out Amazon have disabled these types of applications if you are not developing a public application. 

So i moved on to Mycroft AI that i found had a Spotify Skill, this was a little bit more tricky to get working at first, it was a easy install but the output audio would not play, after some work i finally got it working, it's slower than Alexa, but i read you can do some optimizations and i do really like that it is Open Source, it was really accurate and it seems easy enough to develop my own Skill to do a custom skill to control the robot. So the direction i will be going in will be with Mycroft AI.

Next step is to create the custom skill and to install a speaker, since im just using a Bluetooth speaker right now which is a bit unnecessary, but i have some speakers from an old build of my own bluetooth speaker that i will probably re-use.

You probably get the idea of how a voice assistant work, but here is a demo video anyway, since some content is always nice :) Something is weird with the sound in the demo, because it jumps into the middle of the first sentence. But i think that is some weird problem with my speaker since i don't get that with headphones connected.

So while it's not the most important part of the project it's always fun to do some designs. But it's also really easy to just end up with a box with wheels. It's easy to build and everything fits great together. But... it is a bit boring, i will see if i can come up with something better than this and of course suggestions are welcome!

Been doing some work in cad to try and get some more parts 3D-printed. Been doing these bottom plates that is attached to the frame for holding components. Had to do a total of 4 plates to be able to fit them in the printer.

In place it will look something like this. Could probably do a better connection between the plates to keep it nice and flat. So perhaps that will change later.

While i'm waiting on some components and more filament to arrive in the mail i will be doing some more CAD work to hopefully get a complete design and overview of the robot.

Put some more work into the interface for the teleoperation part of the robot. From here i can see the video feed with object detection, steering and some controls for making the robot autonomously find and follow objects. It's actually quite nice to make it play music and make it follow you arround the apartment :)