Smart Speaker from Scratch

Last time I showed a web app of sending data over sound to setup Wi-Fi. A further step is to integrate it with a web terminal - ttyd and JupyterLab. These web apps are included in VOICEN OS which is based on Armbian OS. Have a look at the video to find out how it works.

Steps to get started with VOICEN OS:

1. Press and hold the touch key of the device for 4+ seconds to turn the device into Wi-Fi setup mode

2. Boardcast WiFi SSID and password through sound on the hey_wifi web app

3. After the device received the SSID and  password and connected to the Wi-Fi network, it sends its IP address to the web app, and then the web app generates a link which is linked to the device dashboard.

4. Click the link with the device's IP address, we will get the dashboard of the device.

5. We can open a web terminal (ttyd) to add a new user. The default user is root. The default password is 1234.

6. We can also open JupyterLab which is a very powerful web IDE. It allows you to create and share documents that contain live code, equations, visualizations and narrative text. In this document, you can run python or shell scripts to play sound, plot graph and analyze data.

That's it. Do you like it? It would be great if you kindly give me some feedback.

When we get a smart speaker, the first thing we need to do is to connect it to the WiFi network. There are a lot of ways to setup the WiFi connection, such as SmartConfig, Bluetooth Setup. While we have a cool way to setup the WiFi connection for a smart speaker. As the title says, we can transmit WiFi settings through sound. The smart speaker decodes WiFi settings by analyzing sound, and then connect to the WiFi network.

The application contains two parts:

1. web app, based on quiet.js + pure.css

2. python app on the smart speaker, based on quiet.py

The code is at https://github.com/voice-engine/wave

Last time I used Microsoft's 3D Builder to make a 3D model. The 3D Builder is a handy tool and very easy to learn, but it lacks some functions. So I switched to Blender. 

Blender is the free and open source 3D creation suite. It supports the entirety of the 3D pipeline—modeling, rigging, animation, simulation, rendering, compositing and motion tracking, even video editing and game creation.

The story of Blender is very interesting and inspiring (youtu.be/qJEWOTZnFeg).

We designed a new mic array to build an open source smart speaker, you may wonder what it looks like. So we  created a simple 3D model to show a preview of the smart speaker. The model is created with 3D Builder of Windows 10. You can find the model at https://cdn.hackaday.io/files/1642217026031264/speaker1.3mf

Top view - 4 R/Y/G/B LEDs, 1 touch pad

Side view

Front view

This is a presentation of  this project for a hackaday event at Chaihuo Maker Space.  It is worth sharing it here. You can also find the .pptx file at https://cdn.hackaday.io/files/1642217026031264/Smart%20Speaker%20from%20Scratch.pptx

As an engineer, I always enjoy finding ways to complete a task. For me, the current task is making an open source smart speaker.

Why we should make an open source smart speaker? What's the to-do-list? How to make it? The presentation is to answer these three questions.

There are lots of existing smart speakers in the market. They are well-designed products. But they have two common issues. One is that they are always-listening black boxes, which scares many people. The other is that their functions is limited by their vendors. Smart speakers have very powerful ARM processors. They are just like smart phones with touch screens. Smart phones have so many apps, but smart speakers only run one app. Amazon Echo only has Alexa, Google Home only gets Google Assistant. We have no access inside of the smart speakers. We can't install any app on them.

Maybe we can get some ideas from the history of routers and the OpenWrt project. Before we have OpenWrt, routers were just black boxes. After that, we replace routers' firmware with OpenWrt, and then install a variety of apps, such as web servers, proxies, file storage and etc.

Maybe we can do the same to a smart speaker.

But it's a little bit different. Although Amazon released Echo's source code, it's extremely hard to hack a Echo device. For the 2015 and 2016 edition of the Echo, we can boot it from an external SD to hack it. For the latest Echo, it have been fixed.

For most of us, hacking an existing smart speaker is harder than making a new one. So why not make an open source smart speaker from scratch.

To find out what we need to do, we should learn how a smart speaker works.

These can be divided into 3 parts: hardware, software and document.

 
Hardware is kind of simple. We need a Single-Board-Computer (Raspberry Pi, ...), a mic array and a speaker. Software is the main part. Fortunately, there are lots of open source resources we can use.

Figure out what we need to do, then we can just find a way to do it.

The first prototype is made with Raspberry Pi, ReSpeaker 4 Mic Array and a 45mm Speaker. The cost is a little high.

So next step is to find some more affordable hardware. It seems NanoPi Neo Air is a good choice. It runs Armbian which is based Debian and is very similar with Raspbian.  A new mic array is being designed to reduce the cost.

Looking forward to your feedback! We can also follow or join the project on hackaday.io or github.

The first prototype based on ReSpeaker 4-Mic Linear Array has an issue that its microphones and FPC connector are on top of the PCB. It's not easy to attach the mic array to the case.

Besides, we want a cheaper, more compact and lower power consumption option.

So we just designed a new mic array. It works with NanoPi Neo Air, which is cheaper and lower power consumption than Raspberry Pi

The initial thought was like:

There are 4 microphones, 3 LEDs and 2 mounting holes. The mounting holes are also touch pads which can be used with metal screws as touch buttons. While it was hard to place all the components with this design, so we made some adjustments.

The new design is:

There are 4 microphones, 4 LEDs and 2 mounting holes and 1 touch pad + mounting hole.

You may notice that "VOICEN - Linear 4 Mic Array" is on the PCB, which is the product name.

We got the design work done at March 19th, and sent the gerber files to makerfabs, which provides PCB prototyping service at March 20th.

Today, we got the PCBs!!!

Based on the date on the packages, we can find that the PCB manufacturing process was actually super rapid. Its delivery took most of the time.

Do you like the design or the name "VOICEN - Linear 4 Mic Array"? Just leave a comment here.

This is a paper case smart speaker made with Raspberry Pi and ReSpeaker 4 Mic Linear Array. See also on Github

Hardware

The hardware includes:

• Raspberry Pi 3B (or 3B+)
• ReSpeaker 4 Mic Linear Array
• 45mm Speaker
• Laser-Cut Paper Case
• + 2 Nylon Rivets (R3035 or R3045), the diameter of mounting holes is 3mm (ф3). Screws will also work, while rivets is easier to use.

  

Raspberry Pi and ReSpeaker 4 Mic Linear Array can be found at Seeed Studio, and we get the 45mm speaker and nylon rivets from Alibaba or Taobao.

Design a Paper Case

To use a laser cutter, we need a CAD design file. I learned how to use a CAD software at college, but have not used CAD for many years. Fortunately, it is easy to design a paper case. We only have to know how draw lines and circels. Commercial CAD tools are quite expensive, but for this simple task, the open source CAD tool - LibreCAD is enough to get it done. You can design a unique paper case based on my work.

If you have not used any CAD software, it will take some time to be familiar with a CAD software like LibreCAD. You can also use Python script to genrate the CAD DXF file. Manfred Moitzi's handy Python library - ezdxf will help us to do the design work. For example, to draw an array of speaker holes, using Python script is much easier.

The Python script is like:

import ezdxf

dwg = ezdxf.new('R2007')
msp = dwg.modelspace()
d = 40.
n = int(40 / 1.5)

delta = d / n

start = (100, 100)

msp.add_circle(start, 20)

for x in range(-n, n):
    for y in range(-n, n):
        if y & 1:
            offset = 0.5
        else:
            offset = 0
        rx = ((x + offset)**2 + y**2)**0.5
        if rx <= (n/2 + 0.1):
            r = 0.35 # (0.25 * delta * (n/2 - rx) / (n/2) + 0.15 * delta)
            msp.add_circle((start[0] + (x + offset) * delta, start[1] + y * delta), r)

dwg.saveas('speaker_hole.dxf')

 The output will be:

We can use both LibreCAD and Python script to make a complex design.

Software

For a smart speaker, the software is the key point. The easiest way is using Alexa Voice Service or Google Assistant SDK. If you want to know how a voice assistant works, you should try MyCroft. If you prefer to offline voice assistant, Snips.ai is a good option. You can find a full list of resources at voice-engine/make-a-smart-speaker.