Hacking Voice UIs

Nadine Lessio joins us to talk about hacking voice interfaces

HackChat

Event Details

Nadine Lessio will be hosting the Hack Chat on Friday, July 13, 2018 at noon PDT.

Time Zones got you down? Here's a handy time converter!

Nadine Lessio is a designer and technologist based out of Toronto, Canada with a background in visual design, and DIY peripherals. Nadine holds an MDes from OCADU where she spent her time investigating the Internet of Things through personal assistants. Currently she works at OCADUs Adaptive Context Environments Lab where she’s researching how humans and devices work together. By mixing development, humour, and criticism, Nadine’s work considers the interplay between technology and life, sometimes with amusing or unexpected results.

Twitter: https://twitter.com/_nadine

Hackaday Profile: https://hackaday.io/sharkwheels
__________________

We’re going to be talking about experimenting with personal assistants and voice user interfaces. As devices like Alexa and Google Home become more prolific what are some ways to play with speculative ideas around agency or a device's possible internal motives? What are some interesting interactions that can be done with them, and what are some of the challenges faced in trying to do creative things with these devices?

Nadine will be answering the questions:

- What languages do you use to program voice UIs?
- How do you use voice and hardware together?
- What do you consider when designing for a voice UI?
- How do voice UI's interface with Arduinos, raspberry pis, and other common electronics platforms?

Event Logs

Collapse

Transcript: Hacking Voice UIs
Stephen Tranovich • 07/13/2018 at 20:02 • 0 comments

Stephen Tranovich12:05 PM
Hello hello hello, everybody, and welcome to another epic Hack Chat. This week we're talking about hacking voice user interfaces with @Nadine !

Thom12:05 PM
Thank you for putting this chat together.

Stephen Tranovich12:05 PM
@Nadine can you start by giving us a little bit of background about yourself and why you want to share this topic with us?

Stephen Tranovich12:05 PM
You're welcome, @Thom !

Nadine12:06 PM
Sure! I'm a designer / technologist working out of Toronto. I've done quite a bit of installation work, prototyping and DIY game peripherals, and lately I've been combining that w/ voice interfaces.

Nadine12:07 PM
I'm really interested in how we can use voice in non-standard ways.

Nadine12:07 PM
ie: in more art-like contexts, or how they work in an installation context, or even just doing some DIY stuff with them vs just home automation.

Stephen Tranovich12:08 PM
Could you share with us some of the applications you've been hacking voice UIs into recently?

Nadine12:09 PM
Well right now, its mostly controlling different peripherals. so I've been looking at how to control stuff like printers, or making applications where maybe you have to bring the device specific objects (nfc). I've also started playing w/ some embedded versions of them, vs just the consumer devices.

Stephen Tranovich12:11 PM
Awesome stuff!

Stephen Tranovich12:11 PM
Let's dive into the community questions, shall we?

Nadine12:11 PM
Sure!

Stephen Tranovich12:12 PM
We'll start out with the first question thrown up by @Thom , even though you touched at the answer in the discussion section already. Can we adapt devices to speak to other devices, yet? UI to UI

Nadine12:14 PM
You can. Its a little futzy, but the devices to recognize their wake words / phrases if they are within ear shot of one another.

Nadine12:14 PM
I've used Siri to control a google home and its pretty bizarre.

Stephen Tranovich12:15 PM
What are the good use cases and drawbacks for connecting systems in this way?

Nadine12:15 PM
Also you can rope them into an on going circle by using something like Eliza, or writing your own skills to keep them answering one another.

Nadine12:15 PM
Hmm. drawbacks /cases...

Nadine12:16 PM
well one is just entertainment. They aren't that great at context, so its sorta like having parrots in the room. You could also have a use case of just trying to streamline your commands.

Nadine12:17 PM
So rather than having to double up your programs for diff devices, just have one queen bee. Drawback tho...it could just not work, or stop working depending on if the platform is updated (this happens a lot).

Stephen Tranovich12:17 PM
Hah, I love the parrot analogy.

Nadine12:18 PM
You could also try to trip them off programaticially through notifications sent from a skill, tho again, its kind of touch and go because notifications are still sorta new, and the big two really want you to use them in certain ways.

Nadine12:19 PM
hmm, i guess other drawbacks are just how many internet microphones do you want in your house? heh.

Stephen Tranovich12:21 PM
They leads well into our next question

Stephen Tranovich12:21 PM
Our next question has been asked in various ways by multiple members of the community, including @Andrew and @Paul Stoffregen : what options are there for people who want to use their own voice UI but aren't comfortable with putting an internet-connected microphone in their house? Any ideas or tips about doing voice recognition locally, without any internet connectivity?

Nadine12:23 PM
Yeah! you could try something like Pocket Sphinx. Its being developed by carnegie mellon and its meant for doing commands locally.

Nadine12:24 PM
I've tinkered with it, but I'll be honest, it was a bit of a pain to get going in a virtual environment, I might try w/ out one. As I think its interesting.

Nadine12:25...
Read more »

View event log

Discussions

Paul Stoffregen wrote 07/13/2018 at 19:13

Any ideas or tips about doing voice recognition locally, without any internet connectivity?

Are you sure? yes | no

ThunderSqueak wrote 07/16/2018 at 10:03

There are a number of projects that do this, I have experience with https://cmusphinx.github.io/ in particular.

Are you sure? yes | no

Lutetium wrote 07/13/2018 at 18:59

How can we use these existing voice-command platforms to control functions of our own hardware hacks.?

Are you sure? yes | no

Andrew wrote 07/13/2018 at 18:59

There probably isn't an actual answer to this question, but what options are there for people who want to use their own voice UI but aren't comfortable with putting an internet-connected microphone in their house? What do you think of progress made by the open source community (e.g. jasper, mycroft, etc)?

Are you sure? yes | no

Thom wrote 07/13/2018 at 19:04

Kinda along my line of questioning. Is there a voice to text attribute in these devices so that speaking in my UI delivers the message to the IoT device instead of that device listening all the time or even needing to be there.

Are you sure? yes | no

Thom wrote 07/09/2018 at 15:33

My question: Can we adapt devices to speak to other devices, yet? As in, I don't want to have all the Google, Alexa, Siri devices in my house. I would rather have one set throughout the house and use it to tell the others to do it's thing. Example:: "Hey Google, tell Alexa to play (song x) from my Amazon collection."

Thank you for hosting this discussion.

Are you sure? yes | no

jess.t.moody wrote 07/13/2018 at 18:34

"Hey Google, tell Alexa to tell Siri to tell Cortana to tell Bixby to turn off the lights." 😁

Are you sure? yes | no

Nadine wrote 07/13/2018 at 18:41

I have gotten my devices into a bot circle before, so they do trip each other off. Its pretty funny! You could also try a chromecast library to trip off music etc to your google home, as they are just beefy chromecasts.

Are you sure? yes | no

Hacking Voice UIs

Become a Hackaday.io member

Just one more thing

Event Details

Event Logs Collapse

Transcript: Hacking Voice UIs

Enjoy this event?

Discussions

Become a Hackaday.io Member

Interested in attending?

Report project as inappropriate

Send message

Remove Member

Event Logs

Collapse