Close
0%
0%

Room Assistant

A project for an voice and gesture controlled

james-wynhoffJames Wynhoff
Following
Liked Like project

Just one more thing

To make the experience fit your profile, pick a username and tell us what interests you.

Pick an awesome username
hackaday.io/
Your profile's URL: hackaday.io/username. Max 25 alphanumeric characters.
Pick a few interests
Projects that share your interests
People that share your interests

We found and based on your interests.

Choose more interests.

OK, I'm done! Skip
Similar projects worth following
1.7k views
0 comments
7 followers
View Gallery
Things this *thing* should do:
0.) Be unobtrusive, be helpful, be self explanatory, be VALUABLE.

1.) Have a microphone, we want to talk to it
2.) Have a speaker system of some sort, it needs to talk to us
3.) Have a camera capable of at least 15fps and 640x480(minimum), it needs to see us, and react to both our presence, and our gestures
4.) PIR Sensor, that way we can get a fast motion detection event, and in the event privacy mode is enabled, still respond to motion
5.) It should have sensors for Temp/Humidity, CO2/Smoke would be nice as well, because it should know the state of the room its in.
~~~~~~~~~~~~~THINGS I *HOPE* IT WILL DO~~~~~~~~~~~~`
6.) netboot, with no sd card, big hope
7.) pull power via real POE, or at least fake POE for now (+12 on 4/5 Gnd on 7/8)
8.) boot *FAST*, it should never reboot in an ideal world, but if it does have to reboot, it should be FAST

All parts of the project can be found by clicking here:

https://hackaday.io/projects?tag=JAW.HA

So far I have picked the orange pi one as the SBC of choice for this application.

Orange Pi One (and camera):

https://www.aliexpress.com/item/Orange-Pi-One-SET-9-Orange-Pi-One-and-Camera-Camera-with-wide-angle-lens-for/32660642513.html

I will be using the PAM8403 as the amplifier, the boards I got for prototyping are these:

https://www.aliexpress.com/item/10PCS-PAM8403-Super-mini-digital-amplifier-board-2-3W-Class-D-digital-amplifier-board-efficient-2/1547727101.html

0.16usd each? Holy cow batman...

Hands on photos coming soon to the logs

  • CPU stable!

    James Wynhoff • 03/28/2018 at 22:39 • 0 comments

    As per the last post I changed the CPU to run at a fixed speed, and here is the output of cpufreq-info now :)

    cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009
Report errors and bugs to cpufreq@vger.kernel.org, please.
analyzing CPU 0:
  driver: cpufreq-sunxi
  CPUs which run at the same hardware frequency: 0 1 2 3
  CPUs which need to have their frequency coordinated by software: 0 1 2 3
  maximum transition latency: 2.00 ms.
  hardware limits: 1.20 GHz - 1.20 GHz
  available frequency steps: 60.0 MHz, 120 MHz, 240 MHz, 312 MHz, 408 MHz, 480 MHz, 504 MHz, 528 MHz, 576 MHz, 600 MHz, 624 MHz, 648 MHz, 672 MHz, 720 MHz, 768 MHz, 816 MHz, 864 MHz, 912 MHz, 960 MHz, 1.01 GHz, 1.06 GHz, 1.10 GHz, 1.15 GHz, 1.20 GHz, 1.25 GHz, 1.30 GHz, 1.34 GHz, 1.44 GHz, 1.54 GHz
  available cpufreq governors: interactive, conservative, ondemand, powersave, userspace, performance
  current policy: frequency should be within 1.20 GHz and 1.20 GHz.
                  The governor "performance" may decide which speed to use
                  within this range.
  current CPU frequency is 1.20 GHz (asserted by call to hardware).
  cpufreq stats: 60.0 MHz:0.00%, 120 MHz:0.00%, 240 MHz:0.00%, 312 MHz:0.00%, 408 MHz:0.00%, 480 MHz:0.00%, 504 MHz:0.00%, 528 MHz:0.00%, 576 MHz:0.00%, 600 MHz:0.00%, 624 MHz:0.00%, 648 MHz:0.00%, 672 MHz:0.00%, 720 MHz:0.00%, 768 MHz:0.00%, 816 MHz:0.00%, 864 MHz:0.00%, 912 MHz:0.00%, 960 MHz:0.00%, 1.01 GHz:0.00%, 1.06 GHz:0.00%, 1.10 GHz:0.00%, 1.15 GHz:0.00%, 1.20 GHz:100.00%, 1.25 GHz:0.00%, 1.30 GHz:0.00%, 1.34 GHz:0.00%, 1.44 GHz:0.00%, 1.54 GHz:0.00%  (1)
analyzing CPU 1:
  driver: cpufreq-sunxi
  CPUs which run at the same hardware frequency: 0 1 2 3
  CPUs which need to have their frequency coordinated by software: 0 1 2 3
  maximum transition latency: 2.00 ms.
  hardware limits: 1.20 GHz - 1.20 GHz
  available frequency steps: 60.0 MHz, 120 MHz, 240 MHz, 312 MHz, 408 MHz, 480 MHz, 504 MHz, 528 MHz, 576 MHz, 600 MHz, 624 MHz, 648 MHz, 672 MHz, 720 MHz, 768 MHz, 816 MHz, 864 MHz, 912 MHz, 960 MHz, 1.01 GHz, 1.06 GHz, 1.10 GHz, 1.15 GHz, 1.20 GHz, 1.25 GHz, 1.30 GHz, 1.34 GHz, 1.44 GHz, 1.54 GHz
  available cpufreq governors: interactive, conservative, ondemand, powersave, userspace, performance
  current policy: frequency should be within 1.20 GHz and 1.20 GHz.
                  The governor "performance" may decide which speed to use
                  within this range.
  current CPU frequency is 1.20 GHz (asserted by call to hardware).
  cpufreq stats: 60.0 MHz:0.00%, 120 MHz:0.00%, 240 MHz:0.00%, 312 MHz:0.00%, 408 MHz:0.00%, 480 MHz:0.00%, 504 MHz:0.00%, 528 MHz:0.00%, 576 MHz:0.00%, 600 MHz:0.00%, 624 MHz:0.00%, 648 MHz:0.00%, 672 MHz:0.00%, 720 MHz:0.00%, 768 MHz:0.00%, 816 MHz:0.00%, 864 MHz:0.00%, 912 MHz:0.00%, 960 MHz:0.00%, 1.01 GHz:0.00%, 1.06 GHz:0.00%, 1.10 GHz:0.00%, 1.15 GHz:0.00%, 1.20 GHz:100.00%, 1.25 GHz:0.00%, 1.30 GHz:0.00%, 1.34 GHz:0.00%, 1.44 GHz:0.00%, 1.54 GHz:0.00%  (1)
analyzing CPU 2:
  driver: cpufreq-sunxi
  CPUs which run at the same hardware frequency: 0 1 2 3
  CPUs which need to have their frequency coordinated by software: 0 1 2 3
  maximum transition latency: 2.00 ms.
  hardware limits: 1.20 GHz - 1.20 GHz
  available frequency steps: 60.0 MHz, 120 MHz, 240 MHz, 312 MHz, 408 MHz, 480 MHz, 504 MHz, 528 MHz, 576 MHz, 600 MHz, 624 MHz, 648 MHz, 672 MHz, 720 MHz, 768 MHz, 816 MHz, 864 MHz, 912 MHz, 960 MHz, 1.01 GHz, 1.06 GHz, 1.10 GHz, 1.15 GHz, 1.20 GHz, 1.25 GHz, 1.30 GHz, 1.34 GHz, 1.44 GHz, 1.54 GHz
  available cpufreq governors: interactive, conservative, ondemand, powersave, userspace, performance
  current policy: frequency should be within 1.20 GHz and 1.20 GHz.
                  The governor "performance" may decide which speed to use
                  within this range.
  current CPU frequency is 1.20 GHz (asserted by call to hardware).
 cpufreq stats: 60.0 MHz:0.00%, 120 MHz:0.00%, 240 MHz:0.00%, 312 MHz:0.00%, 408 MHz:0.00%, 480 MHz:0.00%, 504 MHz:0.00%, 528 MHz:0.00%, 576 MHz:0.00%, 600 MHz:0.00%, 624 MHz:0.00%, 648 MHz:0.00%, 672 MHz:0.00%, 720 MHz:0.00%, 768 MHz:0.00%, 816 MHz:0.00%, 864...
    Read more »

  • Asic Errors, USB Woes, and !winning!

    James Wynhoff • 03/28/2018 at 16:09 • 0 comments

    I think the next thing I'm going to work on will be SPI / netboot

    I'm going to start with a newly compiled uboot on the SD card to get netbooting working, then I will transfer that working uboot to the SPI flash, cant wait to be done flashing and reading sd cards all day.

    The results of that are this correctly configured fex though (opi one), use fex2bin and replace /boot/script.bin

    this gets rid of the asic CPU set errors, but it also sets the CPU to a constant speed (which is ok with me), and fixes USB issues

    --------------UPDATE-----------------

    I was just told that the site malformed this file, DO NOT COPY/PASTE THIS

    I will post the file in the files section

    [product]
version = "100"
machine = "Xunlong Orange Pi One"

[platform]
debug_mode = 1
eraseflag = 1
next_work = 2

[target]
boot_clock = 1008
storage_type = -1

[key_detect_en]
keyen_flag = 0

[fel_key]
fel_key_max = 7
fel_key_min = 2

[card_boot]
logical_start = 40960
sprite_work_delay = 500
sprite_err_delay = 200
sprite_gpio0 = port:PL10<1><default><default><default>
next_work = 3

[box_start_os]
used = 1
start_type = 1
irkey_used = 1
pmukey_used = 1
pmukey_num = 3
led_power = 0
led_state = 0

[boot_init_gpio]
used = 1
gpio0 = port:PL10<1><default><default><1>
gpio1 = port:PG11<1><default><default><1>

[recovery_para]
used = 1
mode = 2
recovery_key = port:PL04<0><default><default><default>

[pm_para]
standby_mode = 1

[card0_boot_para]
card_ctrl = 0
card_high_speed = 1
card_line = 4
sdc_d1 = port:PF00<2><1><2><default>
sdc_d0 = port:PF01<2><1><2><default>
sdc_clk = port:PF02<2><1><2><default>
sdc_cmd = port:PF03<2><1><2><default>
sdc_d3 = port:PF04<2><1><2><default>
sdc_d2 = port:PF05<2><1><2><default>

[card2_boot_para]
card_ctrl = 2
card_high_speed = 1
card_line = 8
sdc_cmd = port:PC06<3><1><2><default>
sdc_clk = port:PC05<3><1><2><default>
sdc_d0 = port:PC08<3><1><2><default>
sdc_d1 = port:PC09<3><1><2><default>
sdc_d2 = port:PC10<3><1><2><default>
sdc_d3 = port:PC11<3><1><2><default>
sdc_d4 = port:PC12<3><1><2><default>
sdc_d5 = port:PC13<3><1><2><default>
sdc_d6 = port:PC14<3><1><2><default>
sdc_d7 = port:PC15<3><1><2><default>
sdc_2xmode = 1
sdc_ddrmode = 1

[twi_para]
twi_port = 0
twi_scl = port:PA11<2><default><default><default>
twi_sda = port:PA12<2><default><default><default>

[uart_para]
uart_debug_port = 0
uart_debug_tx = port:PA04<2><1><default><default>
uart_debug_rx = port:PA05<2><1><default><default>

[force_uart_para]
force_uart_port = 0
force_uart_tx = port:PF02<3><1><default><default>
force_uart_rx = port:PF04<3><1><default><default>

[jtag_para]
jtag_enable = 0
jtag_ms = port:PA00<3><default><default><default>
jtag_ck = port:PA01<3><default><default><default>
jtag_do = port:PA02<3><default><default><default>
jtag_di = port:PA03<3><default><default><default>

[clock]
pll_video = 297
pll_ve = 402
pll_periph0 = 600
pll_gpu = 576
pll_periph1 = 600
pll_de = 864

[dram_para]
dram_clk = 624
dram_type = 3
dram_zq = 0x3b3bfb
dram_odt_en = 1
dram_para1 = 283377664
dram_para2 = 0
dram_mr0 = 6208
dram_mr1 = 64
dram_mr2 = 24
dram_mr3 = 2
dram_tpr0 = 0x48a192
dram_tpr1 = 0x1c2418d
dram_tpr2 = 0x76051
dram_tpr3 = 0x0
dram_tpr4 = 0x0
dram_tpr5 = 0x0
dram_tpr6 = 0x64
dram_tpr7 = 0x0
dram_tpr8 = 0x0
dram_tpr9 = 0x0
dram_tpr10 = 0x0
dram_tpr11 = 0x6aaa0000
dram_tpr12 = 0x7979
dram_tpr13 = 0x800800

[wakeup_src_para]
cpu_en = 0
cpu_freq = 48
pll_ratio = 273
dram_selfresh_en = 1
dram_freq = 36
wakeup_src0 =
wakeup_src_wl = port:PG10<4><default><default><0>
wakeup_src_bt = port:PL03<6><default><default><0>

[twi0]
twi_used = 1
twi_scl = port:PA11<2><default><default><default>
twi_sda = port:PA12<2><default><default><default>

[twi1]
twi_used = 1
twi_scl = port:PA18<3><default><default><default>
twi_sda = port:PA19<3><default><default><default>

[twi2]
twi_used = 0
twi_scl = port:PE12<3><default><default><default>
twi_sda = port:PE13<3><default><default><default>

[uart0]
uart_used = 1
uart_port = 0
uart_type = 2
uart_tx = port:PA04<2><1><default><default>
uart_rx = port:PA05<...
    Read more »

  • I need a beer (or 6)

    James Wynhoff • 03/27/2018 at 06:54 • 0 comments

    So, after bashing my head against the wall for a long time I finally got the camera working, I had to start with a 5.20 image because hte fex in later build is wrong.....

    Ill be posting the image later, it will save you much much trouble....

  • Soup is done!

    James Wynhoff • 03/27/2018 at 00:21 • 0 comments

    That's a lie.

    I came back and realized I didnt add  linux-headers-sun8i to my first apt-get, as they are not installed by default.

    Edit:

    4 hours later, still futzing to get it to load the drivers and detect the CSI camera.......

    Sigh

  • Lunch time

    James Wynhoff • 03/26/2018 at 19:23 • 0 comments

    The first few packages, ran this in a file named step1.sh:

    sudo apt-get update -y
sudo apt-get upgrade -y
sudo apt-get install python-dev python-pip python-opencv libv4l-dev -y
git clone https://github.com/umlaeute/v4l2loopback
cd v4l2loopback/
make && make install
cd ..
git clone https://github.com/lhelontra/vidcopy
cd vidcopy/
gcc vidcopy.c -o vidcopy
cd ..
echo "v4l2loopback" | sudo tee -a /etc/modules
echo "gc2035" | sudo tee -a /etc/modules
echo "vfe_v4l2" | sudo tee -a /etc/modules

     That will take a while, time for lunch and to start the the next project!

  • Let there be sound?

    James Wynhoff • 03/26/2018 at 18:45 • 0 comments 


    root@orangepione:~# speaker-test

speaker-test 1.0.28

Playback device is default
Stream parameters are 48000Hz, S16_LE, 1 channels
Using 16 octaves of pink noise
Rate set to 48000Hz (requested 48000Hz)
Buffer size range from 256 to 524288
Period size range from 128 to 65536
Using max buffer size 524288
Periods = 4
was set period_size = 65536
was set buffer_size = 524288
 0 - Front Left

    Oops, looks like I flipped the speakers though!

  • Armbian, and the need for speed

    James Wynhoff • 03/26/2018 at 18:32 • 0 comments

    First thing was first, I had done some research on reading temp sensors and they fail at 408mhz, which is the "slow" speed, this board doesn't have a proper voltage selectable voltage regulator, just a VR that is settable via resistors, and that's 1.2ghz or 408mhz

    sudo nano /etc/default/cpufrequtils

    ENABLE=true
MIN_SPEED=1200000
MAX_SPEED=1200000
GOVERNOR=performance

    sudo watch -n 1  cat /sys/devices/system/cpu/cpu*/cpufreq/cpuinfo_cur_freq

    Every 1.0s: cat /sys/devices/system/cpu/cpu0/cpufre...  Mon Mar 26 20:31:38 2018

1200000
1200000
1200000
1200000

    Much better :)

  • WHY?!?

    James Wynhoff • 03/26/2018 at 17:47 • 0 comments

    Why is it that yesterday, when I didn't need them, I came across at least 3 usb > 3.3v serial adapters... now that I need ONE to debug this I cant find any.....


    Prop plug to the rescue!

    U-Boot 2017.09-armbian (Nov 22 2017 - 16:47:31 +0100) Allwinner Technology

CPU:   Allwinner H3 (SUN8I 1680)
Model: Xunlong Orange Pi One
DRAM:  512 MiB
MMC:   SUNXI SD/MMC: 0
*** Warning - bad CRC, using default environment

In:    serial
Out:   vidconsole
Err:   vidconsole
Net:   phy interface0
eth0: ethernet@1c30000
230454 bytes read in 195 ms (1.1 MiB/s)
starting USB...
USB0:   USB EHCI 1.00
USB1:   USB OHCI 1.0
scanning bus 0 for devices... 1 USB Device(s) found
scanning bus 1 for devices... 2 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
Autoboot in 1 seconds, press  to stop
=>

U-Boot SPL 2017.09-armbian (Nov 22 2017 - 16:47:31)
DRAM: 512 MiB
Trying to boot from MMC1


U-Boot 2017.09-armbian (Nov 22 2017 - 16:47:31 +0100) Allwinner Technology

CPU:   Allwinner H3 (SUN8I 1680)
Model: Xunlong Orange Pi One
DRAM:  512 MiB
▒
U-Boot SPL 2017.09-armbian (Nov 22 2017 - 16:47:31)
DRAM: 512 MiB
Trying to boot from MMC1


U-Boot 2017.09-armbian (Nov 22 2017 - 16:47:31 +0100) Allwinner Technology

CPU:   Allwinner H3 (SUN8I 1680)
Model: Xunlong Orange Pi One
DRAM:  512 MiB
MMC:   SUNXI SD/MMC: 0
*** Warning - bad CRC, using default environment

In:    serial
Out:   vidconsole
Err:   vidconsole
Net:   phy interface0
eth0: ethernet@1c30000
230454 bytes read in 194 ms (1.1 MiB/s)
starting USB...
USB0:   USB EHCI 1.00
USB1:   USB OHCI 1.0
scanning bus 0 for devices... 1 USB Device(s) found
scanning bus 1 for devices... 2 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
Autoboot in 1 seconds, press  to stop
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found U-Boot script /boot/boot.scr
3708 bytes read in 198 ms (17.6 KiB/s)
## Executing script at 43100000
U-boot loaded from SD
Boot script loaded from mmc
115 bytes read in 150 ms (0 Bytes/s)
3812646 bytes read in 447 ms (8.1 MiB/s)
4751880 bytes read in 509 ms (8.9 MiB/s)
Found legacy kernel configuration
34704 bytes read in 494 ms (68.4 KiB/s)
## Loading init Ramdisk from Legacy Image at 43300000 ...
   Image Name:   uInitrd
   Image Type:   ARM Linux RAMDisk Image (gzip compressed)
   Data Size:    3812582 Bytes = 3.6 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK

Starting kernel ...

[sun8i_fixup]: From boot, get meminfo:
        Start:  0x40000000
        Size:   512MB
ion_carveout reserve: 160m@0 256m@0 130m@1 200m@1
ion_reserve_select: ion chipid  [0x2c00081!
ion_reserve_common: ion reserve: [0x56000000, 0x60000000]!
Loading, please wait...
      Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... Scanning for Btrfs filesystems
 done.
 Begin: Will now check root file system ... fsck from util-linux 2.25.2
[/sbin/fsck.ext4 (1) -- /dev/mmcblk0p1] fsck.ext4 -a -C0 /dev/mmcblk0p1
/dev/mmcblk0p1: clean, 34994/72720 files, 199522/290816 blocks
 done.
done.
Begin: Running /scripts/local-bottom ... done.
Begin: Running /scripts/init-bottom ...   done.

Welcome to Debian GNU/Linux 8 (jessie)!

          Expecting device dev-ttyS0.device...
[  OK  ] Reached target Remote File Systems (Pre).
[  OK  ] Reached target Encrypted Volumes.
[  OK  ] Reached target Paths.
[  OK  ] Set up automount Arbitrary Executable File Formats F...utomount Point.
[  OK  ] Reached target Swap.
[  OK  ] Created slice Root Slice.
[  OK  ] Listening on /dev/initctl Compatibility Named Pipe.
[  OK  ] Listening on Delayed Shutdown Socket.
[  OK  ] Listening on Journal Socket (/dev/log).
[  OK  ] Created slice User and Session Slice.
[  OK  ] Listening on udev Control Socket.
[  OK  ] Listening on udev Kernel Socket.
[  OK  ] Listening on Journal Socket.
[  OK  ] Created slice System Slice.
         Starting Increase datagram queue length...
 Starting Restore / save the...
    Read more »

  • Not that easy, OFC

    James Wynhoff • 03/26/2018 at 17:43 • 0 comments

    So as expected, NOT that easy, its boot looping, mashing space bar got the boot to stop

    -------ALSO-------

    An annoyance with documentation, taking pictures of stuff stuff when its dirty, I'm going to be forced to clean everything during this....

  • OS Image

    James Wynhoff • 03/26/2018 at 17:04 • 0 comments

    Ok, im going to start with this image and then explore more from there.....

    https://dl.armbian.com/orangepione/Debian_jessie_default.7z.torrent

    Should be easy... Right?

View all 13 project logs

  • 1
    Enabling Camera and hardware h264 encoding

    From a 5.35 raspbian image for the orange pi one

    sudo apt-get install libgd2-xpm-dev libjpeg-dev libpng-dev libva-dev libv4l-dev libpulse-dev libx264-dev libmp3lame-dev libfreetype6-dev -y

git clone https://github.com/avafinger/ffmpeg_cedrus264_H3_Armbian_5.23.git
sudo dpkg -i ./ffmpeg_cedrus264_H3_Armbian_5.23/ffmpeg-3.1.4_1.0-3.deb

modprobe -r -v vfe_v4l2

modprobe -r -v gc2035

modprobe gc2035 

modprobe vfe_v4l2

echo "gc2035" | sudo tee -a /etc/modules
echo "vfe_v4l2" | sudo tee -a /etc/modules

    now we can start playing with the webcam in a meaningful way

    sudo ffmpeg-3.1.4 -f v4l2 -channel 0 -video_size 1920x1080 -i /dev/video0 -pix_fmt nv12 -r 22 -c:v cedrus264 night_video_test4_1920x1080.mp4

View all instructions

Enjoy this project?

Share

Discussions

Similar Projects

4-Port USB2.0 Hub based on HS8836A.
Project Owner Contributor

HS8836A USB 2.0 Hub

stefan-wagnerStefan Wagner

USB to Serial Converter based on CH330N
Project Owner Contributor

USB to Serial Converter

stefan-wagnerStefan Wagner

An adapter that allows you to connect to the serial console port of a router or switch over wifi.
Project Owner Contributor

ConsoleMate

ryan-aguilarRyan Aguilar

UART hacking a low cost DVB-S2 receiver box based on MStar MIPS32 chipset
Project Owner Contributor

Manhattan SX - Hacking a low cost DVB-S2 receiver

sphaleronsphaleron

Does this project spark your interest?

Become a member to follow this project and never miss any updates

Going up?

About Us Contact Hackaday.io Give Feedback Terms of Use Privacy Policy Hackaday API

© 2024 Hackaday

Yes, delete it Cancel

Report project as inappropriate

You are about to report the project "Room Assistant", please tell us the reason.

Send message

Your application has been submitted.

Remove Member

Are you sure you want to remove yourself as a member for this project?

Project owner will be notified upon removal.