Tron Identity Disc upgrade.1

Parts layout & wiring has made a lot of progress. Power switch is working. The halves still close.

In the pic below:

• button/speaker
• audio amp
• pager motor
• battery
• Using the LED common connections on the old main/switch board as a power bus.
• charger is beneath the switch board
• dual FET for main power switch and pager motor
• battery door space for: RTC, accelerometer, external USB connector
• RPI Zero (USB power, data, and SDIO wiring visible, other connections on the other side)
• ESP8266 for SDIO WiFi
• audio and button connectors, LED power

I investigated realtime and deadline schedulers to see if they could improve reliability of WS2812 LEDs connected to SPI1 on a RPI.

In short: no.

Additionally, the bcm2385 main & aux SPI drivers definitely need some work to support bit lengths other than 8.

Surprisingly, I found that USB (or the USB network gadget) has a huge impact (I read USB on RPI makes use of a special high priority, fast interrupt).
Activating the network, or a data transfer of several kB causes major glitches in the SPI/LED timing - until the traffic dies down.

More details below...
The docs I've read suggest that the scheduler/interrupt latency varies around 75-100us. Generally, every task or interrupt handler should get a chance to run in that time period, even in the worst case of a 100% cpu load (give or take priorities). But for the 9us LED reset, that is too quick for an interrupt (so can't do something tricky like connecting SPI1.CS0 to a GPIO pin and monitor it with an interrupt)

The linux command "chrt" changes the task scheduler and priority for a process. This allows you to test the "realtime" scheduler to give a process the highest available priority, and longest runtime slice. Unfortunately this doesn't help my SPI/WS2812 problem.

There is a recent kernel option for SCHED_DEADLINE that can guarantee scheduling for recurring tasks, which could help avoid timing out that 9us reset. The chrt app should allow testing sched_deadline -- but the util-linux package that contains it is old. Building chrt from source was not hard (./configure; make chrt).

Deadline has a few parameters: process expects to run for T (it can finish early), must finish before D each period, and period is P.
T<=D<=P and T >= 1024 (all microseconds)

But I only get an error "chrt: failed to set pid 0's policy: Invalid argument"
So I guess that deadline wasn't built into the RPI kernel.

So, can we make the SPI transmission more efficient?
To review: I'm sending zero as 0b100 and one as 0b110 (2x3 bits padded to 8).
Theoretically, stretching out the transmitted LED bit 1x0, into 111xxx0 and transmitting at 3x the rate saves some time. But the buffer also empties out 50% faster (3x rate and 1/2 the LEDs per byte) .

Can we make the transmission slower, to get more time to refill the SPI buffer?
The LEDs lose all data at 1.9 MHz (estimates say they should work down to 1.7Mhz).
The internet suggests there is a 1us padding before & after a SPI transmit. If the data line idles at 0, we could transmit __1x__ at 1.88 MHz. (Or a little bit more gain with variable length __1__ and __11__)
Unfortunately the main and aux SPI drivers only support 8 bit transfers. Furthermore, the aux driver sets variable width SPI mode, wasting one of the bytes in the 4 byte buffer. (It is a 4 byte buffer? The Broadcom doc is not so good)
Additionally, if the SPI transfer will take longer than 30us, the driver will setup to wait for the IRQ instead of busy-wait. At 1 MHz that is less than 30 bits. That may be a big problem for this application.

First: Many Thanks to Tim's Blog for great work on Understanding the WS2812.

Short version is, I now have a basic & good enough WS2812b SPI driver in Python, working with the RPI SPI1. But it was pretty rough. I ran into some really weird issues...

I'm packing 2 LED bits into 6 SPI hardware bits (padded to 8 SPI bits) at around 2.4 - 2.8 MHz.

LED 0 bit = 0b100 = 0x4
LED 1 bit = 0b110 = 0x6
LED 0x7F = 0x46, 0x66, 0x66, 0x66

Efficiency might prefer 5 into 15 bits (or 10 into 30). But apparently the spi_bcm2835aux driver doesn't handle any bit length besides 8. The SpiDev doc says 8-16, and Broadcom docs say 1-24/32 bits.
(I looked at SpiDev source, it looks like it is trying to pass the bit length along)

SPI0 seemed to work great - it has DMA (but this project will be using SPI0 for SDIO).

Switching to SPI1, the problems start... SPI1 does not have DMA.
The first write to SPI1 after opening the SPI1 device was always corrupt. Temporarily fixed by padding the start with zeros.
Adding multiple updates: There were a lot of glitches, a few per second.

But a HUGE problem was: after 5-6 seconds (sometimes less) of the script running, the LEDs would get stuck on bright white. Not the whole strip, just the ones I'm controlling. Nothing changes it: number of LEDs driven, update frequency, open/close the SPI device in the update loop, checked voltages, twiddled timings.... Only re-running the program resets the LEDs for another several seconds.
And SPI0 had the same problem!

I had to sleep on it.

A sleep(4) at the start of the script (before opening the spi device) would give me 2 seconds of good LEDs -- so somehow, seems related to program runtime.
Replacing the sleep in the update with a busy loop was a successful, but inefficient, workaround.
Nice/Priority didn't change much (maybe made it worse).

I looked into CPU speed

sudo cat /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_cur_freq

Switching to "performance" (always 1GHz) fixes just about everything: the 6 sec problem, the first write, less glitches (about 1 per 10 sec).

echo performance | sudo tee /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor

Tweaking:

It seems that spi.xfer() takes slightly longer when glitches occur -- presumably due to a context switch.
(writebytes() seems to always take the same time)

Using that as a filter, the script can immediately re-send the LED data -- theoretically reducing the visible glitches (but some still get through). It doesn't seem to get worse under high cpu load (tested with "stress -c 1")

The current result is approximately one glitch every 20-30 sec -- but there can be bursts of glitches for unknown reasons. (For now, just think of of it like the brightness variation in the original Tron -- or like Max Headroom glitches).

Networking the PI through Windows isn't a primary goal of this project -- but not having network access (or even worse, having it intermittently) makes setting things up much, much harder. On top of that, a flaky network setup messes with Windows' cloud sync -- making some apps unresponsive, lock up, and possibly lose changes (bad if you're trying to take notes).

The problem(s) appears to be mismatches between the linux dhclient and DHCP server in Windows Internet Connection Sharing (ICS), and a flaky RNDIS driver/implementation.
This could be avoided in various ways, but ICS seems like a nice, simple way to go (if it worked better).

The workaround version is:

First boot: Wait at least 2-4 minutes for the PI to give up on dhcp and assume a link-local 169.254 address
Repeat: down-up the interface / disconnect usb / reboot -- until dhcp succeds
(it may succeed the 2nd time due to starting from the link local address & netmask?)
(but may still take a couple minutes after the login prompt appears)
Later boots: will probably still be slow because dhcp is intermittent, but it may fall back to the last saved dhcp reservation settings (if the address is not in use).

More Details below:

---------- more ---------

PreReq:

Setup usb gadget /boot/cmdline.txt :: dwc2,g_ether g_ether.host_addr=xx:xx... g_ether.dev_addr=xx:xx...
Change device driver to RNDIS
Setup ICS

First: how to spot part of the problem in windows

Open "View Network Connections" (typing in the start menu to search seems most consistent).
Windows may disable the RPI network device driver: re-enabling will fail, networking will not work. Then the RPI must be disconnected from USB and rebooted. (if it's not disconnected, or if you tried to re-enable it -- then windows may also require a reboot)

(the windows event log will show repeated 10400 NDIS "The network interface ... has begun restarting." -- "Reason: The network driver did not respond to an OID request in a timely fashion.")

Second: getting the right commands, to trigger a dhcp request

This first requires adding to /etc/network/interfaces
(otherwise you will get an error "ifdown: interface usb0 not configured")

allow-hotplug usb0
iface usb0 inet dhcp

Then, this command will restart dhcp on the interface

sudo ifdown usb0 && sudo ifup usb0

it will also show the output of dhclient. A typical result looks like this

DHCPDISCOVER on usb0 to 255.255.255.255 port 67 interval 5
DHCPDISCOVER on usb0 to 255.255.255.255 port 67 interval 12
DHCPREQUEST on usb0 to 255.255.255.255 port 67
DHCPOFFER from 192.168.137.1
DHCPREQUEST on usb0 to 255.255.255.255 port 67
DHCPDISCOVER on usb0 to 255.255.255.255 port 67 interval 3
DHCPDISCOVER on usb0 to 255.255.255.255 port 67 interval 5
DHCPDISCOVER on usb0 to 255.255.255.255 port 67 interval 13
DHCPDISCOVER on usb0 to 255.255.255.255 port 67 interval 19
DHCPREQUEST on usb0 to 255.255.255.255 port 67
DHCPOFFER from 192.168.137.1
DHCPREQUEST on usb0 to 255.255.255.255 port 67
DHCPACK from 192.168.137.1
bound to 192.168.137.118 -- renewal in 236 seconds.

Note: dhclient recieved two offers (from 137.1) but did not take the first one -- I've even seen it refuse 4 offers...
Unless it works immediatley -- there always seems to be 4 discovers before a request...
Where or how are the requests/offers lost or corrupted?
Future: there is a tool called dhcpdump that logs all dhcp traffic on an interface

Space inside the disc was roughly estimated by examining pictures and video, and similar projects.
http://www.instructables.com/id/Custom-Tron-Disc-Mod/?ALLSTEPS
https://www.flickr.com/photos/blackfeathers/sets/72157625208305212/

LED: The RPI can drive WS2812b LEDs with DMA https://github.com/jgarff/rpi_ws281x
This conflicts with PWM sound, but we have options:

• No Sound / simple beeper
• Disable LED updates while playing sound
  
• Use existing disc chip sounds
• square wave sound
  
• I2S audio
• "Upsample" audio to the shared PWM frequency?
• Add a microcontroller for LED strip
  ESP8266: switch between sdio firmware & on board flash
  (using WiFi would disable LEDs)
  
• SPI LED drive - previously dismissed because no DMA
  • But now Kernel 4.0? has DMA support for SPI? spi-bcm2835
    https://github.com/raspberrypi/linux/pull/1085
    for transfer length >= 96 bytes, and < 65536 bytes
  • Aux SPI can do variable bit length (which may help?) but no DMA?

WiFi: The RPI has support for a 2nd SD / SDIO port, and the ESP8266 has a SDIO bootloader mode. https://hackaday.io/project/8678-rpi-wifi
The just released Pi Zero W (if/when you can get one) may make this redundant. But the ESP8266 may have future potential here as a scriptable WiFi/SPI/GPIO expander.

Battery: The "Custom-Tron-Disc-Mod" project above stated AA would not fit but AAA would.
But for this project, with more LEDs -- a LiPo can have a larger capacity (for more runtime and brighter lights), and fit more compactly (allowing more peripherals).
Early plans were to fit the battery in the battery area - possibly removing the screw for the USB jack.
But latest plans are to put the battery in the largest open area - to eliminate sharp corners that could puncture the LiPo.

Other technical hurdles:

• Organizing notes & plans: OneNote App has issues: slow to paste, flaky search, can totally hang up if network issues
• RPI USB Ethernet Gadget trouble (device recognized, but no data transfer)
  • Has worked in the past
  • Always seemed flaky (maybe due to Win10 drivers and some OS corruption from multiple small disk failures)
  • Then worked on one machine but not another
  • Now not working anywhere: may be corrupt SD? (dmesg does show some errors)