Low Power ESP32 Handheld

This latest software update adds support for iOS notifications. This way, the ESP32 handheld can be used as a notification display.

Besides Wi-Fi the ESP32 has integrated Bluetooth and BLE. Similar to a smartwatch, this allows the ESP32 to receive notifications that are broadcast from your phone. On Apple Devices the Apple Notification Center Service (ANCS) makes this possible without the need for any third party software. The only thing left is a bit of code on the ESP32 to subscribe to these messages. Neil Kolban Is responsible for most of the groundwork on the ESP32 BLE libraries. I first became aware of the efforts to support ANCS though an Issue on GitHub. Now, two years later James Hudson (jhud) has released an easy-to-use library.

Integrating the library into my existing code was simple. An incoming message is shown as a popup on top of the screen. Interactions like answering an incoming phone call are possible in theory but not implemented yet.

Since Wi-Fi and Bluetooth share the same interface and antenna it's difficult to get them both to work simultaneously. Right now the BLE connection has to be stopped to enable Wi-Fi. I couldn't get rid of a problem where the ESP32 cannot reconnect after switching from Wi-Fi to BLE. Another thing to mention is power consumption. Although BLE should be low energy the ESP32 draws just as much power as when using Wi-Fi.  

It's still a fun feature that might be helpful to some people.

How do you add keyboard and mouse inputs to a tiny device? With a Gyroscope!

This new version brings some interesting new feature to the ESP32 handheld while also nearly doubling the standby battery life. 

Gyro Pointer

Since I started the project I always wondered how I could integrate some more precise controls like an analog stick. Because of size restrictions I had to settle for the 5-way SMT joystick. For the new revision of the device I first thought about adding a capacitive trackpad but I quickly abandoned the idea. There was just not going to be enough room on the PCB, even when moving the battery. A capacitive or resistive touchscreen is also not widely available for this screen size.
The approach that I went with instead is a movement sensor for controlling the device. As it turns out a gyro sensor can quite easily be used to emulate a mouse, just be tilting the PCB. This actually is not a novel concept. Many mobile games use gyro sensors for precise aiming, most notably "Ocarina of Time" for the 3DS. To implement this I picked the MPU6050 which is still one of the cheapest (3€) and most commonly available motion sensor. It does not only contain a 3-axis gyro but also an accelerometer and a DMP motion processor that can fuse all 6 axes together. 

In software making a mouse pointer move with the sensor is as simple as it gets. I just add the gyro value, which is the speed of rotation, to the pointer's x and y coordinates. This way the pointer moves with the device, regardless of its orientation. I added a button to the back of the device that can be used as a shoulder button to enable the gyro movement. When the button is released the device can be rotated without the pointer following, just like lifting up a computer mouse.
Having a controllable pointer enables lots of cool functionalities. I already added a few "tech-demos" to the firmware:

A simple drawing app lets you move the brush with the gyro and paint on the screen. I had to get creative with the actual pixel drawing to keep the frame rate high. With every frame the ESP32 saves the canvas area under the brush, then draws the brush and finally rewrites the cached area to the pixel buffer. This way, the entire canvas/pixel buffer doesn't have to be refreshed.

The gyro pointer can also be used to press buttons on a virtual keyboard. This means that any text can be entered into the device without the need for a physical keyboad. There are lots of possible applications for this, e.g. command lines and writing code directly on the device.

Better Standby Time

At less than 100µA the current draw of the device was already acceptable. But it was still bugging me that the voltage regulator wasted almost half the standby power with its 55µA quiescent current. Good regulators are hard to find if you are looking for high current in on mode and low Iq in a package that can be soldered by hand. Thanks to Hackaday.io user Jose Baars suggestion, I switched to the XC6220B33 which has an excellent quiescent current of 8µA while still supplying up to 1A to the ESP32. 

Power consumption:
- MPU6050 8µA
- ESP32 in Standby 10µA
- LCD: 10µA
- XC6220 Iq 8µA
- voltage divider 3.7µA
- Charge pump Iq 19µA 
-------------------------
theroretical total: 58.7µA
actual total: 56.2µA

With all parts combined and assembled the new version of the device draws around 56µA in standby. Even when considering that the display needs to be refreshed by the ULP core once per minute the standby time should exceed 200 days. There is still no accurate real time clock (I kind of forgot to add one), so synchronizing with NTP time every few hours costs precious power. The device should still run for more than two months easily.

Other improvements

Some minor changes include switching to a USB Type-C port. It is a bit more robust than Micro USB and a cable can be plugged in both ways. Because I didn't like the way the battery cable was running over the entire PCB I also moved the connector pads to the bottom. OSH-Parks have again done an amazing job with their After-Dark PCB. The silkscreen is so sharp that the QR code that I added can be scanned very easily.

While the battery can power the handheld for only a few hours, I wanted to include a standby mode that utilizes the low power display. This way the device would be usable as a clock and calendar. There are a few things necessary to keep the display active and the battery life long. Although the memory display does not need constant refreshing over SPI, a >1Hz clock signal must be supplied to its EXTCOMM pin. Otherwise the screen shows burn-in. This means that part of the ESP32 needs to remain active during standby mode.

At 70-200mA, the ESP32 can be power-hungry with its dual cores and WiFi modem. But not everyone knows that it has some nice power saving features. In sleep mode the ESP32's main cores are powered down, reducing the current to only 10µA. During this, the internal Real Time Clock controller is still active. The RTC itself can't do much, but it can wake the ESP32's third core, the ULP coprocessor. This Ultra Low Power processor can do basic tasks like checking sensors or reading and writing pin states while only needing ~150µA. Exactly what it need for the 1Hz signal. The ULP can be woken up periodically by the RTC using:

ulp_set_wakeup_period(0, 1000 * 1000);

For the 1Hz signal the ULP coprocessor has to be programmed in assembler. Luckily Espressif has an example for a blinking LED that does exactly what I wanted:

https://github.com/espressif/esp-iot-solution/blob/master/examples/ulp_examples/ulp_rtc_gpio/main/ulp/rtcio.S

The ESP32 can activated by different wakeup sources. For this project GPIO interrupts are needed as the device should wake up with the press of a button. The two external wakeups are ext0 and ext1. While ext0 can only be assigned to one pin, ext1 can be assigned to a map of different pins. There are a few things to consider when using the ULP core: Only the pins that can be accessed by the RTC can be used (RTC GPIO). Also I found that the internal pullup/pulldown setting are not reliable. I used hardware pulldowns for my design.

// This wakes the ESP32 with buttons on GPIO32 & GPIO33
#define BUTTON_PIN_BITMASK 0x300000000
esp_sleep_enable_ext1_wakeup(BUTTON_PIN_BITMASK,ESP_EXT1_WAKEUP_ANY_HIGH);

I want to update the time every full minute. This can't be done without waking the ESP32 for a short time:

esp_sleep_enable_timer_wakeup((60-now.tm_sec) * 1000000);

With the standby mode configured, the ES32 should only draw a few µA with short spikes when the ULP or the main cores are woken up. But there are other components on the board that need to be powered as well. The display itself draws another 10µA but also needs a 5V supply. As the LiPo battery is only at 3.7V a DC-DC converter is needed. After some searching around I found the MCP1640 from Microchip. It's a boost regulator with a quiescent current of just 19µA. The IC is small and only needs a couple of external parts. The 3.3V power supply for the ESP32 was a bigger problem. It turns out that it is hard to find a regulator with a high output current but small quiescent current, that can also be hand soldered. I ended up with the AP2112K, which can supply 600mA with a quiescent current of 55µA. 

Once I had the actual PCB, testing the real power consumption was also tricky. The shunt resistance for the µA-range on most multimeters is too high to power up the ESP32. And constantly switching between ranges or swapping wires was to tedious. A user on stackexchange came up with a clever solution (https://electronics.stackexchange.com/questions/340330/measure-wide-range-of-current-800-%c2%b5a-1-5-a/340353#340353): 

A diode in parallel to the multimeter will limit the voltage drop when the current is high. In sleep mode the diode barely lets current though. This way I was able to measure the power consumption in sleep mode conveniently with my 20$ multimeter. 

With the ESP32 in sleep mode the current is at around 97µA. 59µA are caused by the 3.3V power supply. 

These are links that I found useful:

https://www.youtube.com/watch?v=-QIcUTBB7Ww (Video about ULP programming by Andread Spiess)
https://lastminuteengineers.com/esp32-deep-sleep-wakeup-sources/

Earlier this year Panic, the software studio behind the Untitled Goose Game, introduced their Playdate console. It's a retro handheld device with a monochrome display and a crank as an input device: https://play.date/. I really liked the simplicity of it and decided to make something similar. It was also a chance to improve on my 2016 Chronio Smartwatch concept which features a similar display but only an ATmega328.

The display board is using the same 2.7" Sharp memory display as the Playdate. These displays only consume a few µAs and because of their in-pixel-memory they don't need constant refreshing. They are also readably under direct sunlight with no need for backlighting.
As its brains the board is using an ESP32. The dual core chip has enough processing power to drive the display while also featuring WiFi and Bluetooth. And it has a ULP co-processor for low power applications. I decided to include a small 5-way joystick and two buttons to enable simple games. As I wanted the device to work as a bare PCB without 3D-printed parts, a mechanical crank was out of the question.

The size of the needed PCB is defined by the display and buttons. With the display on the front, the ESP32 and all other components fit nicely on the back of the device. A milled slot enables the display flex cable to go through the PCB and into its connector.

I was able to reuse some of the parts and circuits from my ESP32 robot but it was challenging to keep the power consumption to a minimum. More on that in a separate project log.