Newt: Always On, Low Power Digital Assistant

I've launched a crowd sourcing campaign for Newt on Crowd Supply. As of this post. we're at 197% of target! The plan is to have Newts in the hands of backers by June 2022.

I'm maintaining a log of updates on that site - I won't double-post, but will provide links to the last three updates... for those not following over there:

• Dec 16 - We are Live (an overview of the Newt)
• Dec 22 - FAQ from Streaming sessions (answering questions on the Newt)
• Dec 29 - Details about the included software

As I mentioned in an earlier post, the Newt uses an external Real Time Clock (RTC), specifically, the Mirco Crystal RV-3028. Some people may wonder why an external RTC is needed for an ESP32s2, since those microcontrollers come with an internal RTC. Well, there are a few reasons:

Need to add a crystal for accuracy

The internal RTC looses seconds when a crystal is not used. I could have added a crystal to the PCB, but that would have (a) reduced the number of available pins and (b) made use with Arduino or Circuit Python slightly more complex (I wanted to create a device that was "accessible" to a novice developer).

The internal RTC can operate without a crystal, but it looses seconds. This loss is exacerbated when the device goes into deep sleep. I ran a test with my ESP32s2, comparing the internal RTC with the external RTC. The device sleeps for 60 seconds, prints the time from the internal and external RTC, then returns to sleep.

Comparison at start: 22:46:31.453 -> Sunday, March 14 2021 22:46:31

Comparison 23 hours later: 21:18:00.583 -> Monday, March 15 2021 21:47:18

As you can see, we lose 13 seconds in a day... that's a minute in week, and a 4-5 minutes over a month.

Of course, you can always sync the time - but that takes a wifi call. The Newt is built for low power, so we want to minimize any unnecessary wifi calls.

Timezone craziness

The other reason an external RTC - and specifically the RV-3028 - is useful in this build is around timezone management - especially when it comes to daylight saving time changes. The RV-3028 is able to store the UNIX time separate from Local time. For example:

The Unix time at this moment is 1633184251 (this represents the number of seconds since Jan 1, 1970 in UTC). The local time equivalent (for Dallas, TX) is Saturday Oct, 02 2021 09:17:31 GMT-0500 (Central Daylight Time).

Unfortunately, in 2 weeks, Dallas will be in Central Standard Time, and the offset from UTC will be 6 hours, not 5 hours. So if I hardcode an offset in my code, then my time will be off.

However, using the UNIX time with the ESP32 "setenv" command allows me to adjust the time based on the specific month/day/time. ESP32 use the POSIX format to define the timezone. Here is the format for Dallas, TX (America/Chicago): CST6CDT,M3.2.0,M11.1.0.

Here's the breakdown of the format (from IBM developer website):

• CST6CDT is the name of the time zone
• CST is the abbreviation used when DST is off
• 6 hours is the time difference from GMT
• CDT is the abbreviation used when DST is on
• ,M3 is the third month
• .2 is the second occurrence of the day in the month
• .0 is Sunday
• /2:00:00 is the time
• ,M11 is the eleventh month
• .1 is the first occurrence of the day in the month
• .0 is Sunday
• /2:00:00 is the time

So, in short, I can use the UNIX time from the RV-3028, calculate the local time with the ESP32 "setenv" command, then save that local time in the Local Time register of the RV-3028 - all without making a series of wifi calls!

So, I've been playing around with a lot of names for the smart display. I first called it the SmartSlate; then the SlateMate. I even toyed around with calling it the Tabula Rasa (latin for "clean slate" and more importantly, the name of one of my favorite Justice League episodes).

None of those names resonated with me. I wanted something simple, that was inspired by the device, but not a "on the nose" name. I started thinking through the characteristics of the device, and decided to focus on one of my expected use cases: sticking the display on a wall, mirror, fridge, dry erase, etc.

From there, newt just kinda stuck. I liked it for a few reasons:

• the word "newt" is short and sweet
• newts are cute
• "Newt" is also subtle nod to the Apple Newton, one of the first PDAs.

My smart display will have a number of time-based functions, so I knew that time management needed to be a key part of the solution. The best way for me to manage time, with minimal calls/updates from an NTP server, was to integrate a Real Time Clock (RTC).

RTCs are integrated circuits that - very simply - keep time. Hackaday did a write up last year on choosing the right RTC (https://hackaday.com/2020/09/30/choosing-the-right-rtc-for-your-project/). I used this guide as a starting point for my research.

I had four requirements for the RTC:

• Must operate at low power
• Must be able to set and trigger alarms
• Must be able to set and trigger timers
• Must have a clock out feature (needed for the Sharp Memory Display)

I pretty quickly zeroed in on MicroCrystal's RV-3028 and RV-8803. Both operate at low power, are extremely accurate, and come in small packages.

I was able to get samples of both from MicroCyrstal (huzzah for samples!!) and I got to work.

After a week of testing, I landed on the RV-3028, for the following reasons.

1. Insanely low power - 45nA!!!
2. Ability to set a periodic timer (interrupt every minute or second), countdown timer (e.g. countdown from 5 minutes), and alarms - AT THE SAME TIME
3. Separate clock out and interrupt pins
4. Separate Unix and "Local" time tracking

The last two points are the post important, so I'll go into more detail.

Clock Out and Interrupt Pins

The Sharp Memory display requires an "refresh" command every second or so. This can be done with hardware or software - and a HW refresh command was going be better from a battery perspective. I toyed with using the ESP32s ultra low processor as a refresh "engine", but decided against it because (a) I would lose a valuable GPIO and (b) programming the ULP is difficult to do with Arduino or MicroPython. 

So, with the RV-3028, I can use one pin to control my alarm/timer/countdown interrupts (each with separate registers, so that I can have all three running at the same time) and I can have a separate line for the a 1HZ pulse on the clockout pin. This ultimately means that my microcontroller can spend more time in deep sleep.

Separate UNIX and and "Local" time tracking

Unix time, or epoch time, is typically calculated as the number of seconds (or milli/micro seconds) that have passed since January 1, 1970 (UTC). The RTC allows you to store UNIX time (separate from local time). This allows me to have a "time" value that can be used for local representation (stored in a separate register). In short, I can store a UTC time and a local time. This is extremely important when it comes to Daylight Savings Time; twice a year (usually), at 2AM (usually), the local time needs "spring forward" or "fall back".

In the past, I've hardcoded very specific daylight savings offset rules for my projects. This works for an individual maker, but doesn't work for a potential commercial product (that could sit in any number of time zones with arbitrary rules). Fortunately, having the epoch, and using the  "setenv" functionality in the ESP32, gives me the ability to dynamically change the local time (based on DST or timezone) without complex logic or repeated calls to an NTP server.

Running BOM (@250 units)

I got a great suggestion about switching the physical buttons to touchpad, and finally got the PCBs a few weeks ago... they look beautiful. Now to assemble them!

Slow but steady progress... wrote some code for a Habit Calendar. I'm still not happy with the number of clicks needed to navigate. I might need to consider capacitive touch (for a better user experience).

Using an ESP device was a no-brainer for my design, as the ESP family of microcontrollers come with wifi (without a secondary chip). 

An ESP8266 wasn't an option, given the lack of pins and the security gaps. So, that left me with the ESP32 and the ESP32-S2. Both have plenty of IO and operate at low power. Cost is roughly the same as well.

I ultimately decided on the ESP32-S2 for a few reasons:

1. Native USB - which means no USB to Serial chip is needed for programing
2. Ability to run Circuit Python (checking another box in the "hacker friendly" category)
3. Lower power consumption during wake and wifi usage
4. Slightly lower cost

Running BOM (@250 units)

The largest power draws on an always on smart display are:

1. the display
2. the microcontroller 
3. the internet connection method

This Log entry focuses on the display

Given the desire for low power, TFT displays were out. The backlights would drain any battery in a matter of days.

OLED displays are better (from a power consumption perspective), but they still require 20 mA of power.... fine for a periodic display, but no good for what I wanted.

After looking around, there were really only two viable options

1. a eINK display
2. a Sharp Memory Display

E-Ink is all the rage these days. They can retain their screen image without any power (ultimate low power) and they aren't too expensive... but, there's one big drawback: the process of updating the screen - which can take 1-3 seconds and is somewhat jarring (here's a great video on the process). The sequence wouldn't allow a user to quickly toggle between screens, so eInk was a no-go.

That left the Sharp Memory Display as my only option. Sharp Displays are low power (the 2.7in version only requires 10-35 uA). They look as sharp as an OLED (without the current burden). They are relatively easy to use (SPI).

The drawback: they are  just plain ol' expensive... $20/each to purchase 250 (per Digikey).

Running BOM (@250 units)

In software systems design, we typically split requirements into two camps:

• Functional requirements - what a system should do ("display the current weather")
• Non-Functional requirements - how a system should perform ("data should be encrypted on transmit")

There is sometimes debate on what is Functional vs Non-Functional.... so I didn't bother categorizing my requirements. That being said, I want to be sure that my requirements were true requirements and not design decisions. For example, I need to be able to connect to the internet to get periodic weather updates, but that doesn't automatically mean that Wifi is a requirement.

Anyway, here's my initial list of requirements

Must Haves Functions

• Alarm
• Timer
• Calendar
• Habit Tracker
• Weather
• To Do List

Nice to Have Functions:

• Heath tracking
• To Do List
• Shopping List
• News

Stretch Goals

• Alexa integration
• Multi-user
• Music control

In addition, there were a few non functional requirements that I captured:

• Easily hackable
• Open Source
• Low Power (go 9-12 months between replacement or recharge of batteries)

I'll admit it: I'm lazy. I appreciate hard work... but I don't want to work hard when I don't have to (which is why I don't like eating crawfish... but that's another story). 

More simply put, I want information as quickly as possible with as little effort as possible. Let me explain:

When I wake-up, I usually like to understand the weather before I get dressed. To find the end of day temperature, I have to

1. Find my phone
2. Unlock it (either with facial recognition or a passcode)
3. Find the "weather" button in a sea of icons
4. Click the button
5. Scroll to find the weather at the end of the day

That's 15 seconds of work for 2 seconds of reading.

And through I love my Alexa devices (I have 10+ on my home), they aren't much better; it's a 20 second exchange to ask for the weather and have Alexa respond.

What I wanted was a device, centrally located, where I could, with a simple click, get key information (weather, calendar, etc.). 

So, I started looking into existing devices and found that nothing fit exactly what I was looking for.

The DakBoard is a great device, but it's not battery powered... which would restrict it's locations (I couldn't attach it to a bathroom mirror or a fridge door).

Adafruit's MagTag is another great alternative... and was the inspiration for my own design. My only issue with MagTag was the refresh cycle for the eInk display... which would limit the ability to quickly toggle between screens.

Finally, I looked at standard displays on Amazon... and I felt that they weren't as extensible as I wanted. 

So, that lead me to building my own... but before I could start, I needed to understand what the display would do.