I'm a C++ programmer (among other languages) so I set out to virtualize the ESP into various components like Sockets, MQTT Client and HTTP Server (to change the MQTT Client settings). My first project for the board was the ever popular NeoPixel Christmas Lights controller. I know, cliché, right? But hey, it's good for proving out various types of settings. I had settings for effects (i.e. blink, theater chase, etc.), appearance (i.e. colors) and effect enhancements (i.e. speed, pixel density, etc.).

Additionally, I created an Android app that was also an MQTT Client that would allow me to view the current settings of one or more boards (at one time I had 3 going) and allow it to alter those settings.

My MQTT approach was simple:

• Each client would use the ESP's Hostname as the Client ID
• The Client ID would also be used as the top branch of the MQTT Topic
• For each attribute I wanted to control, there were 2 MQTT subtopics, set and value
• The attribute was a subtopic of the set and value super-topics
• Upon startup, the board would connect to MQTT Server with a keepalive setting (10 seconds), set a 'retained' Will Topic of 'client_id/Active' with a value of 'No', and then finally publish a topic 'client_id/Active' with a value of 'Yes'. This way the Android could subscribe to the topic of '+/Active' and detect when a board came online and went offline.
• Each attribute that the board kept, after it established a connection/will topic, etc. with the MQTT Broker, would publish the attribute as a topic 'client_id/value/attribute'.
• The Android MQTT Client would publish the setting with the topic 'client_id/set/attribute' with the 'retain' so that the board, upon waking, connecting, etc., would subscribe to the topic 'client_id/set/#' to obtain the initial setting.
• Whenever a 'set' topic came in, the board would publish the corresponding 'value' topic (also 'retained') with the new setting to confirm to the Android app that it received it ok. Later, when I developed sensors, switches, etc. on the board, every change in the state would also publish a 'value' topic.
• The Android app would subscribe to 'client_id/value/#' topics to get the values and update the GUI.

I was largely successful in setting up my framework, but there were issues. They were the same issues everyone faces with these microcontrollers: memory (both RAM and Program) and speed. By the time I had the framework for the HTTP server and MQTT client, I had nothing left to include the application! Some will say that C++ chewed a lot of it up, but it really doesn't and I was judicious in my use of memory.

Another issue was the complexity of the IP protocol with just the AT command set of the ESP. First, socket programming is by its nature, asynchronous, but the nature of the AT command set is synchronous. What that means is that I had to create a complex state machine to keep track of the asynchronous traffic in and out of the sockets over the synchronous stream of the UART. You can't start sending data to a socket in the middle of receiving one or you'd get 'busy' from the ESP. Updating a string of 300 NeoPixels takes longer than 2 character transmits from the ESP and since interrupts were off during an update, you could miss data coming from the ESP. I fixed that by altering the NeoPixel code to check the interrupt bit on the UART and bail when it saw it, but still the complexity of parsing the UART stream and reacting to various possible data coming from the ESP was, in the end, just too much to bear. The parser was recursive (Note: recursion is bad when you have only 2K of RAM!) so that if a message from the ESP came in (i.e. +IPD... or +AT+CLOSE) in the middle of issuing a command and waiting for the response, it'd stop what it was working on, parse the message, issue the appropriate method on the Socket, and continue on with what it was last working on. When using a 'slider' on the Android app would cause a flurry of messages to come in (and respond to), all the while maintaining the keepalive messages to the MQTT Broker. It was a nightmare to debug!

In the end, I abandoned the approach in favor of a better one: move all that crap onto the ESP! That will be the subject of my next log.