Screvle - Lua Web IDE

After getting Lua up and running it was still rather inconvenient to develop applications. It required moving files to and from the SD card frequently only to discover a small typo kept it from running. Also, to get feedback from the Lua Runtime a serial port or some other way of communication was required. Time for something new...

I already developed a basic webserver, called microWebServer, so I started to extend it to support a json based REST API to store and run Lua applications. In its initial (simple) form, sending { "cmd":"run", "filename":"..." } to /lua/command started executing the referenced Lua script. I got some help with the actual web page designs (not my expertise) and many iterations later Screvle got the current intuitive, user-friendly webinterface shown in the screenshots and demonstrated in the video. Using the CodeMirror library, advanced features like code coloring and auto completion could be added.

Console Feedback

One issue remained, how was Screvle going to report back information and errors to the webinterface? Several options were investigates before settling for a long-polling scheme. In the future I hope to include WebSockets in microWebServer.

There would potentially be several (web-)clients connected to Screvle, but I only wanted to buffer the console information once to conserve memory. Additionally, because the data is polled, it is possible to lose data as it gets overwritten before it is read back and that needs to be detected by the webinterface (otherwise, how can you trust the console output?). This let me to implement a special circular buffer.

CBuffer is a circular buffer like we all know and used, but keeps track of the total amount of bytes written into the buffer. More specifically, it keeps an absolute value (in number of bytes) for the "tail" of the buffer. When the console first polls the buffer, the tail index is returned. Subsequent polls from JavaScript will include the tail value so the buffer knows which byte index was previously returned to this webclient. If there is more recent data available, that is returned, together with a new tail value. If there is no new data the connection is kept open for a certain timeout period before returning an empty message.

Based on the amount of data that is received, the previous tail value and the new tail value, the webinterface knows if any data was lost and reports this if it happened. That way the developer knows the console output is incomplete.

If for whatever reason the webinterface did not receive a reply from the console within a timeout the connection is considered to be lost and this is reported to the user by a red light. Next the webinterface automatically tries to restore the connection.

And so the Lua Development Environment was created and allows to write Lua applications from any browser without installing a single tool, driver or application. It even works from your smartphone or tablet. Screvle only requires power and a network connection. You don't even need to be near Screvle to develop, it can be done from across the world if you like. Very useful for remote installations.

While the Screvle firmware and hardware platform is very useful as a starting point to create the next tool/project, it remains an embedded system which requires some expertise and experience to use. For someone to start developing Screvle firmware you need to install an IDE, cross-compiler, JTAG/Flash utility, some drivers,... This was too cumbersome for quick proof of concepts and hacks. Ideally, Screvle would support "apps" that can be dynamically installed and executed.

By introducing Lua to the platform, I could do that (and more). Lua is an (under appreciated) scripting language written completely in C and designed to be embedded in other applications. Lua has little RAM and ROM requirements and the available Lua API can be easily extended to provide more functionality.

What followed next was a huge porting effort, making the C API I had created previously available to Lua. This includes the microGraphics Framework, Settings Framework, Peripheral Framework and many more. It is now possible to dynamically run Lua scripts stored on the SD-card. This means there is no longer a need to change the actual microcontroller firmware, so you no longer need a cross-compiler or JTAG/Flash utilities. The only things needed is a text-editor to write the Lua script, put it on the SD card and execute it from there.

Wait a minute... a scripting language on an embedded system... are you sure? Yes. Screvle hardware is many times more powerful platforms that are often used by the community (read: Arduino's) and some modifications were made to the Lua runtime to further reduce the overhead. Some of these modifications were originally developed by the eLua project ( http://www.eluaproject.net/ ).

 - Support for read-only tables that can be stored in Flash

 - Use 32-bit signed integers instead of double precision floating point numbers

 - Emergency garbage collector

How could this be made even easier? By using the build-in Lua Development Environment explained in the next project log.

After designing the first hardware revision of Screvle I started to create the software platform that would run on top of it. When another idea or project arises, the goal is to start working on the actual application from day one and not waste time writing drivers or other building blocks.

One challenge was having a flexible code tree that allowed me to quickly create a new project and keep older projects up to date with new revisions of generic code. This resulted in a directory tree where all software modules are stored in a separate directory and a project consists of a configuration file defining which modules to include. More details about how this works exactly will be explained in a later project log.

The initial firmware of Screvle roughly consists of the following components:

- microR2k Real Time Kernel

- microGraphics Framework (GUI)

- Settings Framework (stores settings on the filesystem)

- Peripheral Framework (makes using peripherals as easy as open/read/write)

- FatFs ( http://elm-chan.org/fsw/ff/00index_e.html )

- lwIP TCP/IP Stack ( http://savannah.nongnu.org/projects/lwip/ )

- microWebServer

- generic utilities ( string manipulation, button debouncing, circular buffers,... )

- peripheral drivers

- board specific configuration/initialization

The history of the Screvle project starts many years ago. As an embedded systems consultant and hobbyist I was often looking for an easy to use, open hardware platform to start from when building the next thing. There are our beloved Arduino's and mbed's, but there is still a lot missing before they are useable in some applications (e.g. buttons, display, ethernet, SD-card,...). When you put all the shields and boards together you end of with a nice proof of concept, but the result is not useable outside the lab (read: by non-engineers). Expensive custom PCB design was often needed.

Out of frustration of always re-inventing the wheel I started to create Screvle hardware. I wanted a modern CPU (ARM Cortex-M3, later upgraded to Cortex-M4), a decent display (QVGA + touchscreen), a few buttons, an LED, Ethernet, USB and an SD-card. And expansion headers with all types of embedded I/O to connect project specific hardware. Currently at revision 3 of the PCB, it now also comes with a Li-Ion charger and optional Wifi module.

The Screvle PCB is designed to fit in a commercially available enclosure (Hammond 1553C), so as a whole Screvle is a real product, not an engineering tool. This allows to take your hacks out of the lab into the real world for everyone to enjoy.