Close

Mass o' Masochism

A project log for NYETduinoPlusLua

Wherein a Netduino Plus 2 is repurposed with alternative firmware based on Lua

ziggurat29ziggurat29 01/04/2018 at 16:110 Comments

Summary:

Continuing eLua toolchain setup masochism.  I got it mostly compiling, but there is another build tool challenge with 'romfs'.

Deets:

Continuing the self-flagellation with the build system masochism yesterday, my freshly healed wounds are ready for more abuse.  But armed with my newfound experiences, I find I have grown stronger in the broken places.

First I needed to get the Lua-based build system up, which entailed getting Lua 5.1.x up (the scripts as-is are not 5.2+ compatible), getting Microsoft Visual Studio installed (required for Lua Rocks when you have to build some modules from source, as we will here), and Lua Rocks.

1)  Visual Studio.  I installed the free 'community edition' VS 2017, and made sure all the C/C++ stuff was selected to be installed.  This is a long, slow process.  Oh, you also have to register some account -- don't know what it is for, but I just bound it to my usual dev email.

2)  Get Lua 5.1.x up.  This wound up being simpler than what I was doing yesterday, because it turns out that Lua Rocks comes packaged with a Lua 5.1 if you don't have a Lua of your own.  So that saved a step.

3)  Get Lua Rocks installed.  This was a little klunky -- certainly not the drop-kick of, say, nodejs and npm, but I was armed for combat after yesterday's boot camp training.

For the curious, setting up the Lua Rocks consisted of:

  1. start an administrative command prompt.  Yes, admin, you'll need it.
  2.  setup the Visual Studio variables, so the compiler can be used.  There is a batch file in the installation that will do this for you.  It's deep down, so I'll give you the path for my system, which is probably the same as yours, or only trivially different:
      "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build\vcvars32.bat"
    Realize that these vars only last for the current command prompt session; you'll have to invoke that each time you create a new command prompt, so don't close it before the next step.
  3. install Lua Rocks.  You can download it from the Internet - it's just a zip file of stuff.  Amongst that stuff is an 'install.bat'.  OK, you're going to need a few options.  Feel free to run 'install.bat /?' to see all the options, but spoiler is you need '/L /MSVC' which will tell it to install the included Lua (conveniently for our needs version 5.1), and also tell it 'trust me, use the Visual Studio'.  The install system tries to be clever, but for some reason it is not clever enough to understand VS2017.
  4. add some variables to the environment:
    To PATH, append 
       ';C:\Program Files (x86)\LuaRocks'
    To LUA_PATH, make it equal, or include 
       'C:\Program Files (x86)\LuaRocks\lua\?.lua;C:\Program Files (x86)\LuaRocks\lua\?\init.lua;C:\Program Files (x86)\LuaRocks\systree\share\lua\5.1\?.lua;C:\Program Files (x86)\LuaRocks\systree\share\lua\5.1\?\init.lua;;'
    note the trailing double semicolons!
    To LUA_CPATH, make it equal, or include 
       'C:\Program Files (x86)\LuaRocks\systree\lib\lua\5.1\?.dll;;'
    again, not the trailing double-semicolons!
  5. Fun:  since you have altered the environment variables spec, those changes will not be reflected into your current command prompt session.  You can either manually set them there, or you can close-and-reopen the command prompt.  If you do close-and-reopen, make sure once again that it is 'admin', and also set the Visual Studio environment variables as mentioned earlier.  Whee!
  6. Install the required rocks:
      luarocks install luafilesystem
      luarocks install lpack
      luarocks install md5
     
    The first and last will require Visual Studio to compile the C-side code, so you should see soem of that happening.  Once you've got these three rocks installed, your pre-requisites are set up.
  7. Note:  the Lua packaged with Lua Rocks is named 'Lua5.1.exe' so that's the name you must use to run it on the scripts, rather than the typical 'lua.exe'.

You won't need the admin console or Visual Studio for the next things, so you can close the current command prompt if you like.  (or not)

For my next amazing feat, we invoke the build system.  At this juncture, I'm just wanting to have it spew out the machine-generated header that I was having trouble with yesterday.  But there is a little bit more to do.

The build system, at a minimum, is invoked thusly (according to handy comments therein):
  'lua build_elua.lua board=MIZAR32'
which in our case translates to:
  'lua5.1.exe build_elua.lua board=MIZAR32'

So, we will need at a minimum 'board' specification, and indirectly a processor specification.  Rummaging through the source tree I found some buried treasure:

Hmm!  Someone has had this thinking before!  However, these are not the ones we need, we still have to make a new one for the Netduino Plus 2.  The first one is for an Atmel processor, and the second one is for an STM32, but a different chip, and different board definition.  The second is logically closer to what we need, so I copied that to 'netduinoplus2.lua', and made a couple edits.  I don't know what I'm doing yet, so I kept the edits to a minimum.

First, I know that the processor is wrong.  The netduino-2 uses an STM32F205, and we need a STM32F405.  There is not a chip support for the '405, but the '405 is similar to the '407', so I changed it to that:
  cpu = 'stm32f205rf',
Also, I changed the 'clocks' since I know this board runs at a higher rate:
  clocks = { external = 25000000, cpu = 120000000 }
Those are the only two things I dared to change yet, until I understand the other stuff more fully.

So, I invoke the build system (from the 'eLua' project base directory):
  lua5.1.exe build_elua.lua board=netduinoplus2
and, presto! it generates a header file, and then immediately fails to carry on because of lack of known toolchain.  I expected the later, so I am actually pleased with the progress.  (Later I may see if I can somehow wire in the System Workbench command-line tools; it is based on gcc by 'Linaro', and seems similar to the 'codesourcery' ones that the eLua build system understands.)

I pluck my freshly minted header from:
  elua\boards\headers\board_netduinoplus2.h
and put that into position in my System Workbench project, and clean, refresh, build (oh, if you haven't every used Eclipse before, you will become familiar with that sequence.  Eclipse tends to lose it's marbles easily, and give really weird compile errors.  This especially happens when adding/moving source around.  Too much caching!)

The build fails spectacularly in
  NYETduinoPlusLua\Src\elua\src\platform\stm32f4\platform_int.c
but at least not on missing the header!  Rummaging through that file, this seems to function at the edge between the eLua C code, and the chip-specific library code.

OK, time to kvetch a little (but not nearly as much as yesterday!).  The eLua project includes copies of old libraries.  As mentioned before the Lua is almost 10 years old (two significant api-changing releases behind), and also the 'fatfs' is old, the 'uip' is old, and the 'STM32 peripheral libraries' are old, old, old.  Also, I am not going to use the STM32 peripherals library, but rather the alternative 'HAL' libraries that are emitted by the STM32CubeMX and apparently are favored go-forward by STMicroelectronics.  (I actually like the older libraries better from a code quality standpoint, but the project that I will be migrating this work to will be using the new libraries, so that drives my decision).

So, back to platform_int.c errors.  I stubbed out the implementation of nearly every function in that module just to get it to compile so that I can carry on assessing what the damage looks like for the rest of the project.  To my delight, the rest of the project compiles up to a point.  In a way this makes sense if platform_int.c is the sole interface to the native chip support libraries, and I certainly hope this bears out to be true, because that will be happy news indeed:  less work == more play!  Live like a caveman!

The build stopped at a point in 'romfs', complaining about the absence of another header:
  #include "romfiles.h"
and later in that source file I see some missing symbols for:
  romfiles_fs

OK, so there is more build system generated code.  I guess when it croaked on the absent toolchain, it did not get to the step of creating that stuff.  I think I know what it is, though.  The eLua system supports a read-only image of a pile of Lua scripts that it builds into a bespoke filesystem, and converts that to literal C data for inclusion.  Now I will need to study how that tool works so I can invoke it manually, or else I will need to figure out the toolchain stuff sooner than later.

Next:

Figure out what to do about the 'romfs' image generation.

Discussions