Better late then never: My official contest entry video is finally on youtube. This is a requirement for participating in the finals and the deadline is tomorrow morning. But now it's online on time - Please check it out and I hope you like it.
I've also double and triple checked that this project meets all the other requirements. So for the moment I can't do more than to keep my fingers crossed and hope for the best. Wish me good luck and thanks for your support.
A bit more than a week ago I talked about separating the API functionality from the USB code as well as integrating the new FAT16 implementation into the SolarCharger firmware. That is done and committed to github.
But there is more: I finally cut several hours of video that I took months ago. The result is two new videos showing how I'm building a SolarCharger and a user interface, respectively. They are relatively short, just a few minutes each, with lots of time lapse condensing hours of hard labour into hopefully joyful minutes on youtube.
Here's the first of the two. Watch a SolarCharger being built. No talk, just some comments in the subtitles.
The second one is somewhat more educational. I walk and talk you through the entire build of a user interface from applying the solder paste over placing the SMD components and reflow soldering to adding the through-hole components.
There's not much to show in terms of pictures but the last few weeks have seen intense work on the bootloader software. Most of the FAT16 implementation has been re-written, API functionality added, tested and debugged. It is now possible to write files to the flash drive with a minimum of write operations. This is possible by cleverly buffering data in RAM before performing an actual write on the flash. This means, each page only needs to be written once, even when sending data in small chunks. So the flash (which has a limited number of write operations) will not wear out even if the firmware is updated very frequently.
With that work behind me, I have once again turned to the acual SolarCharger firmware. A number of code files are shared between the bootloader and the charger software. That includes the FAT16 implementation mentioned above but also all the USB code and hardware drivers. The first step was to integrate them and make everything compile again. That's done. So the charger immediately profits from the work I did on the bootloader.
What remains to be done is to separate the API functionality from the USB code. The charger has its own API which previously has only been available via USB. Now I'm putting this into a separate module so that it can be accessed via any of the interfaces. Exactly the same thing I did for the bootloader a few weeks ago. Once that's done, it's automatically accessible via SPI as well - That code is shared with the bootloader and already in place. I expect this to be all up and running by the end of the week.
After this rather technical step I will resume to work on more visible features again. I'm very much looking forward to it. There are several things to be done:
PWM outputs: Since Rev E the hardware is capable of PWM control on the power outputs but the software to do that still needs to be written.
Explore further MPPT algorithms. I'm currently reading some papers on the topic but I'm looking for more material. If you know any good source of information, please let me know.
Make use of the 12-bit ADC for temperature measurements. As above, the hardwar can now do it but the software only uses 10 bits at the moment.
User interface: Some of the screens were really only intended for debugging. It's time to throw those out.
Low-power features: More needs to be done to reduce standby current consumption. There are many hardware features not fully utilized yet in that respect.
Debug and finalize data logging
So you see, I'm not running out of work. I have one of those chargers set up here in my office with a panel just outside the window and it's harvesting power every day. But there are always things that can and should be improved, particularly in software. I'm on it.
I finally managed to cut two short videos of the bootloader in action and to upload them to youtube. Here they are. I hope you enjoy them and let me know if you have suggestions - I'm relatively new to video.
After countless hours of writing and debuggin (mainly debugging ;-)) bootloader software I can finally program the device via its external SPI interface. The bootloader was originally a pure USB MSD (MassStorageDevice) bootloader. To make it more universal, particularly in an embedded context, the idea was to allow file access via an API which can then be made available via SPI, I2C as well as USB HID (HumanInterfaceDevice).
An important milestone on this journey has now been reached. One can now create, delete, rename and modify files via the API. Once a firmware file has been created in this manner, the original bootloader can do its job and program the new firmware.
Why did it take so long? Hard to tell, really. With hindsight, there was nothing too difficult about it. But as so often with embedded software, sometimes it takes days to get a seemingly simple piece of code to do what it should.
At an early stage, I reviewed and re-wrote a pice of code so many times trying to find out what's going wrong. I read the PIC data sheet back and forth thinking I must have overlooked something. In the end it turned out to be a bad soldering joint...
I've also archieved a massive performance increase in terms of time it takes to write the firmware. It's now impressively fast and only takes seconds - Something I very much appreciate during testing when I re-program the charger many times a day.
That said, some more work needs to be done to further improve the FAT16 implementation. It's still quite inefficient when writing files. I'll need to change that so the flash won't wear out over time. The flash chip I use has a guaranteed 100k write cycles which sounds like a lot but is quite easy to reach if your code is not careful.
It's 2:05 a.m. here so no photos for today but I hope to post a video of the bootloader in action tomorrow.
I've returned from two weeks of summer holidays on Sunday and have immediately resumed working on this project. Before I went on vacation I did quite some coding on the bootloader in order to get it ready to communicate to the outside world via SPI and I2C. But during all that time, I had no means of testing the software. All I did was to make sure my changes didn't break any existing functionality.
In order to test and debug those interfaces, I first needed another device using them. A RaspberryPi was an obvious candidate. I already had one but I haven't used it for a year or two so I first set it up with the latest version of Raspian and searched for a suitable library to handle the low-level details of the RaspberriPy's SPI and I2C ports.
When I was half my age, I used to work on Debian and that Linux experience proved valuable in the process. But my knowledge on the subject has rusted since and it took me a while. Finally, I soldered some wires with 100mil connectors to both the Raspy and the charger so they hopefully soon communicate via SPI. This setup allows me to quickly and easily connect and disconnect the two boards and also make it easy to connect a scope in between.
The scope is also set up and triggers on any SPI communication. The python library also seems to work just fine, at least I can configure and access any of the GPIO pins. So I can start now with the real work ;-)
I'm delighted to tell you that this project has been selected to run in the Hackaday Prize Finals on October 22nd. The last week I've been working on this project day and night. And I mean that quite literally: I was still soldering some more boards at 1:30 a.m. when the email with the happy news arrived.
There are three things I'm currently working on:
Getting some more chargers out into the field in order to get some feedback from real-world use. That's why I'm busy soldering. One charger is now constantly deployed at my home and another one is on its way to Norway. At least two more will go to Croatia where Valent is working with some schools that run them for educational purposes and experiment with different algorithms.
Finalizing the bootloader. I've spent many hours cleaning up the code and extending the API. The goal is to have a universal USB / SPI / I2C bootloader. It turned out to be a lot of work but I'm getting there.
Posting some videos: I'm not really up to speed when it comes to cutting videos I had to notice. I've recorded quite a few videos from outdoor testing, the bootloader in action as well as soldering. But I still have to cut and edit them somehow before they go on youtube.
I'll be on vacation for the next 2 weeks but after having been selected for the finals I'm more motivated than ever to finalize this project.
This project has just reached another important milestone: The USB bootloader is up and running. This part of the project will enable the non-technical end user to easily and reliably update the firmware in the field.
Unlike a USB HID (Human Interface Device) bootloader that requires some application to run on the host computer, this USB MSD (Mass Storage Device) bootloader requires absolutely nothing in terms of host software. It's entirely independent of the OS used. Windows, Linux, Mac, it all doesn't matter. As long as they can deal with a USB drive, they're good to go. Just copy the new software (in the form of an .hex file) to the Solar Charger drive and follow the instructions on the display.
It might even be that this bootloader is the world's first of its kind for the PIC18 platform. To be sure, this kind of bootloader has been around for years for more powerful 32-bit microcontrollers like ARM Cortex and the like. But in my online research I have been unable to find any other such project for the PIC18 family (or any other 8-bit microcontroller). So I had no choice than to write my own. If you know of any other implementation, please let me know.
Once the file has been found and the user has pressed the button, the file is checked. If all those checks pass, we can be confident that we have a valid hex file. Of course, it doesn't tell us anything about the quality of that code, that's an other issue. But technically we should be fine.
Once the checks have passed, the user is once again requested to press the push button to confirm that this file should be programmed onto the chip. While it's programming, it keeps displaying the current hex file entry it is processing to give the user an idea of the progress. It also keeps track of the number of flash pages it has written. One page corresponds to 1024 bytes on this architecture.
Once all the new code is flashed onto the chip, a message is displayed and the user is asked to once again press a key to re-boot the device into normal operating mode.
There are two different ways to enter bootloader mode. One is to press the push button at power-up. The other one consists in writing the value 0x94 (an arbitrarily chosen value) to the EEPROM address 0x100. In this case, the device will start up in bootloader mode no matter the state of the push button. The bootloader then overwrites this value (to 0x00) in order to start up normally next time.
After the reboot, you should be greeted by the startup screen of the solar charger firmware as shown above.
As much as I enjoy C programming, today the weather was just too nice to lock myself inside and stare at a computer screen. So I changed my schedule and did some real-life outdoor testing.
So I moved my 35 watt solar panel outside, grabed one of the Rev F solar chargers, connected a battery, my laptop and some more gear.
This is an ongoing project but it has been going on for almost 3 years so that the charger is mature enough to be deployed. Some software features are still missing and others are still a bit rough around the edges but it can easily be used as is.
As a hobbyist, I do most of my development and testing late at night when there tends to be little sun so this represented a rare opportunity for me to do some testing with a real solar panel as opposed to a lab supply.
As mentioned above, I only used a 35 watt panel which happens to be the only panel I have. So this test was not so much about pushing the charger to its limits but rather how usable it is at this stage of development and about how it performs in a more real-world setting.
The test was rather unspectacular but successful for that very reason. I set it up and let it do its thing. And it just worked. It never crashed or performed weirdly. It just harvested energy all day long. Typically around 20 watts which was what the panel was able to provide.
The solar charger app which I ran on my laptop also performed as it should. It worked reliably monitoring and controlling the charger without ever crashing. USB connection was never an issue. I plugged it in, started the app and it just worked.
The 45Ah (I think) battery was relatively empty when I started so the battery was charged at full current without overcharging ever being an issue. Besides that, I used some of the energy to charge my cell phone.
During most of the day, the charger sat in the sun while performing without getting overly hot. The on-board temperature stabilized around 45 degrees centigrade which can mainly be attributed to the sun, rather than losses in the charger.
Later, after the sun had long set, I used a stip of warm white LEDs to provide some light. Rarely has embedded engineering been such a pleasent task.
It's programming time once again. More specifically, I'm working on the USB bootloader. The bootloader is extremely user-friendly: When connected to a computer via USB, it behaves just like a memory stick or USB drive. You then just copy the new firmware hex file to that drive and confirm that you want to use it.
As you may have guessed from the photo above, the bootloader looks for a file named FIRMWARE.HEX. Once such a file is found, it will display its size and ask the user if this file should be used.
Once the button is pressed, the content of the file is checked. Is it a valid hex file? How many entries are in that file? Do the memory locations correspond to what we expect? Are all the check sums ok? There are a lot of checks to perform in order to make sure we have a valid file.
In order to perform all these tasks, the first step is to understand what data is on the drive. For that, I had to implement a simple, lightweight FAT16 implementation. Only so will the bootloader know when there is a suitable drive on the file and where to read its content from. That was quite a bit of work but I have learned a lot about the inner workings of that file system in the process.
All the steps described above are already implemented and somewhat tested. The software is able to parse, understand and verify the hex file. It is also able to write data to the internal flash but I still have to put the two pieces together.
Somehow I manage to hang the software when writing real code from the hex file to the flash. When I use the same routines to write dummy data, everything works. Anyway, I have to continue working on this...
Lukas Fässler
