-
Docs for apps + lsusb and script app!
04/20/2016 at 22:09 • 2 commentsNice, four Raspberries at my desktop running my system now! Mainly because I use basic features of the system a lot, but also invaluable for testing since all I do now is developing on pyLCI.
Added documentation for tvservice and system_info applications. I should add more documentation, it seems to be hella easy =) I also finished documenting output drivers yesterday, don't know why I've been putting this away for so long. Now it should be easier to configure output drivers in case you don't have a shield =)
I missed a day in my "app-a-day" approach. Granted, I was writing a driver yesterday, but it still don't feel good =) Wonder if I can make up for it today...
Okay, I made a simple lsusb app! So, I was at a hackathon, and I needed to see USB VID/PID of a device quickly to make some decisions. I needed to connect to my Raspberry through a wireless network using my mobile phone and use commands through SSH... Exactly what I'm trying to avoid in the first place, and it took precious time.
So, I made an app for that!
...OMG that sounds so sketchy =D
It was very simple to make, it just parses "lsusb" output. Doesn't signal if there's no lsusb, though. Guess I'll add that right now... just did and now it throws a message =) Oh, and docs.
I also have "run scripts" app on one of my Pis. I can make it run arbitrary scripts placed in the app's directory - as an app for today! Wait for it...
Yeah, pretty much done. As usual, I've got all kinds of fancy stuff - it can run system commands, scripts by relative and absolute path, as well as scripts placed in a separate directory. Has some pretty examples, too =) Lemme just make some docs about it...
Done! Guess I'll go to sleep now. At least I'll try =)
My workplace nowSo, three days - three apps. If it weren't for me not being very effective yesterday, they'd even be spread evenly =-)
Still to be done: GPIO hardware and drivers for Arduino. Couldn't finish those yesterday, fell asleep =( Guess tomorrow would be the best time...
I know! I want to make a clock app! Either tomorrow or the day after =) I'm not going to get invited to White House for it, but I'll have something to display time, and I'll also have something to stress-test displays =)
-
New hardware support&a step closer to autoconfig!
04/20/2016 at 00:16 • 1 commentYaay! I received some hardware today, namely, some HD44780 displays and a Pi shield with LCD&5 buttons! It's a shield with the same pinout as Adafruit one, works with Adafruit CharLCD_Plate examples and is generally very nice - @Craig Hissett, that's the one you were talking about! =) It costs 6$ on eBay, so for just 6$ you can get a pyLCI-compatible shield like this:
Also, by supporting it, I support the Adafruit shields, too, and I have hardware to check if it works or not =) I checked how it works, then went to write libraries, wondering about some design decisions Adafruit made while making this. After some time coding, failing and coding some more, it was done:
Rewriting the drivers from the ground was worth it. Now this thing doesn't depend on Adafruit libraries, and doesn't have all the overhead there could be if I were to use them. Guess PiFaceCAD libraries will face the same fate sooner or later =)
I made one step towards auto-config - that is, not needing to edit config.json if you're using a shield for which no config is typically needed! Now, config.sh automatically adds appropriate config.json contents if you have told it you're using a shield of those supported, so you can use the system straight away without editing any files! That's the way it has to be, and I'll be moving towards more and more simple pyLCI installation methods, even given how complicated it can be sometimes with some drivers.
Now - onto making docs, then - GPIO hardware I could easily test things with, and then making some Arduino-based I/O device drivers! Guess I'll keep my build logs shorter now, feel like it's better to post them early.
-
App-a-day - added tvservice app
04/19/2016 at 18:00 • 0 commentsFirst day of app-a-day! Being featured by Makezine is a great way to start a week like that ;-) Hope I can keep up and have 5 apps by Saturday.
I've got a tvservice app and it's now possible to issue basic tvservice commands from pyLCI! What does it mean? You can easily change HDMI output resolution on your Raspberry Pi without keyboard/command-line!What's there?
- Changing resolutions to those supported by display connected
- Turning display on/off
- Seeing tvservice status
What's missing?
- fbset call to change framebuffer resolution
- systemctl restart $DESKTOP_MANAGER.service call to restart X
- Changing NTSC/PAL modes
- Documentation
- Using custom modes
Some thoughts:
- I do need a dialog box UI element. Would fit well in "Do you want to restart X?" situation, as that's likely to require user input.
- I do need app making guidelines, but I haven't yet formulated them well enough to start writing.
Started an Arduino driver. However, I'm feeling sleepy and I need to assemble hardware to test it - and being sleepy & hardware don't mix well. Good night to me!
Oh, I forgot to post this build log =) Lemme fix it now... -
Having a break, "App-a-day" approach and other ideas.
04/13/2016 at 17:33 • 1 commentI now have 3 Raspberries using pyLCI on a regular basis!
First is a Raspberry Pi 3, with a HID numpad and I2C screen. I use it as a desktop, actually. It's very cool to have it, it's very fast and the experience so far is awesome. pyLCI on it is great for some quick settings, like switching volume and tracks in the music player - as well as shutting it down without having to touch the keyboard, enter my password or do anything else.
Second is a Raspberry Pi model A, it's #Raspberry Pi portable workbench(Project Christoph). It's an awesome way to employ my model A, and it does a good job. It uses a MCP23008 I2C LCD backpack and PCF8574 for buttons - with interrupts! It's slow though and the buttons so far are hard to press because of the wood layer I have on top of them for decorative purposes (you can see a photo of the panel I made in the pyLCI project pics). It's why I have an I2C detect app, it's been great for prototyping and debugging. There is a ton of other apps to be made for electronics, though.
Third is a battery-powered Raspberry Pi 2 with a PiFace Control and Display shield. It's very compact and I carry it around it a small box. It runs from 2 Li-ion batteries. I can connect it to a wireless network and connect to VNC from my Windows laptop, and then I can run apps that are Linux-only, use USB-UART adapters with fake ICs whtat don't work with Windows or generally just a workplace with a familiar environment and all necessary software&settings accessible from a spare computer. Once I get more familiar with my Pi3 and setup a work environment on it, I'll use a Pi3 for my work computer. I'll probably also need to get a better DC-DC for that. Oh, how I wish Raspberry Pi could just suspend-to-RAM!
I should make a GPIO-based LCD&button shield. I need this for my 4th board, OpenHAB Raspberry Pi room automation project, and I also don't have a way to test GPIO drivers right now - other than assembling everything on a breadboard from time to time. I've already got a piece of bredboard and some components, I just need to solder it all together =)
I am having a break this week because of multiple events happening, and my health is not that good lately. There's a hackathon on medical tech this week, Friday-Sunday, and it's going to be... Well, let's say I'm not really going for any serious projects - want to have some rest =) Maybe though, maybe I'll join some team which's gonna do "serious business". But generally, I don't wanna be stressed out by anything this week. I will need to be alert to answer questions at all the places my system goes in media,
Next week is going to generally be free from events I'd want to attend. This means it's occupied by pyLCI-related things - especially after people start finding out about it and try to install it. With that in mind - I'm gonna build apps. Plenty of them, thanks to the fact it's very easy to make them. And thinking about this, my old "app-a-day" plan comes to mind. It's a plan to take a list of necessary apps and write one app at a day. That means I can make 5 apps at a week, and have enough time to think about UI elements I should make. For example, there's this huge problem with input UI elements, even the most simple ones. I can design them, yeah. It's not very hard. The hard part is designing them so they're flexible enough for other applications, and still are easy to define and use. Guess that 5 different applications will let me think through it better, make some mockups and finally make a couple of elements which are best for their application.
I need more visibility. A good thing would be converting some of popular projects to pyLCI, which is literally "assemble project's hardware and write an app". I could even make a separate Hackaday.io project with all the projects I'm going to repeat, especially due to the fact that the process is going to be much, much shorter and I can make about one build log per project. The funniest thing is, since RPIs are quite powerful compared to how we use them, I can easily have one Raspberry Pi stuffed with all kinds of projects there are. Especially given that I have all kinds of additions, such as Pi Camera, some shields and tons of USB/SPI/I2C devices I can use for hardware, this sounds like a great idea.
If I look through my perfectionist glasses, I don't like the docs. Yeah, they're there, but they could be much more concise, especially on install&configuration matters. Also, ``config.sh`` should actually ask more questions and change config.json. It also should warn about I2C drivers not loaded... Maybe I should put that in the docs right now.
I also need apps that are more configurable. There's a read_config helper, it just helps with JSON and it's easy to use - and there's plenty of configuration variables in most apps that could be moved to config files. Now that I think about it, there should also be a document which'd describe best practices for writing applications.
-
20x4 displays tested, nested app folders and HW/SW plans
04/07/2016 at 20:44 • 0 commentsStarting another work session now. Ain't got much time, about 6-7 hours, and I don't feel all that energetic, but I hope I stil accomplish things today.
One more usage for my system. I wrote a simple script using curl to login to my university wireless network, and it only took me 2 minutes to hook it to pyLCI. It could be better written in Python, with HTML parsing and all that, but it's still very good. Actually, sounds like one more small universal utility to be made =) I guess there could be use for a "Ping/DNS check application"
So, I received a 20x4 HD44780 display and adjusted the config.json "output"."kwargs" to say "rows":4 and "cols":20. And guess what? It worked right away, with both Menu and Printer UI elements!I have to say though, I might need to optimise HD44780 library to avoid re-drawing the screen each time I scroll down in a menu - it's very noticeable on a 20x4. The library certainly needs to store the output data somewhere and do char-to-char comparisons. Overall, there's still a healthy chunk of things to optimise in hd44780.py, and, thankfully, it can be done without breaking the interface.
I also need to add output.type=["char"] definition in all current display objects. Forward compatibility, you know =) There's also a necessity to automatically turn off the backlight after the screen hasn't been updated for a while. Just one more simple thread and one flag to be added and it'll mostly sleep all the time, hardly impacting performance at all but heavily impacting power usage. Oh, I'd also probably need to check whether sending a noDisplay command impacts power consumption. I run things from batteries, after all.
Main menu system needs improvement. It's not nested, there's yet no way to categorize elements and it already makes hell of a clutter. I'm talking about re-making the main menu alike to menus like this:
|Main menu |-System apps |--System info app |--Shutdown&reboot app |-Media apps |--MOCP control app |--Volume control app |-Electronics |--I2C toolkit app |--EEPROM read/write app |-Networking apps |--Bluetooth app |--Wireless app |--Network interface app
I'd be damned if those folders weren't infinitely nestable and would require a configuration file, and I've figured it out - it could easily use __init__.py in a clever way. I could even specify how to order apps in a menu using same __init__.py! It's a problem right now because, well, they are ordered chaotically. When you think about it, any order of applications other than "user-preferred" is to be perceived as chaotic.
That's the first thing to start with. Other priorities:
First of all, I've got an unfinished "System info" application, I worked on it yesterday. It's already shipped with pyLCI and can show you uptime, loadavg and memory stats both for x86 and ARM Linux systems. It can detect if it's run on RPi too - I plan to make some custom menu entries concerning Raspberry Pi-exclusive features =)
Second thing is eliminating and debugging all kinds of LCD redraw problems. When the display is re-drawn too often, it might start showing garbage, either recovering in a couple more refreshes or never recovering. AFAIK it might miss/misdetect a nibble once in a while and fuck the "byte-by-4-bits" part up completely, it might be a race condition between threads... There's plenty of things to test.
I also found a 24x2 display at home. Surprisingly though, it doesn't work with pyLCI. It doesn't even show anything, it's just stuck at "first line filled with white blocks" part when I init it, even when launching a driver directly. That's quite a problem, this display is cool - especially given it's a piece of hardware excellent for testing for some well-hidden bugs. I'll try to wire it to a different I2C expander now, as a start to my tinkering sesson. If it won't work, I'll put it away for a while until I can research it further.
So, I'll start working. First, the display...
Nah. It shows a couple of blocks, but that, maybe the pinout is not right? Or, maybe the RS line is inverted. I have this feeling because, well, it shows 3 zeros and it gets sent 0000;0011 three times during the init sequence - but as a command. Seriously, look at it once again!...Nah. Didn't do shit. I guess I need to make a "display testing stand" for displays like this one, which would use 8-bit and 4-bit modes, try to read busy flags and so on. Maybe it'd be possible to swap data lines, use different headers and header pinouts, negative voltages for backlight... Wonder if it'd pay off.
Hmm, zeros... Maybe it cannot into 4-bit? But still, fail. Moving on...Fail. I'm getting distracted by the display, it still feels like a challenge =D
Screw this display......Okay. I went as far as to have detection of arbitrarily nested subdirs and modules in them, except for subdirs nested in the module dirs - it doesn't make sense to use them. See, every subdir of "./apps" represents a Menu object, and, as a subdir can contain both other subdirs and module dirs, its contents are either calls to Menu objects created by subdirs or application callbacks. I yet have no idea how to make it so that it's logical and expandable. Maybe return something like contents of os.walk? Sounds good, and, hey, Python devs didn't have this function with such weird output just for fun, there's definitely something to learn from it.
Aaaand... Done. Made it "walk". Now just let me test it with subdirs...
('./apps/ee_apps/ee_apps', [], ['i2ctools']) ('./apps/ee_apps', ['ee_apps'], ['i2ctools']) ('./apps/media_apps', [], ['mocp', 'volume']) ('./apps/system_apps', [], ['system', 'shutdown']) ('./apps/network_apps', [], ['network', 'wpa_cli']) ('./apps', ['ee_apps', 'media_apps', 'system_apps', 'network_apps'], ['test', 'skeleton'])
Yep, it behaves like os.walk, only that I omit anything that's not a module or a subdir. Now, let's see if my intuition is right about telling me that behaving like os.walk is the best solution...Also, I have success making apps to load with this technique =) Creating menus is a problem, though. I need to think it through better...
And done. Behaving like os.walk was *the* best way, indeed. I needed to iterate over app_walk output once, then over all the detected subdirs to build a tree of them and them over all the apps to link them to correct subdirs. That's overhead of looping through things 3 times, yeah, but I think it can be avoided if necessary - and it happens only once, at system launch, and hasn't slowed down things any significantly. Single-app mode is working - moreover, it's working with paths, and shell autocomplete! And, yeah, arbitrarily nested folders, of course.
Only thing left before merging this in the main branch is documenting all of the added stuff, making sure nothing's overlooked and generally making this exception-less.
I have an idea for adding hardware. I could just add support for many GPIO-based LCD&button shields, as well as some I2C-based, test the code (adding mockups with I2C) and release it as "supported&untested", then talk through the issues with all the people that have those shields and decide to use them. GPIO is, well, easy to map for different things, so as long as I have one tested GPIO setup, I basically have them all supported - but unless I list GPIO-based shields as "supported", nobody's gonna think about it. Also, I got an MCP23017 coming my way - in case of modules using that, which is plenty of them, I can easily make a breadboard circuit and it'll be no different from the hardware. Then, I can just wait until I have more hardware =) I have 3 different Pis sporting my interface right now, and it's very good so far, but obviously not enough for the "hardware support" page.
PiFaceCAD module needs a rewrite. It's got its own library with its own quirks, and it's one more thing to think about - which ain't so great considering it's, once again, just an MCP23017 on a shield, nothing special. The library will need to be scanned for workarounds for problems discovered by whoever made the library, though, I won't discard such a source of knowledge =) As for now, though, it's not a priority as long as it works correctly.With that in mind, I'll take a short break for the weekend. As soon as I finish my "system info" app and document all the changes I've made =) I hope it won't be another build log that's stretched over two or three days, like this one - not that it's a bad thing, I just don't have that much time.
-
Release announcement!
04/04/2016 at 01:06 • 0 commentsHi!
Today I want to finally show you the project I'm working on for about 2 years and release the first ready-to-use version. This project is pyLCI - Python-based Linux Control Interface, an external interface for Linux computers which allows to interact with the system using character displays. You can write applications for this interface which will allow you to change your system's settings, control different desktop and console applications, as well as launch scripts and execute commands - and all of that without depending on monitor&keyboard!
My interface is:
- Cheap
- Simple
- Easily extensible
- Universal
Interested? Read on.I've been feeling lack of proper headless interfaces for a long time while using Linux systems. We've been using simple menu-driven interfaces on mobile phones for ages, but I sometimes need to plug in a UART adapter to understand, why my Raspberry Pi home automation hub won't do some specific task! Last time I checked, it was because I swapped 2 identical boards and eth1 changed to eth0 - which wouldn't get a network address because it wasn't in /etc/network/interfaces. Adding a screen and some buttons to at least be able to check on those situations would cost about 5-10$, but there also needs to be a software side...
Here comes pyLCI.
What can pyLCI be used for?
- Connecting to wireless and wired networks, getting information about network connections and their settings.
- Connecting Bluetooth devices
- Controlling media players and volume
- Scanning/printing documents, taking pictures and videos using a camera
- Controlling running services
- Mounting/unmounting partitions and backups
- GPS and navigation
- Pentesting
What can you use pyLCI with?
- Raspberry Pi and other SBCs. That's the thing it was originally developed for, and it fits perfectly. Want to connect to a wireless network? Sure. Want to shutdown it to avoid SD card corruptions? Same. Launch a script to fix something? Easy thing. Can't connect your Bluetooth keyboard? You don't need to search for and plug in a wired one to reconnect it.
- Home servers and routers. Want to see DHCP addresses currently leased? Want to turn off WiFi or maybe block a specific site? Or maybe you want to mount a flash drive and launch a backup script? You can achieve all of that without using the web interface from another computer.
- HTPCs. Got tired of this specific track and want to switch to the next? Or maybe the volume is too high? Or, maybe, you want to connect your Bluetooth speakers.
- Desktop computer. Especially if it's a Raspberry desktop =) Even if not, it's actually a pretty good addition to the system tray.
- Linux tablets and laptops... Especially DIY ones.
This month, I decided to make pyLCI more accessible to all the other Linux users. As a result, I'm releasing v1.0!
What can it do at the moment?
- Connect to wireless networks (known secure and any open networks)
- List network interface IP/MAC addresses
- Adjust your volume
- Change tracks in my music player (I use MOCP, but it's easy to adjust)
- Scan I2C bus for devices
- Shutdown and reboot
The list is not big yet, but it's very easy to add applications. pyLCI is a framework that is easily extensible by Python-written applications to add your own functions. These applications get access to the hardware abstraction layer, which can be used to communicate with the user, as well as a set of UI elements to allow focusing on the application's goal instead of things like hardware quirks or nested menu interaction logic.
What hardware do you need to start using it?
First, a HD44780-compatible display, like this one:
Easy to come by in various "starter kits", and it's super cheap.
Also, you need buttons. Like these ones:
5 buttons is the lower limit, feel free to use more. You can even use this:
To clarify, my interface now supports GPIO-connected buttons and displays (for Raspberry Pi), I2C->GPIO expanders and USB HID keyboards&keypads. Additionally, I support PiFace Control and Display Raspberry Pi shields.To appear soon:
- Support of Arduino boards with LCD&button shields, to be connected over USB. All that's necessary is simple Arduino firmware and a corresponding driver - which are hella easy to write. That'll make a very simple peripheral.
- A wireless device that'd have buttons&LCD and allow you to connect to pyLCI over WiFi (most likely, using ESP8266).
Where to start?
Take a look at the documentation page, as well as the installation instructions.
Want to develop your own app? I have a small crash course for that, as well as already written applications for code examples.
I had a last-minute project name recently ;-)
Some temporary shortcomings:
- pyLCI is a single process, including all the loaded applications. That means that, for an average user, it's much more simple to run pyLCI as root than to set up all the necessary permissions&groups for things to work. It's not a huge flaw per se, because a user sufficiently privileged to change various system settings isn't that far from root in terms of damage which can be done. However, this will eventually be resolved in future versions.
- My install scripts are currently developed for and tested under Debian/Raspbian only. If anybody could help with adjusting them for other distributions, I'd be very grateful for that. Alternatively, open an issue on GitHub if you want to run them on other distributions but lack the skill necessary, I'll do my best to help you =)
- I don't have a sysvinit script yet, the system is systemd-only. If you have a tested skeleton script suitable for this, please do send it my way, I'll be happy to incorporate it.
- If you give pyLCI a USB keyboard/numpad, it grabs it completely. I'll soon work on a driver that grabs only a part of keyboard, leaving all the other keys accessible for normal operation =)
Guess that's all for today's announcement. I'm very interested in your opinion about my system, as well as your thoughts about how you'd want to use it for your Linux devices. Here you can check out my plans for system's future, as well as a list of applications soon to be added to the system. Leave comments at my project page and don't be shy to open GitHub issues if you encounter problems on your way.
-
Yaaay, public docs!
04/02/2016 at 08:23 • 0 commentshttp://pylci.readthedocs.org/en/latest/
I've missed the release date =( I've spent 2 days not feeling so great, so my productivity has been drastically reduced. Regardless, I'm all good now and continuing, although I've missed the release date To this day I've been polishing all kinds of things for the release. But mainly, docs. Docs are great and necessary. And RTD is awesome.
I've taken my time to document things. So far, everything's been great. I still need output modules documented, as well as some small aspects, such as a skeleton app, but it's a huge step. =) Also, I need to autodoc more of my code. Guess I'll finally learn to write docstrings and stuff.
Tomorrow, I'm sitting at home and writing promotional materials in Russian and English. I'm also trying to cold-install my system on various Raspberry Pi with clean OSes, as well as using all of my input/output devices I support now on different systems - just to see how it works, I've already fixed a fair share of bugs that way. That way, I can release it on Monday.
Time to unpack my Pi3! I'll be setting it up as my desktop replacement very soon, and I'll sure have pyLCI-capable devices. Wireless and BT is a really nice touch. Wireless will work out-of-the-box with pyLCI and latest distro... Not so much with Bluetooth. I'm yet to find a good API.
Applications... My system will need plenty of those. I like how I've written the wireless application. It's quite flexible and capable, all that holds it back is lack of input UI elements.
I just hope that there'll be enough early adopters. Guess I need to check on my feedback channels - is there enough info to contact me and is GitHub link in the descriptions?
I've discovered problems with those HD44780-compatible character LCDs. Basically, possible concurrency, D0-D3 line noise, EN line ringing - more about that here. Sometimes they just start displaying trash characters. Has to do with slow displays, threading, hardware... Sucks. Something needs to be made more atomic, something - more slow and something - more protected. Guess it'll soon be time to get my logic analyzer hooked up to those data lines to see what's up. Not that it's not good enough for launching it, but it's definitely not good enough for long-term with those problems.
-
A wireless app, Printer UI element, input system and docs!
03/28/2016 at 05:10 • 0 commentsSo, yesterday was productive. I stocked up on Cola&potato chips, played some games to unwind and started working on a wireless connectivity application.
I've been using wicd suite for my Linux devices, a lot. It sucked here and there, but it wasn't something extraordinary. Now I've moved on to wpa_cli because that's the option Raspbian currently ships with and I can assure you: Wicd sucks.Now, DBus is cool. Daemons are cool, too. Python is cool. What's not cool:
- Crashes I was experiencing once in a while, with wicd-curses in particular. Shame, shame.
- Lack of documented DBus API - I needed to read through wicd_cli to get an example together, and it ain't as easy as it sounds.
- Its inability to work before somebody has logged in on the system - which is exactly the case for pyLCI on many occasions. Users don't notice that, but it's critical for my system. Has that got something to do with DBus, I wonder?
- Its inability to work with different wireless interfaces it wasn't set up to work with.
There are more, I won't go further. I'm waiting for 2.0 because, hey, the idea is still cool, but I wonder if the implementation is going to be good.
What else? Wpa_supplicant is great. It's great for scripting, too. Unlike bluetoothctl, say - that's why there's no Bluetooth app in pyLCI yet, doesn't seem to be easy to do it properly!
Wpa_cli is wonderful for scripting. It was also easy to parse through its output. I had a library for it in, like, a couple of hours. This library can even act as a standalone Python-based wpa_cli interface. I think my favourite part was writing that library in 250 lines, then shrinking those to 180 much more beautiful lines while adding plenty of things =) When I got finished having fun, I wrote a pyLCS wrapper application. Now there's a wrapper (pyLCS) around a wrapper (my library) around a wrapper (wpa_cli) around a wrapper(wpa_supplicant) around a wrapper(wireless driver) around a wrapper(ioctls and stuff)... Sorry, couldn't resist. Maybe wrappers are all operating systems are around. (Nevermind, they also help calculate things sometimes... Or do they wrap around CPU capabilities?)
Now is time for last minute changes that I won't be able to do after v1.0 - such as changing init arguments for things, or finally changing the project's name. It's pyLCI now, you can see that, and if you see something else somewhere, do notify me =) I changed arguments for Menu to more intuitive input, output, instead of output, input - you know, that's not how I'm used to recount these.
...
Right now I'm opening input.py file. I'm quite excited. I've still got a video to complete and stuff, but this is also necessary. Or it isn't, whatever. I have to get rid of it as soon as possible, the sooner, the better.nano input.py
...Good news. It was easy and rewarding. Bad news. There's a loop somewhere in new input system which is a CPU hog. ... Actually, never mind. I understood the reason while I was writing this sentence and fixed it.
Also, I cleaned up the output.py file. I didn't refactor it, just deleted a lot of code I planned to use that I'm sure I won't. This code is long implemented in v2.0 and already planned to be refactored =)
I added a Printer UI element. Capable of both printing strings and lists of strings, it's quite valuable. You can also skip something by pressing Enter on your input device =)
I started my documentation with Sphinx and autodoc. It was easy and I'm now writing docstrings, a lot of them =) Already documented existing input modules and UI elements.
Sorry, I just finished documenting input modules and understood I'm feeling sick and terribly sleepy. I'll go to sleep, right now. Making this log more verbose, testing code and pushing to GitHub is first thing when I wake up =)
Woke up. Feeling better, so I can continue working =) Talking about documentation, it's complicated. My main modules don't depend on any custom Python libraries, but my input modules, for example, do. The problem is that I'm using autodoc for these, and planning to push everything to ReadTheDocs so that, you know, the documentation is actually available to others. ReadTheDoc builds my docs on its own, however, so it won't be able to import all kinds of Python modules I'm using since many of them are using C libraries - such as evdev. That might take more time than I have, I'm afraid. Will se how this can be worked around.
I don't remember anything else worth recording in the log. Not that the things described before aren't enough. It seems I didn't take any pictures. (Of me typing on my keyboard?) I did, however, make a first promotional video for my system and will start processing files from the camera right now =)
Starting the next day and the next build log. What's now? Well, video. Also, I have to document output devices and possibly make a HD44780 lib of some sorts for them where I can describe init sequences and commands. Not to mention I need GPIO-connected LCD driver made!
Also, I need an ability to set custom key mappings to input devices - will probably make a base class of some sort and make an "mapping_file" argument so that you can have multiple driver instances running for different devices and combine all of their key in your application =) Then, maybe some simple apps (not Bluetooth though, that needs at least a spare day, I don't think that'll be released till 1st.). For example, I can write an app to manage systemd units and services, wouldn't that be good? If I have enough time today, I'll also make 'evdev' input devices and test my system fully on an another Raspberry with PiFaceCAD, as well as with MCP23008 output driver.
I have no extra days left. There seems to be more and more to be done before launch, I just don't know. On top of that, I still need to write promotional materials. Sucks. Hope I manage to do it in time. -
More refactoring, testing and new apps!
03/26/2016 at 03:56 • 0 commentsToday and yesterday were quite, quite productive. Long story short, I stayed at home all the time, coding, soldering and writing. Tomorrow and the day after are really likely to be the same.
The day before yesterday all I did was go to my friend who has a laser cutter and get a new case prototyped for #Raspberry Pi portable workbench (Project Christoph). It's going to be one of projects which will show how pyLCS can be used, and I'm going to make it interesting.
On a side note, it's not really a control system. I'm thinking of a last-minute-change to pyLCI, which is Linux Control Interface, reflecting the true meaning more.Yesterday, I mainly worked on "automagic apps", that is - easily adding apps and managing them, as well as wrote some misc apps, such as I2Ctools and Volume control. Nothing too spectacular, most time was invested in assembling a new faceplate for Christoph, which contains both the screen and buttons. IMO looks sweet, can't wait until I get to test it. Maybe the day after tomorrow, after some hot glue jobs to keep the newly-cut case together till I find the proper screws =)
Today was a good day. I made a network app, which shows network interfaces and their addresses. All that's left is some common actions, like up/down/so on. Wireless is not yet done. Maybe tomorrow, together with Bluetooth... about that.
I also researched about what needs to be done to add Bluetooth. I think for PoC I'm going to use bluetoothctl and pexpect, while kludgy, this is the best solution to not spending a week on it when I just need something that works. I spent plenty of time on Bluetooth today, tomorrow's definitely the time to get it working.
Also, I refactored the Menu UI element, it now supports different screen sizes, different meny element heights&element content types, and, as a side effect, has scrolling implemented just the way it's in all those menus we love. Different screen sizes are yet to be tested, I only have 2-row ones =) I spent a lot of time on that, it seemed to be necessary for networking app to be a little bit more fabulous.UI elements seem to be an easy target to be covered with tests, so I might do that as soon as first category is finished. I won't refactor input subsystem tomorrow, but I'm certainly going to do it in the next couple of days. A lot of code to get rid of. While I'll be at it, it'll need its callbacks to be exception-proofed since that's the main cause of system crashes now.
I should make READMEs about apps, their capabilities and requirements. .md-style will certainly do, and they'll also look nice on GitHub. Oh, nice features - I've added a skeleton app as an example! I've also aded single-app mode, when you do "python main.py -a app_name" and it loads only that app, which is very cool for testing and also maybe will be handy for somebody's setup.
Videos... I need to shoot some ASAP, but I'm lacking apps. Should I make some mockups? Won't be fair maybe, but I need videos.
-
First day of refactoring&testing
03/22/2016 at 08:15 • 0 commentsThis is a project I've been working on for some time, but with long interruption periods. However, this project is great and it's worth working on, and Hackaday Prize provides me with inspiration, motivation and a deadline - and I don't even know which of them is the most important factor, they all are.
With that in mind, my intentions are to release the system in the wild for testing by any interested users so that I can receive valuable feedback and see what's there to improve, as well to spread the word about it so that more people think about such an interface and therefore see which problems it can solve for them. Given the contest time constraints, it seems to me that the best idea is to release minimal runnable version which supports some basic and most necessary functions, as well as all the hardware that I currently have written drivers for. I'll designate that version to be 1.0 - in a nutshell, that'll already be useful and sufficient for many users. From that point, even if I become unable/unwilling to work on the project (highly unlikely, but still), the idea and basic working code will be available for users and developers among them, so that the project will be able to go on if there are enough concerned people.Here you can read more condensed info about V1.0 if you're interested.
After a break, read my thoughts about the current system state and what I've done this week, up to this day.
There's next version of the system I've been working on for a long time, or rather what's planned to be the next version of the system. It has got many important points right, but is also wrong in some crucial aspects and would need much more time than is available until my release date. (That time also includes writing some promotional materials!) Telling about the next version would need much more time than I have, so let's just skip this until later.With all this in mind, today I've sat down and re-designed the most basic parts of the system.
First of all, the system consists of:
- output and input devices
- applications, which provide functions the end user wants to perform using the system
- core, which ties the applications and IO together
1. IO - hardware and drivers.
Till this day, I've worked on using 1$-a-piece I2C IO expander boards for interfacing with buttons, which is not the designed function because these boards are designed for HD44780-based LCD displays. However, the idea worked great and I've written an Instructable on it which I need to revise a bit before I am sure it's good enough to be published. I've also refactored the code which was driving such a backpack which was used, as intended, with an LCD, so now it's cleaner and also works better in some aspects.
Also, I've added GPIO-driven buttons, working with RPi.GPIO library - it's important because it drives the hardware requirements down along with minimal price, and I will add a GPIO-driven LCD code this week for same reasons. Can't get any cheaper and more basic than 4$ and wires, I think. Hey, it's a perfect project for Pi Zero! ;-) I yet need to test the system with PiFaceCAD - it's been quite some time since that combo was used. However, it should work great once I include a couple of fixes for problems I found out about this week.
One great thing - writing drivers for input&output is hella easy. However, there are things to be done as right now there's an evdev layer between input devices and pyLCS input interfaces, which in most cases is not necessary. Once I remove it, I think it'll remove a significant obstacle for running pyLCS on OpenWRT, removing python-evdev requirement, as well as simplify the code and make it much more fast and less bloated.
That feeling when you confuse variables and then stare blankly at things which seem to have no reason not to work
2. Applications
This is the hardest part, I think. There's a lot of compatibilities to keep in mind and a lot of things to be known about different Linux environments to make applications which work reliably for most of people, if not for everybody. I can't really reach "everybody happy" goal with applications more complicated than a couple of external command calls, so I need to make my applications understandable and system to add them in a way as simple as possible. I'll be describing applications one-by-one as I make them, as well the decisions that drive me.
The simplest approach which fits my requirements is simply unpacking applications in designated separate folders, then having the system scan those and import them all automagically so that application installation is at most a simple "tar xvzf file&&./update.sh", the last script just syncing the working directory with the system's distribution of the pyLCS... More about pyLCS&distribution relationship later.
No packaging of files is planned in this system, but none seems necessary as well. I've used that approach in two big projects of mine, and I know what works and what doesn't, so I figure the "unpacking-tarballed-apps" approach is more than enough for what's required and really, really simple.
Of course, that means dynamic location of applications in different folders. Those folders, however, are all to be located in a single folder - which might seem to be problematic with a big count of applications, but I don't think there are going to be THAT many so as to for it to become a problem.
Testing the applications is a great thing, and we need to make it easy. My first step towards it is going to be a command-line argument for the main.py script (the part which launches the system), so that you could specify an application name and it'd load that and that application only, skipping other applications, menus and all that fuss, which takes precious debugging time. I'm not yet thinking of any more testing helpers, but of course I'm going to automate my problem-solving, and it's going to be open-source.
As a proof-of-concept, today I've designed a simple system shutdown&reboot application and hard-coded it in the system as the one and only application to be detected (there's one more menu entry in the code because there's a bug fix yet to be backported from the next version of system I've been working on.) It works great and I fixed some more bugs in other parts of the system due to the testing. With this application done, I'm going to design the auto-detection part, remove the hard-coded parts and move on to writing more applications for the release.
Hey, I've made some exception handlers! This one is good for SyntaxError!3. System core
First of all, I'm making the core parts (such as main.py) output diagnostic messages to the screen used as pyLCS output so that even without launching the script it can be seen which part of it failed and whether applications or something else are to blame. Also, I'm making things more exception-proof so that one application crashing doesn't crash the entire system but merely gets that application unloaded (I've done it for the next version already, but for this one I yet have to try.) Of course, that means making a boot log of applications that fail to load and showing it on the screen while booting pyLCS, as well as making time limits for the applications to limit freezes (now for this one I have no idea, it just seem to be the best solution imaginable), or, alternatively, make guidelines for applications and make it so at least it's visible which one of applications is blocking the pyLCS boot process. Anyway, I've got plenty of ideas to be taken from the second LCS version, with the exception that the second version is much more asynchronous when it comes to working with applications.
Second thing is that core has to be small and fast. Excluding evdev from the input device management is certainly a thing that has to be done. Once again, I'm not sure it fits in the release - and it works great, apart from the bloatedness. Of course, I removed plenty of junk code and what once was "applications" for the previous system revisions. They're probably going to be refined to suit new system, provide some basic set of tools which works without any problems and do it well.
I'll also change input interface button mappings a little bit for devices that I own and therefore am developing with. Maybe I even could make it a separate file, you know, that'd be nice - but it'll need to be capable of being autogenerated and be scalable for future versions. Button mappings have to be separate for each driver, as drivers correspond to some physical devices out there which have their buttons mapped so that they're more or less comfortable to use, but there also have to be options to specify alternative button mappings in parameters, as you can have multiple devices using same driver that you'd want to assign different keys to. IMO, the best idea would be to use config files and to pass filenames as initialisation arguments for drivers, and that's probably what I'm going to make it. For that though, I need to re-work input subsystem - again! - so that is supports multiple devices. It's possible to specify many in the config file, as I knew this would be necessary, but it's not yet supported.
AFAIK input system was the first one to be written. Now I know why it's such a mess. Also, output system is much more simple in terms of what it needs to do so there are less possibilities to mess it up. The problem with input system is that I wrote it having only USB HID devices available, thus the overhead and evdev layers. Thankfully, I worked around it so that once the thing's fixed, there's even no need to change config files once it's updated. Also, seems like menus suffer from that too - there are quite some buttons that might need to be re-mapped for different users.
I've made an installer script which makes an /opt/lcs folder, puts all the files there and installs a systemd unit for the system. Currently, it assumes Raspbian Jessie and therefore systemctl and all new shiny tools coming with it, however, you can easily modify setup.sh to detect whether it's run on a systemd system or not and make it use different autorun files, I guess you can even convert systemd units to init.d scripts - can't you?
Oh, updates. I've also made a basic updater script - it pulls any new commits from GitHub, then moves contents of those and also all local changes to the /opt/lcs directory where, again, the copy of software which's used to be run automatically is located. I will need to improve on it but right now it does what it has to do.
:devil:What's next?
GPIO-connected LCDs, app subsystem and some basic apps, as well as a nice configuration script. Oh, and promotion. I think I need plenty of that to have chances of going through Hackaday Prize. If I have time before the release, I'll also re-make the input subsystem - it all depends whether I'll have enough time and whether I'll manage to develop a clear idea of how exactly to structure it.
I also want to use a VGA I2C breakout and a USB numpad to show that the system can be used with all kinds of computers, not just RPi but even desktops and laptops. See, even though there's hardly anything limiting it, that might not be as obvious.
So far, so good. I even have a couple of schedule days reserved for emergencies, illnesses and alike. Now making videos... I need at least a couple of those, and I have no experience. Guess that could take longer than I think.
Hey, this is my new shutdown&reboot app and it works great! Shutdown&reboot is about everything it does though.