Project Logs

Collapse

• ZeroMQ vs DBus for Pub-Sub pattern

  Valentin • 01/31/2018 at 21:37 • 0 comments

  I know, I know, comparing ZeroMQ and DBus is like comparing apples and oranges… But, in my small tiny use case, it does make sense !

  The use case is simple, I want to implement an Publish-Subscribe IPC architecture. And both ZeroMQ and DBus permit this kind of architecture.

  What is DBus ?

  “D-Bus is a message bus system, a simple way for applications to talk to one another. In addition to interprocess communication, D-Bus helps coordinate process lifecycle; it makes it simple and reliable to code a "single instance" application or daemon, and to launch applications and daemons on demand when their services are needed.” (https://www.freedesktop.org/wiki/Software/dbus/)

  Sounds like it should work right ? Plus, before restructuring the project, I used a bit the QtDBus module, which is easy enough to grasp and use.

  But, without Qt, in C++, how do you implement a DBus interface ? Or, what library should one use to use DBus in C++ ?

  Well that’s where it falls apart from my point of view.

  You either get very old and/or under-documented projects, or libraries that are meant to be used in a specific context (Qt, GLib, openBMC, etc.).

  You could go with the official C lib but it is a low level implementation, and they even discourage you from it !

  “If you use this low-level API directly, you're signing up for some pain.” (https://dbus.freedesktop.org/doc/api/html/index.html)

  I tried my hands with a few of them but nothing was really satisfying.

  And the whole ‘interface’ and ‘adaptors’ DBus concepts in C++ means adding somewhere some kind of proxy class for every class you need to expose to the outside and I never felt comfortable with using those.

  And them came ZeroMQ !

  I remembered watching the video a while back : https://www.youtube.com/watch?v=v6AGUeZOVSU

  I would advise watching this video if you want to know what ZeroMQ is. But to quote them :

  “ZeroMQ (also known as ØMQ, 0MQ, or zmq) looks like an embeddable networking library but acts like a concurrency framework. It gives you sockets that carry atomic messages across various transports like in-process, inter-process, TCP, and multicast. You can connect sockets N-to-N with patterns like fan-out, pub-sub, task distribution, and request-reply. It's fast enough to be the fabric for clustered products. Its asynchronous I/O model gives you scalable multicore applications, built as asynchronous message-processing tasks. It has a score of language APIs and runs on most operating systems. ZeroMQ is from iMatix and is LGPLv3 open source.” (http://zguide.zeromq.org/page:all#ZeroMQ-in-a-Hundred-Words)

  I started using it and in literally 4 lines of code…

  zmqpp::context context;
  // [...]
  zmqpp::socket zmqPubSocket(*context, zmqpp::socket_type::publish);
  zmqPubSocket.bind("tcp://127.0.0.1:6661");
  // [...]
  zmqPubSocket.send("... spectrum data ..."); 
  

  Booom ! I was publishing messages on a tcp port.

  The cool thing is : just by setting the endpoint of your communications you can send messages through the network (“tcp://0.0.0.0:8080” for example) or locally through the device filesystem (“ipc:///tmp/messages/topic” for example).

  This means a lot for this project, because, by making these endpoints configurable at runtime, you will be able to stream spectrum levels across the internet with no effort.

  There is much more to ZMQ than just Publish-Subscribe, I will also use to build a controller interface with a Request-Response pattern, add a security layer, etc…

  Anyway I believe it is both a solid and flexible foundation for what is to come.

  Imagine a “smart” city using it to gather noise levels from their many sound monitoring raspberry pi everywhere …

• Restructuring the project

  Valentin • 01/02/2018 at 10:12 • 0 comments

  Sometimes it is good to take a few steps back in order to move forward !

  I started this project a long time ago (a few years) and at the time I had no real vision about how the project should be structured.
  

  I just started coding … and features stacked on top of each other, and most of the time, if not always, I was learning how to do it while coding it.

  I ended up with a monolithic program, written entirely with Qt that was responsible of :

  capturing the audio, filtering the audio in third octave bands, writing spectrum data to files, sending data over the network, displaying the data, etc…

  Then I started playing with the notion of IPC with DBus. Data can be sent in “almost” real-time between programs !! Did you know ? Well I didn’t.

  I realized that instead of a big program that did everything I could make several programs that did their own thing independently. That would also bring us closer to the Unix philosophy (programs that do one thing and do it well, or something like that).

  After playing a bit with DBus interfaces, I thought I’d rearrange the project structure like so :

  That “distributed” structure would let any “block” subscribe to the flow of generated spectrums and do whatever they need to without interfering with the publisher or any of the other subscribers.
  

  I also decided to get rid of Qt and to write everything with the C++ STL only. Qt was here for historical reasons and ease of coding. But I feel comfortable enough with plain C++ now, and Qt is a BIG dependency for a console daemon application. I first thought I would keep it around in case I wanted to add a touch screen and a neat UI at some point. But the new “distributed” structure would allow for such thing easily without Qt in the “core” block.

  Here we are. I started coding the thing and *almost* everything went well so far.

  I will write another log on a few issues I ran into with dbus (spoiler: I am now using zeromq instead of dbus) and maybe another about designing the IIR filters and the difference between base 10 and base 2 ⅓ octave bands...

  Cheers !

• Using The Yocto Project

  Valentin • 10/03/2017 at 11:59 • 0 comments

  We distribute an SD card image with a custom distribution, built with the Yocto project.
  

  The idea was to provide something that only needs flashing a SD card in order to just work. I also found it was the simplest way to install Qt on the Raspberry Pi and to have a clean cross-compilation environment on my development machine. 

  It is also very convenient when patching the linux kernel. Which is needed in order to load the WM8782 driver.

  You can find the meta layer used to bitbake the image on github :

  https://github.com/SonoMKR/meta-sonomkr

  The SD image can be found here :

  https://github.com/SonoMKR/meta-sonomkr/releases
  

View all 3 project logs