SmartRoom

Here is a little example plugin that uses the CloudFileManager, Properties, RaspiCam, Logging and MQTT functionality provided by the plugin API.

When you publish a string representing the desired file name into the topic home/room/picam/upload, this plugin takes a photo with dimensions and quality as specified in the PiCamUpload.properties and upload it to dropbox. If "current_time" is sent , the current date and time will be used as file name.

public class PiCamUpload extends BasePlugin {

 @Override
 public void onStart() {

  // get CloudFileManager implementation from plugins (dropbox in this case)
  CloudFileManager cfm = getSystemManager().getCloudFileManager();
		
  Objects.requireNonNull(cfm, "A CloudFileManager implementation is needed for this plugin to work.");

  // get properties from this plugins's .properties file
  int width = Integer.parseInt(getProperties().getProperty("width"));
  int height = Integer.parseInt(getProperties().getProperty("height"));
  int quality = Integer.parseInt(getProperties().getProperty("quality"));

  // subscribe to a fitting topic
  MQTT.get().subscribe("home/room/picam/upload", (MqttMessage m) -> {

  // convert binary payload to ascii string
  String filename = new String(m.getPayload(), StandardCharsets.US_ASCII);
  if (filename.equals("current_time"))
    filename = new SimpleDateFormat("yyyy-MM-dd_HH:mm:ss").format(new Date());
			
  File img = new File("./" + filename + ".jpg");
			
  try {
    img.createNewFile();
  } catch (IOException e) {
    getLog().error("Exception", e);
  }
  RaspiCamera.take(img, width, height, quality);
  try {
    cfm.uploadFile(img, null);
  } catch (FileNotFoundException e) {
    getLog().error("Exception", e);
  }
 });
}
}

Since quite some time the project has a Android app to control the lights, turn on the alarm system (just the PIR) and to view all important logs. Later I will add the possibility to live stream (or to slide show :P) from the RasPi-Cam.

The app has a ForegroundService with a permanent notification that allows quick access even from the lock screen and the actual app can be killed in the task manager.

Communication with the RasPi is done via MQTT with a broker on my vServer. Log entries generated on the Pi are currently sent to the App using Google Cloud Messaging; maybe I will switch that part to MQTT later on.

The Rasperry Pi Java software now has a plugin system that allows to outsource all user specific logic into runnable, .jar-packed plugins. Additionally their are non-runnable plugins that implement interfaces. So the user can decide on specific implementations e.g using Dropbox as cloud file service or to use a local text-to-speech engine instead of a cloud based solution.

The system supports hot deployment of runnable plugins.

Thanks to MQTT and the new XML configuration it is easy to have multiple RasPis all running the SmartRoom software. The config allows to do simple wiring without the need to write any code. E.g. controlling 433mhz outlets and relays or publishing the current value of a PIR-sensor to the broker server is as easy as that:

<?xml version="1.0" encoding="UTF-8"?>
<smartroom broker_uri="ssl://bennet-krause.de:1883">
    <states>
	<state topic="home/room/ceilinglight/state" type="boolean">
	    <!-- enables the relay if a message on that topic is "1" -->
	    <pinappender pin="GPIO 3"/> 
	</state>
	<state topic="home/room/desklight/state" type="boolean">
	    <!-- controls a 433mhz outlet -->
	    <outletappender pin="GPIO 0" system_code="9" unit_code="8"/> 
	</state>
	<state topic="home/room/pir/state" type="boolean">
	    <!-- publishes a message with "1" or "0" on that topic depending on the current pin state -->
	    <pinprepender pin="GPIO 4" mode="pull up"/> 
	</state>
    </states>
</smartroom>

This is my current test setup for developing the OpenCV based presence sensing and room position tracking system component. It consists of my RasPi 2 and a 160° wide angle camera module from china (100% compatible with the original camera module) mounted on the power transformer of my ceiling lights (which will later on be disrupted by the Arduino controlled relay).

After quite some struggling with the compilation, I finally managed to compile the brand new OpenCV 3.1.0 (including the Java bindings) on my Pi.

Fortunatly, after loading the preinstalled (on Raspbian Jessie) V4L2 kernel driver, which creates a /dev/videoX, OpenCV can use the camera module out of the box, using it like a standard USB webcam. Therefore it only has a resolution of 640x480, but that is by far enough for motion tracking needs, a higher resolution would overload the RasPi anyways.

That way, I get about 2 to 5 position updates per second (unoptimized), with a single core at full load, leaving enough resources for the other software modules.

The detection of the current position of the occupant is done by taking the thresholded difference between two sequential images, blurring the result to remove noise, get a binary result image and filtering out too small or too close-by blobs and finally calculating the centroid of the result blob.

Support for multiple occupants will be added later on, more important is the interpretation of the constantly updated position. I will need to define rectangular zones for all interesting room areas (e.g. couch, bed, desk).

For presence sensing it is planned that a PIR will initially turn the camera on if a movement is detected. From then on the software constantly tracks the occupants position and (this is to be added) listens for a movement in outside direction through the door and thereby registers that the room is unoccupied again (turn off camera and lights).

Visualized, the position tracking looks as follows: