Project Logs

Collapse

• Progress on quick/dirty UX and system prototype

  AKA • 04/03/2017 at 19:42 • 0 comments

  Following up from my last update, I went with the "Fast and Frugal" prototyping solution for the time being. Here are some photos of the Android interface on my doorframe:

  ...I had the usual web-development foibles while getting the Tornado instance up and running: the most recent jQuery does not work with the most recent jQuery Mobile (!?), and so forth. This is why I usually make non-screen-based projects ;-)

  Most of the UI flow is working, but without an active-devices DB to replace the dummy data. You can swipe any of the rows here and disable the device therein:

  ...that's all for now. I encountered some funny stuff getting the popular

  git pull

  command working on openWRT. The tl;dr is that because it's on such a small system, only the bare-bones git is compiled and included in the distro. Also, classic workaround 'curl-devel' is not available either, so the quickest way is to just not use https:// and instead use git:// (nobody really needs security, right? Heh heh)

  You can check out the latest code on the Mezzo repo. Much, much more to do, but this'll suffice for today.

• UI Hardware

  AKA • 04/03/2017 at 16:07 • 0 comments

  ...I've been working out the best way to present visitors with what is currently a new/unknown option: the option to disable the host's listening technologies.

  Introducing a new affordance like this requires careful physical and interaction design: it should be optimally clear what choice is being offered, and how to exercise that option should be unambiguous.

  Initially, I wanted to replicate the form and placement of the traditional mezuzah. (I go more into the mezuzah as the conceptual origin of this project in another Project Log). The mezuzah is discreet but easy to find, and marks the threshold of the domestic realm; it makes sense for this to be the point at which your guests can make decisions about their privacy when entering your space.

  I like hobo codes and other functional graffiti (really neat example by FFFFFAT here), and so had originally imagined a tall, thin touch-sensitive OLED that displayed symbols corresponding to the types of systems active: an ear or speech bubbles for Alexa's active transcription, a CCTV silhouette for Dropcam's video surveillance, etc.

  However, it's oddly hard to find weird-aspect-ratio OLEDs, and what few I could find were going to be a pain to add touch-interaction to. (My imagined process for adding touch-sensitivity: cover each sensing region onscreen with ITO, wire each ITO to a capsense board (or use Teensy's inbuilt touch sensing), go from there).

  I like the idea of having several form factors for the Mezzo system, so here are the approaches I'm thinking of at present:

  Simple UI

  Because a house doesn't often change the number of listening technologies it contains, one very simple approach is to make a small array of static buttons that have LEDs showing whether the technology is active. I picked up these clicky little pushbuttons from All Electronics, I think, and I like their handfeel and old-school looks. All they need is a simple web-enabled SoC like the Digispark Oak (of which I have many spares lying around). If I wanted to get fancy, I'd add BLE to the OpenWRT router and leverage the RFDuino's smaller form...

  OLED-based UI

  This was the original UI concept - either a roughly 3x1 aspect-ratio vertical rectangle, or three 128x64 rectangles in a unifying enclosure. Adding touch to these will likely be a pain, so this will likely be the last form factor I try.

  Fast & Frugal

  While thinking of these hardware solutions, for this hardware contest ;-), I realized I had a readily-available, hi-res, multitouch system on my desk. Android! I'm going to be prototyping with an old Android handset for the near future because it's ideal for fast iteration: I can design the UI for the standard webkit browser, I can host most of the logic on the server-side, and I can even charge this one wirelessly, so I can make a charging cradle that affixes the phone to the right place on the post.

• UX question: how to track when a guest leaves?

  AKA • 04/03/2017 at 15:00 • 0 comments

  How do we know when to turn systems back on?

  One very important affordance that I want Mezzo to offer is the ability to determine when a guest's stay is over.

  If Mezzo simply allows visitors to disable the homeowner's listening devices, it's not nearly as desirable: this means that there owner has to remember to "clean up" after their guests and re-activate each device, and that's a lot of extra work.

  I worked for quite some time on the idea that I could keep a running list of observed electronic devices by observing wireless probe requests. Put simply, this means I would have a little observer-script always running on my router, and it would look at the regularity of probe requests from unique devices. When a device had not been heard from in some amount of time, we would strike it from our running list of "present" devices and assume whoever was carrying that cellphone/iPad/etc had left.

  Knowing this, I contrived a heuristic that would roughly associate "new" and "just-arrived" MAC addresses with incoming interactions to the Mezzo UI interface. Thus, when Mezzo got a "please disable system FOO" request, we could look at the list of recently-arrived guests and make a note to restore system FOO when those guests had left.

  But!

  Do you see the silly, basic problem that makes this an insufficient solution?

  Dear reader, you should not have to own or use an electronic device in order to be "counted" by my system!

  Embarrassingly, I went quite a ways down this path (some sketching that may be interesting to you is here) before I realized I needed a more inclusive solution. It's early days, so this may change, but my options seem to be:

  • Allow guests to specify the time that systems will be disabled, with a generous default like 2 hours (or a default settable by the homeowner)
  • Use an identification technology to actively view incoming and departing guests (the ironies here of using something like facial recognition to track users of a privacy device are multifoliate, so I will probably not pursue this (although a counterpoint is, if this system runs entirely on a local subnet and is put to good use, that's precisely where the "good" parts of those technologies have the most impact))
  • Use simple motion-detection technology to identify warm bodies moving into or out of the space

  ...for now, I think the simple motion-detection is the way to go (my studio is brimming with different kinds of motion sensors, including some neat thermal cameras), but there are some complexities that I'll go into in future posts.

  One affordance I think is certainly necessary for all implementations is an unambiguous audible chime indicating that listening systems are being turned back on; this way if there is a false positive (we think the guest has gone, but he/she is still present), the participants will know immediately and be able to act on this.

• Basic user experience outline

  AKA • 03/31/2017 at 23:10 • 0 comments

  Visitor Experience

  The experience of a guest is pretty simple: upon approaching the threshold of the house, they see a touchscreen showing icons that represent different listening systems:

  • speech transcription (Alexa, Siri, etc)
  • video recording (Dropcam, IP cameras, local CCTV systems)
  • "Other" (catchall for any other technology that guests can disable, like MAC-sniffing devices, other wireless monitors, etc)

  Here's what the swipe looks like; apologies for the lo-fi prototyping:

  Disabling camera systems

  Homeowner Experience

  Setting up a flexible system that can disable your home electronics is a bit daunting. Part of the reason I'm making this into an open project on Hackaday is that I imagine the audience will be more comfortable with technical setups than the average consumer (though, that being said, it raises a host of issues about the responsibility of selling complex and opaque surveillance technology to a mass consumer market who are unable to probe its complexities).

  I don't have much of an interaction flow designed for the homeowner-setup process yet, but basically it will consist of these steps:

  • OpenWRT router boots and realizes it is unconfigured
  • Router enables a (secured) SSID whose default password is printed on the router (or otherwise supplied with the hardware), prompting the owner to log in
  • Owner sees a configuration page similar to the stock OpenWRT Luci interface; from this list, the owner can identify devices that should be blockable by the Mezzo system.
  • Owner may also identify slave devices on the subnet; a slave device is basically a commodity 110v outlet switch that the router can toggle. This allows any device to be turned off by guests, not just devices whose traffic can reliably be blocked.

  System Design / Complexity

  ...the system's internal logic is significantly more nuanced. In the next few build logs I'll talk through my reasoning for several of these problems, such as:

  • How to detect when a visitor has left the house? There's a lot of work to be done here, since it's not reasonable to ask departing guests to remember to re-enable all the systems. A simple timeout period, like "disable all these devices for 2 hours" might have to suffice initially.
  • How to prevent non-houseguests (ie, passers-by) from interfering with the system? For example, when you go on vacation, you do not want your burglar to disable the Dropcam before robbing you...
  • How to clearly signal the functionalities of this interface to first-time guests without overwhelming them?

View all 4 project logs