BubbleSynth: Bubbles, Lasers, and a Modular Synth

Origin

Back in 2014, while at MIT Media Lab, I built BubbleSynth — an interactive installation where floating soap bubbles became voices in a generative composition. The original ran on a Mac Mini with openFrameworks doing computer vision and SuperCollider handling synthesis, communicating over OSC. Bubbles emerged from a machine, drifted past stage lights against a black backdrop, and a webcam picked them out as bright "blobs." Position, size, and lifespan adjusted a harmonic sine wave; their disappearance triggered samples — including a few from David Lynch's Dune (1984) for the sci-fi atmosphere I was after at the time. "The spice must flow."

The 2014 paper's future-work section ended with a speculation: that one could eventually use a depth camera to eliminate the black-backdrop requirement, but it was unclear whether a Kinect's IR could even see a thin-film bubble. A decade later, the RealSense D415 answers that — its depth stream picks up bubbles cleanly, segmenting them by distance from the camera rather than by visible-light contrast. The new installation runs in ambient light, against any background.

And the thing I'd wanted since I first picked up a galvo laser is now in place: a laser that physically traces each bubble's contour in mid-air, live.

What's New

Three big additions, plus a full software rebuild:

  1. A galvo laser traces each bubble in mid-air, live. As bubbles drift through the tracking area, an RGB galvo projector outlines them in real time — bubbles become luminous, their contours drawn into the same physical space they're moving through.
  2. Depth-based bubble segmentation. The RealSense D415's depth stream isolates bubbles by their Z-distance from the camera, not by brightness contrast. No black backdrop, no controlled lighting, no shinbusters. The system works in ambient room light, against any background — a major freeing of the installation requirements compared to 2014.
  3. A modular synth (VCV Rack 2) voices them. Replacing SuperCollider with a node-based modular environment gives me visual patching, hot-swappable timbres, and a much richer voice palette. Two oscillator voices (Mutable's Plaits in 2-operator FM mode) sing through the bubble positions, with fast LFO vibrato giving them a bird-like warble. Underneath, an ambient drone holds the harmonic foundation in C Lydian.
  4. An Arduino controls a relay + servo for autonomous operation. The bubble machine fires on demand from the host machine; a servo-mounted wand rises into a fan's airflow when the system wants to coax out a big bubble.

The underlying perception loop (camera → bubble detection → OSC → synthesis) is conceptually unchanged from 2014, but every component has been rebuilt with current tools, and depth segmentation removes the original's environmental constraints.

Hardware

  • Computer: Intel NUC11PAHi7 (i7-1165G7, Iris Xe graphics, 16GB RAM, Windows 11)
  • Camera: Intel RealSense D415, using the depth stream. Depth-based segmentation isolates bubbles by their distance from the camera. The IR projector is on (it's what enables depth computation). No backdrop required; works under ambient light.
  • Laser: RGB galvo projector accepting ILDA-format point streams (~30k pps)
  • Bubble machine: Off-the-shelf party bubble machine, controlled by an Arduino + 5V relay board
  • Bubble wand: Servo-mounted wand (TD 8130MG metal-gear, 13 kg-cm torque) that dips into bubble solution and rises into a fan's airflow to produce the occasional very large bubble

What's not needed compared to the original 2014 build: no black backdrop, no upturned stage lights, no controlled lighting environment. The depth-based isolation pipeline tolerates whatever lighting and background the venue offers.

Software

  • TouchDesigner runs the perception and laser pipeline:
    • Depth-threshold isolation — anything within the bubble field's distance range from the camera is treated as foreground
    • Cleanup and binarization
    • Trace SOP...
Read more »