Project Logs

Collapse

• Project Update

  Chris Graham • 07/29/2019 at 14:03 • 0 comments

  Sorry for the long time in not posting.  The project transformed last September and I have not been able to write about it until now. 

  Last September the prototypes were working great but difficult to assemble.  The mouthpiece would be too difficult for the majority of hackers to make.  It also needed to be smaller  which would require tiny surface mount electronics, pick and place assembly and injection molded parts.  i.e.  It needed to be a manufactured product. 

  However, I did not have the resources to run a business by myself as it would take me away from the research.  Last September I was introduced to Rudy Verpaele  of IMOXPLUS  https://www.imoxplus.com/site/ by Pedro Eustache http://www.pedroflute.com/. 

  Rudy was developing an innovative physical modeling synth called Respiro that could benefit from a more expressive electronic mouthpiece. Rudy planned to develop hardware wind controllers when Respiro was completed. 

  Rudy is in Belgium and I am in Canada and I traveled to Belgium last October.  We found we were perfect partners and joined forces.  Since then Rudy has been on completing and releasing Respiro I have been redesigning the mouthpiece making it smaller, more powerful and manufacturing-friendly.

  We are now starting to go public with the mouthpiece which we have renamed the Photon because of its use of light in much of its sensing.  Here is the first public announcement:
  

  https://www.facebook.com/groups/313247028739320/?multi_permalinks=2467348923329109¬if_id=1564141779424965
  

• Improvements to Max Patches

  Chris Graham • 09/21/2018 at 16:02 • 0 comments

  Calibration now has the ability to set an amount of median-style smoothing for each of the sensors.  Also added is the ability to save calibration settings and improved appearance.

  Another big addition is an output mapping window.  It includes controls to  enable/disable output from each sensor, choose the channel, map it to a type of MIDI message and save groups of these settings as patches.  (Output mapping patches are saved separately from calibration patches so that multiple output patches can share the same calibration settings.)

  The next step will be to add the ability for breath-out and breath-in to trigger note ons and note offs.  In the breath controller version the note numbers will be supplied by an external controller such as a keyboard.  In a wind controller version note numbers would come from the "finger unit". 

  Also remaining to be added are user editable mapping graphs, one for each sensor that will allow definition of non linear response curves.  Many other minor details also remain to be added such as the ability to choose 7 bit or 14 bit MIDI controller output, ability to send aftertouch, ability for note-on/off to gate the flow of controller messages, etc.  Support for some other planned sensors also remains to be added.
  

  Other interesting additions will be the ability to correctly handle slurred vs tongued note-ons. i.e. Providing for note-offs to overlap previous note-ons depending on whether a note change was caused by a change in breath pressure vs a change in note number from the external controller.  This is especially important for electronic wind instruments but could be a useful capability even for notes from a keyboard controller.  As far as I know this capability has never before been implemented for a breath controller.  (Other than in my own earlier prototypes for the mouthpiece)

• A Completed Mouthpiece

  Chris Graham • 09/18/2018 at 15:52 • 0 comments

  Here are some photos of a completed mouthpiece.  It's supported on a "neck unit". which is the formed wire device that hooks around the back of the user's neck and supports the mouthpiece in front of the mouth for use as a breath/multimodal controller. 

  This mouthpiece has sensors for:

  • Breath blowing out
  • Breath sucking in
  • Lower lip
  • Upper lip
  • Oral cavity/tongue/throat
  • Forward tilt
  • Side tilt
  • Rotation about the vertical axis

  The red cable is from the USB port of the Teensy 3.2 processor plugged into the bottom of the main PCB.  The Teensy will eventually send MIDI but for now it streams all sensor data to my Mac on which I am developing a Max patch for calibration, generating MIDI controllers and pitch bend and generating note-ons/offs with breath pressure. The tilt and rotation sensing is done with an Adafruit IMU board that plugs to a header on top of the main PCB in the blue mouthpiece housing.

  Here the mouthpiece calibration patch I created in Max 7.  It shows graphs of data from the sensors and provides controls for setting the usable range out of the total raw range of 1024 values measured by the Teensy 3.2 analog to digital converter.

  The upper graph in each sensor's section shows the uncalibrated sensor value and the lower graph shows the calibrated values.  Calibration is done by rescaling the raw value into the range between the "min" and "max" usable values of the sensor's total range.  Eventually it will also be possible to also remap each sensor's calibrated output through a user adjustable non linear curve and then map the values to MIDI controllers, pitch bend, etc.

• Improvements to the Design

  Chris Graham • 09/04/2018 at 13:03 • 0 comments

  I constructed the prototype described in the previous log entry, with the buttons on top and display on the bottom.  All the sensors worked perfectly (see previous log entry) and I got the display showing some text.  

  I mounted it on a prototype "neck unit" which is the formed wire support for use in hands free mode.  Sorry I don't have a picture of the prototype with the buttons as I took it apart before taking pictures.

  However what I learned from it was that the 80mm long mouthpiece housing just feels and looks too long.  I decided that the mouthpiece module does not really need to contain the user interface because when used as a breath controller it could be configured with a host computer, and when used with a finger unit, the finger unit could contain the display.  Also as a breath controller power could come over USB and when used with a finger unit, the finger unit could contain a battery.

  Also making the holes in the housing for the display and buttons was an unnecessary complication in building mouthpiece models and I want assembly of the basic module to be as easy as possible.

  I eliminated the display and buttons from the mouthpiece module which allowed me to shorten it to 45mm long which feels and looks better:

  The mouthpiece still includes a header to mount any of the three types of Adafruit IMUs.  I also also added a MIDI out.  The connector is a 2.5 mm TRS jack compatible with various 2.5mm to standard MID adapter cables that are available.  You can see the MIDI connector in the opened image of the bottom of the PCB.  It's the black component behind the tube on the lower right.

  I then built a real version of the 3D model .  Here are some pictures of the real assembled mouthpiece module on the formed wire neck unit.

  You also see the mouthpiece mounted on a real "neck unit" for the first time.  This is the support that allows it to be used as a hands free breath controller.  This neck unit was hand made, which is quite challenging.  I sent a 3D printed model of it to a wire forming company who should be able make them in large quantities if needed, very inexpensively.  

  The nice thing about this formed wire design is that it provides very stable support for the mouthpiece which is needed because the sensors are so subtle.  Also the user can easily adjust the shape slightly for a perfect fit by hand-bending the steel wire.  Yet another benefit is that the pivot has enough friction to hold the mouthpiece steady but the support can be tilted outward to temporarily get it out of the way and then brought back into position still perfectly aligned.  Other breath controller supports (e.g. Yamaha BC3a or TEControl) tend to leave the mouthpiece in your mouth unless you remove it completely.

  This mouthpiece housing you see here does not yet contain the circuit board.  I designed a new version of the PCB and ordered some to be made.  They should be here later this week.

  If you look closely you can see another significant addition I made to the design:  I added a small "fin" 3mm high across the top front of the mouthpiece.  This is something I had in my best prototypes in the past but had eliminated to save cost.  It allows the mouthpiece to be further stabilized using the upper lip or teeth.  When I finally was able to try the current version of the mouthpiece on the neck unit I was reminded how much benefit came from the extra stabilization.

  I realize it's unconventional to have a "fin" on the top of a mouthpiece as no existing instrument has it, but when you try it, it just feels even more perfect.  

  The fin causes an undercut for injection molding the mouthpiece shell which makes...

  Read more »

• First Live Demo of Sensors

  Chris Graham • 08/21/2018 at 22:52 • 0 comments

  When the epoxy dried (see previous log entry) I assembled the mouthpiece unit to the main board and the main board into the Hammond box housing.  See previous log entries for details.   This is the assembled unit: 

  I tested the sensors with an Arduino program that reads the ADCs of the sensors and streams the data to the host computer.   
  

  Everything was working really well so I created a quick patch in Max/MSP to graph the outputs of each of the four sensors: lower lip, tongue/oral cavity, upper lip and breath pressure blowing out/drawing in.

  Here is an image of the Max patch with the data graphs.

  Here is a link to a one minute recording I just uploaded to YouTube.
  

  Mouthpiece Sensor Demo
  

  The graphs are small but most of the data is very precise, having a dynamic range that uses much of the  0-1023 range of the Teensy 3.2 analog to digital converters.

• Ordering a Few Utility PBCs

  Chris Graham • 08/21/2018 at 16:26 • 0 comments

  While waiting for a milled mouthpiece prototype to arrive from Protolabs (hopefully the final perfect version), and for the epoxy to dry on my other mouthpiece, I decided to design and order three different small PCBs that will be useful for future testing.

  The following board is an adapter to take a cable from the Picoblade serial port (finger unit) connector on the main board and convert it to 0.1" spacing for testing and for prototyping finger units.  It's also also compatible with the FTDI USB to serial TTL adapter smart cable for testing the serial protocol communicating with a host computer.  The jumper is to optionally connect the power lines.

  This board is an adapter to take a ribbon cable from the pins on the small mouthpiece connector board header and convert to a row of 0.1" spacing pins for use on a breadboard.  This board may also be useful if anyone wants to use the mouthpiece with their own electronics, not the Multiwind main board.  The connections are ground, the upper lip touch pad, the oral cavity phototransistor, the oral cavity LED, the lower lip phototransistor, and the lower lip LED.  The other two pins are not currently used in the mouthpiece but are included for future options.

  I also designed a little board is to test an idea for making touch pads to be finger unit instrument keys.  The following images shows the concept.  This is is a very preliminary design for the finger unit of a brass style instrument. I won't take time for now to explain all the details  now but it would have fingering somewhat similar to the EVI.  

  The following is an view with the housing removed.  It's not shown but every finger unit would have a processor, probably a Teensy 3.2 or LC.  It could be mounted on the bottom of the board shown here or on another board that would slide in one level below the one seen here.

  You can see capacitive touch keys plugged into the top of the PCB. 

  I needed a way to be able to connect and attach the keys into the unit after the PCB was slid from the end into the aluminum housing.   I also wanted to avoid having to run individual wires to each key pad and bulky connectors.

   Note that on the underside of a board below this one there might be headers facing downward with keys inserted from the bottom of the housing.

  The following is an exploded view of a single key.  The pad itself is a round two layer printed circuit board 10mm in diameter and 0.8mm thick.  Soldered to the bottom is a four pin 0.127 inch spacing surface mount header.  The ring is a bezel.  Many of these could be 3D printed very quickly to any desired shape.  It covers the edge of the board and to insulates it from the aluminum housing.  The pad PCB snaps into the top of the bezel.  

  The male header and its pins are inserted though a 1/4 inch hole in the housing and plug into a female header on the main board.  The entire top of the round PCB pad is a copper layer over-painted during board manufacture with some colour.  In this case it's black but PCBs can be ordered in other colours such as white, red, yellow and green.  This would permit making key pads in different colours maybe for coding or just to look cool.  It will be cheap to make even hundreds of these boards.  The heads are also inexpensive.  The total materials cost of a key could be under $2.50 and connecting it is free other than making the main board.

  A one-pin header would have been sufficient as there only one electrical connection to pad but I chose a four pin header for these reasons:

  - I'm hoping that the friction of the pins will be enough to hold a key in place.  There will be more friction from four pins.  ...

  Read more »

• The Electronics

  Chris Graham • 08/18/2018 at 16:57 • 1 comment

  Here are some images of the current schematics and PCB layouts.

  Overall, the prototype is working exceptionally well.  

  I wrote a simple program for the Teensy that reads all sensors: the lower lip and oral cavity optical sensors, the upper lip capacitive sensor and the pressure sensor.  All are amazingly controllable as viewed in form of raw ADC values printed to the serial monitor. 

  The dynamic range and controllability of every sensor is exceptional.  Based on past experience they should convert to an outstanding user experience of midi controllability.  I'm pretty sure this will be able to provide an entirely different and superior user experience vs the mouthpiece of any commercially available wind controller that ever existed.  Over years of research I've tried almost every one.

  I have not yet assembled and tested the display, IMU or external connector but all that should be pretty straightforward now that the mouthpiece itself is working so well.

  The next step will be to design and make another version of the 3D model and main PCB adding some more features and correcting a few minor problems:

  - It was difficult to solder the surface mount version of the serial port connector so change to the through hole version.

  - Correct a minor error where a PCB trace was omitted where it appeared to be connected in the schematic but was really not connected.

  - There should be enough room in the housing for up to a 250 mAh Li-Po battery so look into whether the circuit can include this and the ability to charge it over the USB connection.

  - Add the ability to power the board either from USB, from a battery, or from power from the finger unit coming up the power line of the serial connector.  Powering from the finger unit, when present, would allow a larger battery capable of powering the mouthpiece for longer.

  - Look into adding a connector for old style MIDI output.  I reserved the second TX line on the Teensy for this purpose and all that's needed is two resistors and a small connector.  A full size midi connector would not fit in the housing.  I need to look for a small enough three or four pin connector to carry the MIDI signal to an adapter cable.

  - Look into including a vibrator motor to provide tactile vibratory user feedback.  The electronics only require a PWM output, a transistor and a button sized vibrator motor.  I reserved a PWM pin on the Teensy for this purpose.  I've wanted to try vibratory feedback for a long time as it could be a way of communicating to the user states of the mouthpiece such as that they are at a zero level of pitch bend, or that they are on a resonance sweet spot when planing a physically modeled instrument such as a trumpet.  It may also be used in hands free human computer interaction.

  - The other unpopulated daughterboard next to the IMU (in the 3D models of an earlier log update) could be used for various purposes but most importantly might be used to add wireless midi.  This won't necessarily be a feature in the short run but I need to look into it to the extent that there would be the capacity to fit it among the other existing components.

• Assembled Prototype

  Chris Graham • 08/18/2018 at 16:43 • 0 comments

  Here are images of the assembled prototype based on the images in a previous log entry entitled "Major Progress".   Most of what you see below is explained in the comments of the previous log entry.

  The following three images shown the mouthpiece connector board with mounted parts plugged into the main board.  The parts on the connector board are supported perfectly aligned to slide into the mating features inside the mouthpiece.
  

  You can also see I changed from though hole to surface mount resistors.  I find 1206 size resistors to be easier to work with than through hole resistors.  There is no need for lead forming or cutting and they are easier than through hole resistors to unsolder to swap values.  I unsolder them with a hot air soldering iron although they are easy to unsolder with the an appropriate blade on a conventional soldering iron.  Also, using many surface mount parts should allow contracting out board pick and place assembly if it was ever necessary to make more than could be hand assembled.

  The main PCB is two layer but it was necessary to use a four layers for the mouthpiece connector board because so many leads had to cross over each other in a small space.  however even a four layer board this size is very inexpensive. 

  The following image shows the tube to the pressure sensor.  There was not enough height to mount the pressure sensor standing up so it will be mounted flat on its back.  To avoid  kinking the tubing an adapter part will be 3D printed to mate the tube at a 90 degree angle to the pressure sensor.  The air exhaust tube is not installed.

  The brass electrode for upper lip sensing can be seen soldered to the connector board.  In the final version the electrode will have a 2mm wide tab that will go though a hole in the connector board.  I've ordered 50 electrodes from a water jet cutting service at a cost of about $2 per electrode which will be almost the total cost of the upper lip sensing feature in addition to two resistors, which is a pretty good deal.

  The following image is the bottom of the board showing the mouthpiece connector, display connector, the Teensy and the serial connector.  It's possible to buy various lengths of preassembled six conductor cables for the connector used in the serial port.  Each finger unit board would have a similar connector for the other end of the cable.

  The following image shows the PCB and sensor assembly inserted into the mouthpiece. Medical grade epoxy will seal around the connectors in the mouthpiece and securely hold the sensor assembly.  The eight pin connector has pins with a spacing of 0.127" that plugs to a matching connector on the main board.  

  An eight conductor ribbon cable with an IDC connector could also be plugged to the mouthpiece allowing the mouthpiece to be mounted some distance from its electronics.  I intend to design a small adapter board that would take a cable from the mouthpiece and convert it to row of 0.1" spacing pins to plug into a standard breadboard.  This should allow easy experimenting with alternative mouthpiece electronics.  The mouthpiece only needed six conductors but an eight conductor connector was used because that size of ribbon cable and IDC connector is more readily available.  The extra two leads will be reserved for future features that may be added to the mouthpiece.

  If there is enough interest I can offer for sale assembled mouthpiece modules much like the one in this image that other makers could use in their own projects. ...

  Read more »

• Major Progress

  Chris Graham • 08/18/2018 at 14:24 • 0 comments

  It's been a while since I did a log update in that time I tested the PCBs that were on order at the end of the last update, 

  These were the key results:

   - The lip and oral cavity sensors worked very well.  Excellent controllability of both.  For the next prototype I'll inset the light pipes slightly into the front edge of ledge to increase the range of control to higher lip forces.
   - The Pressure sensor and instrumentation amp worked well.  The zero-pressure output was  at about half of the 3.3v supply.  Blowing drives voltage down, sucking drives voltage up. Amp gain is about 800 to get large enough voltage swing. 
   - Tested a small capacitive sensing electrode inside the top of the mouthpiece to measure upper lip position.  Reading with the Teensy capacitive sensing worked surprisingly well.  The the next prototype will use capacitive sensing for the the upper lip.
   - The method of connecting the mouthpiece to the main board by plugging the pins of the optical components into a connector on the main board worked but was too fragile.  In the next prototype I'll use a real connector on the mouthpiece to plug into a connector on the main board.  This  will require a separate PCB for the mouthpiece.

  Based on these findings I revised the 3D models and ordered new PCBs including the main board and a new vertically mounted small board to hold the mouthpiece components and the connector from the mouthpiece to the main board.

  This is what the design looks like now with the mouthpiece shell and main housing removed:

  You can see the new mouthpiece board to which all its components can be assembled before inserting them into the mouthpiece shell.  This also shows the four switches for user control and a rocker to allow them to be pressed through holes in the housing.  In the upper right there is an optional I2C connected Adafruit 10DOF IMU board.  The the lower right is another I2C connected daughterboard for future expansion.  There are two tubes from the mouthpiece, one carrying static pressure to the pressure sensor and the other carrying air flow out the back of the mouthpiece housing.  Having separate tubes allows amount of air flow to be adjusted by a set screw in the mouthpiece without affecting pressure detection by the pressure sensor.

  This is a view from the bottom.  You  can see the mouthpiece components and connector board, the mouthpiece connector on the main board and a Adafruit OLED display board mounted on the bottom of the main board.  The tube is to carry mouthpiece air flow out the back of the housing.   The Teensy 3.2  is connected to the bottom of the main board.  In the lower right is a 6 lead connector for connection to a finger unit.  I dropped the plan to use I2C in favour of a serial link with flow control.  The signals are RX, TX, CTS, RTS, power and ground.  Serial will be faster, easier to program and more versatile.

  Here are a few more images of the design with the housings installed.

  You can see the buttons on the top of the housing.  After some experimenting with ergonomics  it was clearly more convenient for the user to have the display on the bottom and the buttons on top.

  Here you see the display on the bottom The hole is to allow pressing the Teensy 3.2 reset button.

  Here is a version that includes a hypothetical finger unit omitting any details of the controls on the finger unit.

  Here is an image of the mouthpiece mounted on the neck support for hands free use.  In this mode it should be an amazing a breath controller.  I also have plans to develop a version for hands free control for people with motion disabilities.  With its three types of mouth control (lower lip, tongue and upper lip), breath control (blowing and sucking) it could be better than any existing hands free controller. ...

  Read more »

• Assembly and Testing of Mouthpiece Sensors

  Chris Graham • 07/14/2018 at 19:16 • 0 comments

  Things accomplished since previous log entry:

  • Assembled the photo-sensing parts and the air pressure tube into a mouthpiece shell milled from black FDA compliant acetal copolymer by FirstCut (part of Protomold).  The mouthpiece I had  was slightly out of date and did not have a place for an air flow tube but this will be added in the next prototype.  The photo sensors consist of an LED infrared emitter/phototransistor pair for the lower lip and the same for the oral cavity.  I potted the parts into place and sealed the mouthpiece by filling it with silicone.  Eventually medical grade epoxy will be used but I don't have it yet so I used silicone aquarium sealant which is safe and works fine although it's messy to work with and hardens slower.  The silicone's high viscosity makes the job look messy but it's good enough for now.

  Note that the above picture still does not have the lip light pipes installed.

  • The leads of the photo-parts need to exit in a straight line with 0.1 inch (2.54mm) separation so they can be plugged into a standard connector.  

    Forming the leads is tricky to do accurately enough so I designed and 3D printed die for forming the leads: 

    To use: Place the head of the part in the  cavity on the left upper surface of the die, hold the head down firmly, press the leads into the forming surfaces and you have leads with the perfect shape. You can see I made blue and black markings on the parts to distinguish LEDs from Phototransistors as they look exactly the same.

  • When potting the parts it's necessary to hold the parts in alignment until the potting compound dries.  I designed and 3D printed an alignment jig that can be placed on the back of the mouthpiece and has holes for the leads and tubes.  The jig is the blue part in the picture above that also shows the internals of the mouthpiece.  This is what it look like in use.  

    The jig can be removed when the potting compound hardens.  For the future I'm going to make a  version of the jig with a hole to inject the potting compound through.  Also I ordered a tube of this epoxy from Atom Adhesives:  https://www.atomadhesives.com/AA-BOND-FDA2-FDA-Grade-Epoxy-Adhesive-Medical-Food

  • Then I hooked up the mouthpiece to a breadboard for testing.  This picture shows just the lip sensor connected to resistors and power from the breadboard.

    This was the first time trying  this significantly changed design since I last worked on the project five years ago.  It was a big moment!  And it worked perfectly!  Amazingly controllable,  Of course all that can be seen now are changing voltages from the sensors based on lower lip/jaw and  tongue/throat throat position.  However for both sensors it's possible to select, hold and repeatably return to a wide range of positions.  When mapped to music controllers this should give a great playing experience.

  • Also, while potting the parts I realized that it would also be possible to at the same time attach a stainless steel electrode in the top of the mouthpiece.  This would provide capacitive sensing of upper lip position.  I've been looking for a practical way to add upper lip sensing for a long time and this should be perfect!  Inexpensive and not much harder to assemble than without it.  Basically it could be a strip of stainless steel 1/4" (6.35mm) wide, 20mm long and 0.5 mm thick.  There's an available capacitive sensing port on the Teensy.  A slot and support for the front of the electrode could be included in the injection mold design.  The underside of the electrode would sit on the hardened medical grade epoxy.  A slot to align the electrode could be added to the part alignment jig.  Here is the hole for the electrode.  Extending the hole all the way to the back...

  Read more »

View all 12 project logs