Mixed Reality Room Scale Shooting Gallery

Final shot first!

During the first test the protective plastic was left on and caused the acrylic to bubble. This was a spare piece of light guide acrylic which I ended up not liking as much as the standard clear cast acrylic.

For the second try the protective paper was removed and the job rerun. 

The finished piece looked great so the next step was a full 18" x 24" sheet.

I was able to fit nine pins, with one defect, I'm left with the eight needed for a single sheet. An extra sheet is on order to complete the sixteen total targets.

With a phone LED.

Trying some different art.

Added a particle explosion to the ball. The pins are supposed to knock down and return to position.

@Amar said there should be a laser sight on the VR gun. The idea would be that the player uses the laser sight for physical targets, and the mobile screen for VR targets. I don't want the laser sight on all the time so I'll use the Vive Tracker pogo pin to switch it on and off.

In Unreal Engine send a haptic (rumble) event to the tracker and the pin will go high. 

The VR targets also have to be created physically and I want to light them up. I'm going to try a version of the 3d LED illusion lights for the targets.

These can easily be laser cut and etched, and hopefully don't need too many LEDs to still look ok. They should also be small enough for the micro servos to move around.

My version in VR with bowling pins:

And the physical template to be cut and etched in acrylic:

I added a bit of angle to the version on the left for a better 3d effect. The additional lines sell the illusion more, if needed I can adjust the VR model to match.

Built a simple play area in Google Blocks VR

With post processing

The basic game idea is the "throw something at something" else type carnival game; knock out all of the targets to win.

In this game there are two target colors, two projectile colors, and two realities: physical and virtual. 

The targets occupy the same space in both the physical and virtual worlds. When a target is the same color in both the physical and virtual world it can be knocked out with a projectile of that same color.

The player holds a controller with an attached smartphone that allows them to track target colors, aim, and fire.

Additionally the projectile colors behave differently. One color of projectile is slower, and less accurate, but can be fired repeatedly. The second color of projectile is fast, and very accurate, but requires reloading.

Modified from James Bruton's sketch example. This sketch uses the serial connection to listen for servo addresses 100 through 115, then activate, pause, and return each servo called.

The Unreal Engine app only has to send the address followed by a newline.

This first sketch had a couple for loops that were too slow:

#include <Wire.h>
#include <Adafruit_PWMServoDriver.h>
Adafruit_PWMServoDriver pwm = Adafruit_PWMServoDriver();
#define SERVOMIN  150 // this is the 'minimum' pulse length count (out of 4096)
#define SERVOMAX  600 // this is the 'maximum' pulse length count (out of 4096)
int action;
long interval = 500;
unsigned long currentMillis;
long previousMillis;
uint8_t servoAddr[] =       {  0,  1,  2,  3,
                               4,  5,  6,  7,
                               8,  9, 10, 11,
                              12, 13, 14, 15  };
                        
int flagAddr[] =            {  0,  0,  0,  0,
                               0,  0,  0,  0,
                               0,  0,  0,  0,
                               0,  0,  0,  0  };
                        
long previousMillisAddr[] = {  0,  0,  0,  0,
                               0,  0,  0,  0,
                               0,  0,  0,  0,
                               0,  0,  0,  0  };
void setup() {
  Serial.begin(9600);
  pwm.begin();
  pwm.setPWMFreq(60);
}
void run_servo(uint8_t &myServo, int &myFlag, long &myPreviousMillis) {
  if (myFlag == 1) {
    // pwm.setPWM(myServo, SERVOMIN, SERVOMAX);
    for (uint16_t pulselen = SERVOMIN; pulselen < SERVOMAX; pulselen++) {
      pwm.setPWM(myServo, 0, pulselen);
    }
    myFlag = 2;
    myPreviousMillis = currentMillis;
  } else if (myFlag == 2 && currentMillis - myPreviousMillis > interval) {
    // pwm.setPWM(myServo, SERVOMAX, SERVOMIN);
    for (uint16_t pulselen = SERVOMAX; pulselen > SERVOMIN; pulselen--) {
      pwm.setPWM(myServo, 0, pulselen);
    }
    myFlag = 0;
    myPreviousMillis = currentMillis;
  }
}
void loop() {
  currentMillis = millis();
  if(currentMillis - previousMillis > interval) {
    previousMillis = currentMillis;
    if (Serial.available() > 0) {
      action = Serial.parseInt();
      if (Serial.read() == '\n') {
        for(uint8_t i = 0; i < 16; i++) {
          if (action == (100 + i)) {
            flagAddr[i] = 1;
          }
        }
      }
    }
    for(uint8_t i = 0; i < 16; i++) {
        run_servo(servoAddr[i], flagAddr[i], previousMillisAddr[i]);
    }
  }
}

Current sketch with the for loops unwound:

#include <Wire.h>
#include <Adafruit_PWMServoDriver.h>
Adafruit_PWMServoDriver pwm = Adafruit_PWMServoDriver();
#define SERVOMIN  150 // this is the 'minimum' pulse length count (out of 4096)
#define SERVOMAX  600 // this is the 'maximum' pulse length count (out of 4096)
int action;
long interval = 2000;
unsigned long currentMillis;
long previousMillis;
uint8_t servoAddr[] =       {  0,  1,  2,  3,
                               4,  5,  6,  7,
                               8,  9, 10, 11,
                              12, 13, 14, 15  };
                        
int flagAddr[] =            {  0,  0,  0,  0,
                               0,  0,  0,  0,
                               0,  0,  0,  0,
                               0,  0,  0,  0  };
                        
long previousMillisAddr[] = {  0,  0,  0,  0,
                               0,  0,  0,  0,
                               0,  0,  0,  0,
                               0,  0,  0,  0  };
void setup() {
  Serial.begin(9600);
  pwm.begin();
  pwm.setPWMFreq(60);
}
void run_servo(uint8_t &myServo, int &myFlag, long &myPreviousMillis) {
  if (myFlag == 1) {
    for (uint16_t pulselen = SERVOMIN; pulselen < SERVOMAX; pulselen++) {
      pwm.setPWM(myServo, 0, pulselen);
    }
    myFlag = 2;
    myPreviousMillis = currentMillis;
  } else if (myFlag == 2 && currentMillis - myPreviousMillis > interval) {
    for (uint16_t pulselen = SERVOMAX; pulselen > SERVOMIN; pulselen--) {
      pwm.setPWM(myServo, 0, pulselen);
    }
    myFlag = 0;
    myPreviousMillis = currentMillis;
  }
}
void loop() {
  currentMillis = millis();
  if(currentMillis - previousMillis > interval) {
    previousMillis = currentMillis;
    if (Serial.available() > 0) {
      action = Serial.parseInt();
      if (Serial.read() == '\n') {
        if (action == (100)) {
          flagAddr[0] = 1;
        }
        if (action == (101)) {
          flagAddr[1] = 1;
        }
        if (action == (102)) {
          flagAddr[2] = 1;
        }
        if (action == (103)) {
          flagAddr[3] = 1;
        }
        if (action == (104)) {
          flagAddr[4] = 1;
        }
        if (action == (105)) {
          flagAddr[5] = 1;
        }
        if (action == (106)) {
          flagAddr[6] = 1;
        }
        if (action == (107)) {
          flagAddr[7] = 1;
        }
        if (action == (108)) {
          flagAddr[8] = 1;
        }
        if (action == (109)) {
          flagAddr[9] = 1;
        }
        if (action == (110)) {
          flagAddr[10] = 1;
        }
        if (action == (111)) {
          flagAddr[11] = 1;
        }
        if (action == (112)) {
          flagAddr[12] = 1;
        }
        if (action == (113)) {
          flagAddr[13] = 1;
        }
        if (action == (114)) {
          flagAddr[14] = 1;
        }
        if (action == (115)) {
          flagAddr[15] = 1;
        }
      }
    }
    run_servo(servoAddr[0], flagAddr[0], previousMillisAddr[0]);
    run_servo(servoAddr[1], flagAddr[1], previousMillisAddr[1]);
    run_servo(servoAddr[2], flagAddr[2], previousMillisAddr[2]);
    run_servo(servoAddr[3], flagAddr[3], previousMillisAddr[3]);
    run_servo(servoAddr[4], flagAddr[4], previousMillisAddr[4]);
    run_servo(servoAddr[5], flagAddr[5], previousMillisAddr[5]);
    run_servo(servoAddr[6], flagAddr[6], previousMillisAddr[6]);
    run_servo(servoAddr[7], flagAddr[7], previousMillisAddr[7]);
    run_servo(servoAddr[8], flagAddr[8], previousMillisAddr[8]);
    run_servo(servoAddr[9], flagAddr[9], previousMillisAddr[9]);
    run_servo(servoAddr[10], flagAddr[10], previousMillisAddr[10]);
    run_servo(servoAddr[11], flagAddr[11], previousMillisAddr[11]);
    run_servo(servoAddr[12], flagAddr[12], previousMillisAddr[12]);
    run_servo(servoAddr[13], flagAddr[13], previousMillisAddr[13]);
    run_servo(servoAddr[14], flagAddr[14], previousMillisAddr[14]);
    run_servo(servoAddr[15], flagAddr[15], previousMillisAddr[15]);
  }
}