Project Logs

Collapse

• The Depth Quandary

  MasterOfNull • 01/22/2021 at 20:19 • 1 comment

  The resolution of the depth information you can get just from object size is going to drop off really quickly as you move away from the camera.
  

  When you are relatively close to the camera, the ball will occupy a lot of pixels and the depth of the object within the frame is high resolution.  

  As you move away from the camera however, the number of pixels the ball is occupying in your captured image rapidly decreases.  

  As that happens, the possible position of the ball along a line in space drawn from the camera to the center of the ball, gets larger and larger.  Eventually a single pixel error in calculating the ball size will translate to a massive positioning error along that line!

  Some Solutions

  1. We use a larger ball.  
     1. This moves the useable range of the sensing out further at the expense of near
     2. A larger wand is more cumbersome. 
  2. We use a camera with a narrower FOV. 
     1. This decreases the usable width and height of the performance area, but then extends the length.
     2. The camera will need to be further away from the performance area to start with.
  3. We use a camera with a really high resolution.
     
     1. The speed of OpenCV is inversely proportional to the number of pixels you are feeding it.  Bigger images are a lot slower to process.
     2. Using the very low latency integrated CSI-2 camera interface of the PI or Jetson Nano, we end up maxing out at 8MP or about 1080P@30fps of RAW image data.  To do much better than this we would need to use a USB-C camera and the cost skyrockets.
  4. We use stereo imaging and determine the depth via triangulation.  
     1. We immediately lose half of the capture rate.
     2. We lose the ability to track multiple objects without more work as we can't triangulate each object without being able to differentiate them.  
        1. Flashing each ball works, but then your capture rate goes down really fast as only one ball per color can be lit per captured frame and then the flashing must to be coordinated.  
        2. Using different colored balls works, but then you lose the ability to paint in the color you are seeing as the ball.

  For now I think the solution I am going with is to use a narrower FOV CSI-2 camera that can do 1080P@30fps and feed it to a Jetson Nano.
  

• Hackchat of the start of this..

  MasterOfNull • 01/20/2021 at 19:30 • 1 comment

  Daren Schwenke11:14 AM
  Dropping an idea here. Light up and track a ping pong ball on a stick with a stationary camera using OpenCV using size as your depth cue. Re-create as a 3D model, displaying it with rotation around the center of it. Allow changing the color of the ball, duplicate in the model.

  3D light painting in realtime.

  Daren Schwenke11:16 AM

  Suppose you could use IMU and a cell phone and guess the approximate location of the camera as well to eliminate the stationary bit.

  deʃhipu11:29 AM
  you can also use looming to determine distance

  deʃhipu11:29 AM
  that's how insects avoid obstacles

  anfractuosity11:35 AM
  you mean using just a single camera then?

  deʃhipu11:41 AM
  sure, and low-res too

  anfractuosity11:41 AM
  i'm curious how well using a single std. camera would work compared to stereo

  deʃhipu11:44 AM
  one thing with looming is that it requires motion

  anfractuosity11:44 AM
  ahh

  deʃhipu11:44 AM
  works best when you move towards the object at a known speed

  deʃhipu11:45 AM
  you basically see how fast it grows -- objects that are close grow faster than object that are far

  anfractuosity11:46 AM
  could you simply use the diameter of the ping pong ball in pixels, from a single camera, and convert that to depth, which i thought is what @Daren Schwenke meant, or are there reasons that doesn't work well compared to 2 cameras

  deʃhipu11:47 AM
  it's easier than stereo, because instead of finding the same object on two different photos taken at slightly different angles, you just need to find the same object at two different sizes

  deʃhipu11:47 AM
  @anfractuosity that requires pretty large resolution, though, or very consistent lighting

  anfractuosity11:48 AM
  and knowing the size of the ping pong ball in reality too maybe?

  deʃhipu11:48 AM
  ping pong balls have standardized size

  anfractuosity11:48 AM
  oh true heh

  anfractuosity11:49 AM
  i think ToF cameras sound pretty nifty, not sure how pricey they are though

  deʃhipu11:49 AM
  Kinect

  anfractuosity11:49 AM
  the first one used a pattern i thought, is the 2nd one tof then

  deʃhipu11:50 AM
  if you can control the hardware in the wand, there are easier ways to do this

  deʃhipu11:50 AM
  like the wii rmote

  anfractuosity11:50 AM
  oh that uses ir i think?

  anfractuosity11:50 AM
  and a sensor bar thing right

  deʃhipu11:50 AM
  doesn't matter if the light is visible or not

  anfractuosity11:51 AM
  oh actually the 'bar' is the light emitter i think actually

  deʃhipu11:51 AM
  the important thing is it's modulated, so you can tell it from the background easily

  anfractuosity11:52 AM
  just found this, which sounds v. interesting, will have to read the paper

  anfractuosity11:52 AM
  
  

  https://www.youtube.com/watch?v=ZolWxY4f9wc

  YOUTUBE TWO MINUTE PAPERS

  Kristin joined  the room.12:04 PM
  

  deʃhipu12:05 PM
  The Algorithm

  Nolan Hergert joined  the room.12:39 PM
  

  MS-BOSS12:44 PM
  The problem is that it will work well when you move near the camera. As you get further from the camera, the precision will drop fast.

  Daren Schwenke12:46 PM
  yes

  deʃhipu12:53 PM
  put the camera on the ball, and look for blinking light beacons in the anvironment

  deʃhipu12:53 PM
  environment

  deʃhipu12:54 PM
  you will also get orientation info from it

  deʃhipu12:54 PM
  and the farther they are, the more precision you actually get

  Barry Baldwin joined  the room.12:57 PM
  

  Daren Schwenke1:07 PM
  The problem is with any camera held by the user, it has to be oriented towards some reference to work. That limits the size of what you can create versus a ball on a stick pretty quickly. Also with putting multiple sources or points, then you have to deal with orientation feedback or the distance between your points is no longer your relative distance to the object.

  Daren Schwenke1:07 PM
  A bigger ball would work and give you more accuracy at a greater distance.

  Daren Schwenke1:08 PM
  I just liked the small size too. :)

  Daren Schwenke1:11 PM
  button to select each color/turn on. On while held so you can stop your lines. Or select...

  Read more »

View all 2 project logs