Stereo Vision

A fair while ago I got interested in computer vision, and specifically, stereo vision algorithms. Every day I think of new things that autonomous robots could do if only we had really good computer vision software to guide them. So I set out to learn more about computer vision, and in particular stereo vision and 3d reconstruction. But I also wanted to get some practical experience on the side so I started to build a stereo camera.

Here is what I ended up with:


Never mind the plywood support — that is just so I can set it on top of my monitor (stereo selfies!) or (more likely) mount it on a tripod as show in the picture.

The backbone is a 10mm carbon fiber tube, with suitable clamps. You can readily find them on Ebay since they are also used for building multi-rotor unmanned aerial vehicles (UAVs). The clamps are bolted to 1mm thick carbon fiber plates on which the cameras are mounted.

Although the vast majority of stereo vision algorithms runs on monochrome cameras I decided to go with a color camera in case I ever wanted to use color. I settled on a Ximea MQ013CG-E2 camera with USB3.0 interface, mostly because they explicitly supported the compute platform I intended to use, a Odroid XU3. At about $500 a piece the Ximea cameras were also quite a bit cheaper than comparable models from Point Grey. The lenses needed to be bought separately: I opted for Tamron TA814MPMF 8mm lenses with a C-type mount and an aperture between 1.4-16. They run about $120 each. A focal length of 8mm seemed to give just enough field of view. The Ximea cameras can be connected via a cable to enable shutter synchronization in hardware. I didn’t try because I couldn’t find straight-forward instructions on how to wire them up (seemed to require pull up/down resistors etc).

A word about the blue USB 3.0 cables in the picture: you can’t just use any USB 3.0 cable. I first tried some low end ones that I bought off of Amazon that didn’t work at all. Sometimes the camera wouldn’t even be recognized by the kernel drivers, USB packets were dropped en masse, or the connection was lost altogether. After much trial and error I found the L-com ICAVISU3AMICB-03M cables that worked. The USB3 micro B connectors come out easily, you really need high quality cables with thumb screws.

The big fat box in the center is an Intel NUC 5i5MYHE. It’s running on a Core i5-5250U processor. You are essentially looking at a desktop PC squeezed into a 4″x4″ form factor, with a Samsung 850 EVO 250 GB M.2 SSD and an Intel AC7265 wireless card inside. I drilled holes in the chassis to install external antennae which substantially improved Wifi reception and bandwidth. The NUC weighs about 600g, way more than I find acceptable for potential use in a small aerial vehicle. I took it apart and weighed the components. The board (including heatsink and fan) is about 220g, the remaining 380g are due to the solid metal case and mounting brackets inside. In other words there is hope that it can be made lighter. What does really bother me about the NUC is that you cannot install a GPU (the CPU has a built-in HD6000 GPU which cannot be used via either OpenCL or CUDA).

As mentioned earlier, the NUC was not my first choice (but certainly in the end by far the best). Originally I went with an Odroid XU3, a much lighter unit, based on an octa-core ARM processor (4 big cores, 4 little cores). It looked good on paper, even had two USB 3.0 ports, although one of them was offered as a micro-usb port, which requires an impossible-to-find micro-to-micro USB 3.0 cable with thumbscrews. Using one Ximea camera worked o.k. once I had the right USB 3.0 cable, but I never got the Odroid to work with two. It would drop lots of USB packets and finally disconnect after no more than a few minutes, no matter if I used the second USB 3 bus via a micro-to-micro cable (no thumbscrews!), or if I used just one USB3 port and a hub (I tried two different hubs, same problem). Eventually noting that it came with two clean USB 3.0 ports, I upgraded to the Odroid XU4. Same problem!

I suspect that there is some internal I/O restriction that limits the data transfer rate via the USB port. For instance sending packets via a USB 2.0 Wifi card would also cause the camera’s USB3 packets to be dropped. The same happened when I tried to do some OpenCL programming on the ARM processor’s built-in Mali GPU. It seemed like any I/O activity beyond a single USB3 camera would cause packet losses and camera disconnects. After going through the various Odroid’s I tried my luck with an NVidia Jetson TK1. That required using a USB3 hub and had the same packet drop problem as the Odroids. For now I’m cured of using low power cell phone processors to handle stereo USB3 traffic.

What you can’t see is the battery pack: a 4S 4000mAH Turnigy battery pack from Hobbyking. The NUC can handle the voltage without a regulator, and it just so happens to also be the battery pack for the quad copter that I intended to attach the camera to: