Vision Stereo

2007 International Symposium on Field-Programmable Custom Computing Machines

Low-Cost Stereo Vision on an FPGA
Chris Murphy, Daniel Lindquist, Ann Marie Rynning Thomas Cecil, Sarah Leavitt, Mark L. Chang Franklin W. Olin College of Engineering Needham, MA 02492 mark.chang@olin.edu

Abstract
We present a low-cost stereo vision implementation suitable for use in autonomous vehicleapplications and designed with agricultural applications in mind. This implementation utilizes the Census Transform algorithm [5, 6] to calculate depth maps from a stereo pair of automotivegrade CMOS cameras. The final prototype utilizes commodity hardware, including a Xilinx Spartan-3 FPGA, to process 320x240 pixel images at greater than 150 frames per second and deliver them via a USB 2.0 interface.

1.Background
Obtaining an accurate, three-dimensional model of unstructured outdoor environments remains a primary challenge of field robotics. Despite widespread robotics use, at $7,000-$20,000 LIDAR remains a cost-prohibitive technology for many applications. Our work seeks to develop a distance sensor package at much lower cost than contemporary options, enabling a variety of new applications andresearch where cost is an important design factor. Our prototype achieves performance comparable to an ASIC, yet costs less than a dedicated PC and software stack. Related work includes custom ASIC and software implementations, and only one significant FPGA implementation. In [4] a multiple FPGA implementation on a PCI card was developed, consisting of 16 Xilinx 4025 FPGAs connected in a partialtorus with 16 one-megabyte SRAMs. This solution achieved 42 frames per second at 320x240 pixel resolution. Our implementation can be thought of as a modern, low-cost and high-performance descendant of this initial FPGA-based design.

the same “scene”, but from a different viewpoint. By examining features and objects from one camera image and finding their relative locations in the other cameraimage, it is simple to calculate the range of those objects from the camera. Objects that are very far from the cameras will appear in the same relative location within the two frames, while objects that are very close to the camera will have a large horizontal shift, or disparity, from one camera’s viewpoint to the other. We chose to implement the Census Transform [6] after evaluating algorithmsfrom [1–3, 6], due to its high performance on our agricultural datasets. The Census Transform compares a “window” of pixel intensities around a central pixel to several candidate “matching” windows in a second image. This comparison is made by creating a bitstring for each window, which represents the brightness of each pixel in the window relative to the central pixel. The “window” from the secondimage with the smallest Hamming distance from the primary “window” is determined to be the best match for the pixel. The difference in the pixel “window” positions between the two camera frames is inversely related to the distance of an object from the camera. The greater the distance between matching “windows”, the closer an object is to the camera pair. The highly parallel nature of the CensusTransform makes it an excellent candidate for FPGA acceleration.

2. Implementation
Our hardware implementation was done entirely with a single Xilinx Spartan-3 FPGA. We determined experimentally that a 13x13 pixel window and a maximum search disparity of 20 pixels provided a good balance between computational complexity and algorithm performance for our application. The top-level algorithmicblock diagram of the correspond module in the FPGA is shown in Figure 1. The correspond module takes data streaming from our left and right CMOS imagers, synchronizes them through a series of delay units, and performs a Census Transform

1.1. Stereo Vision
Stereo vision systems use a pair of cameras oriented much like human eyes; each camera views approximately

0-7695-2940-2/07 $25.00 © 2007...