Single-camera stereo vision for obstacle detection in mobile robots

The use of stereo images is an attractive approach to mapping obstacles for autonomous robots, and vehicle navigation benefits from the wide field of view (FOV) it supplies. Modeled after human vision, the simultaneous capture of two images in a single camera frame promises to reduce cost and complexity while facilitating image correspondence.

None of the various models presented in the literature is capable of producing the desired image pair.² Some methods capture the images at different times, complicating correspondence, while others separate images horizontally on the same frame, reducing width by half.

Our design¹ uses four reflecting elements to produce an image pair that occupies the full frame width but is vertically separated. A simplified diagram is shown in Figure 1, with a top view illustrating the optical paths that appear in Figure 2. Two stacked right-angle prisms vertically split the FOV into two sections. Two front surface planar mirrors redirect the view forward. The result is two virtual cameras (Figure 2).

Figure 1. The optical system is composed of two prisms and two mirrors.

Figure 2. The prisms and mirrors create two virtual cameras.

Unfortunately, when all of the elements are perpendicularly mounted, the resulting images do not overlap. Figure 3(a) shows the original image as seen by the camera alone. Figure 3(b) shows the resulting non-overlapping images when the elements are vertical. Overlap can be achieved in one of two ways. The mirrors can be rotated about their longitudinal axis, one up and one down, until the optical axis of the two images is in a horizontal plane. Alternatively, the two prisms can be rotated about the camera's optical axis. In either case, the results are identical, as shown in Figure 3(c).

The FOV of the two images now overlaps, but the images are rotated. Correction simply involves rotating the camera to match the rotation of the images. The resulting combined image is as shown in Figure 3(d).

Figure 3. Various fields of view.

Figure 4. A sample image.

Theoretical analysis proves that the optical axes of the two images are horizontal if the prisms are rotated at an identical angle as the camera.

A simple camera was constructed to test this analysis and a sample picture using it is shown in Figure 4. A view of the same area with the camera alone, without the additional prisms and mirrors, is shown in Figure 5. Triangulation of objects in this image produced distances that were roughly within the pixel uncertainty of the camera. Unfortunately, the resulting optical axes are not parallel. In our experimental setup, the axes are rotated from the camera axis by 1.7°. Software can correct this issue, or the mirrors can be rotated about a vertical axis. For this case, however, theoretical analysis has yet to be carried out.

Figure 5. The original scene without prisms and mirrors.

This single-camera stereo vision system is ideally suited to mobile robot obstacle detection. The use of a single camera and identical optical paths means that images are captured simultaneously and with identical exposures, simplifying analysis. The wide FOV is ideal for a mobile robot, and the limited vertical FOV is not a significant limitation. Our analysis indicates this camera design is workable for obstacle detection, and our early work with a prototype confirms that the concepts are sound. However, the prototype requires further refinement and analysis to confirm preliminary results and further characterize performance.