Computing the fundamental matrix of an image pair

The introductory section of this chapter presented the projective equation, describing how a scene point projects onto the image plane of a single camera. In this recipe, we will explore the projective relationship that exists between two images that display the same scene. These two images could have been obtained by moving a camera to two different locations to take pictures from two viewpoints, or by using two cameras, each of them taking a different picture of the scene. When those two cameras are separated by a rigid baseline, we use the term stereovision.

Getting ready

Let's now consider two pinhole cameras observing a given scene point, as shown in the following figure:

We learned that we can find the image x of a 3D point X by tracing a line joining this 3D point with the camera's center. Conversely, the scene point that has its image at position x on the image plane can be located anywhere on this line in the 3D space. This implies...