Camera Models and Calibration

From World to Image

A camera's fundamental job is to capture a three-dimensional world onto a two-dimensional sensor. To work with images computationally, we need a mathematical model that describes this transformation. The simplest and most common model is the Pinhole Camera Model an idealization that imagines light from a scene passing through an infinitesimally small aperture and projecting an inverted image onto a plane.

Using similar triangles, we can relate the coordinates of a 3D point $P = (X, Y, Z)$ in the camera's coordinate system to the coordinates of its projection $p = (x, y)$ on the image plane. If $f$ is the focal length (the distance from the pinhole to the image plane), the relationship is:

x = f \frac{X}{Z} \quad \text{and} \quad y = f \frac{Y}{Z}

The Camera's Parameters

The pinhole model is a good start, but to make it useful, we need to describe the camera's specific properties. These are divided into two categories: intrinsic and extrinsic parameters.

Intrinsic parameters are about the camera's internal construction. They include:

Focal Length ( $f_x, f_y$ ): How strongly the lens converges light. It's measured in pixel units and might be different for the x and y axes if pixels aren't perfectly square.
Principal Point ( $c_x, c_y$ ): The coordinate on the image sensor where the optical axis intersects. This is usually close to the image center, but manufacturing imperfections mean it's rarely exact.

These are combined into a 3x3 matrix called the camera intrinsic matrix, $K$ .

K = \begin{bmatrix} f_x & s & c_x \\ 0 & f_y & c_y \\ 0 & 0 & 1 \end{bmatrix}

Extrinsic parameters describe the camera's position and orientation in 3D world space. This involves a rotation matrix $R$ and a translation vector $t$ , which together define how to transform a point from the world's coordinate system to the camera's coordinate system.

To bring all these transformations together cleanly, we use [{}] This system adds an extra dimension to our vectors, allowing us to represent a 3D-to-2D projection as a single matrix multiplication.

\begin{bmatrix} x \\ y \\ w \end{bmatrix} = K \,[R|t] \, \begin{bmatrix} X \\ Y \\ Z \\ 1 \end{bmatrix}

The Imperfect Lens

The pinhole model assumes light travels in perfectly straight lines. Real lenses, with their curved glass elements, are not perfect. They introduce distortions that warp the image. The two main types are radial and tangential distortion.

Radial distortion is the most common. It causes straight lines to appear curved, especially near the edges of the image. It can be either where lines curve outwards, or pincushion distortion, where they curve inwards.

We can model radial distortion mathematically:

\begin{aligned} x_{\text{distorted}} &= x(1 + k_1 r^2 + k_2 r^4 + k_3 r^6) \\ y_{\text{distorted}} &= y(1 + k_1 r^2 + k_2 r^4 + k_3 r^6) \end{aligned}

Tangential distortion occurs when the lens is not perfectly parallel to the image sensor. This causes the image to appear tilted or stretched in one direction.

\begin{aligned} x_{\text{distorted}} &= x + [2 p_1 xy + p_2(r^2 + 2x^2)] \\ y_{\text{distorted}} &= y + [p_1(r^2 + 2y^2) + 2 p_2 xy] \end{aligned}

Finding the Parameters

The process of finding a camera's intrinsic, extrinsic, and distortion parameters is called camera calibration. The most common method involves taking pictures of a calibration pattern, like a checkerboard, from various angles and distances.

Because the 3D geometry of the checkerboard is known with high precision, we can use algorithms to find the camera parameters that best explain the observed 2D images. Software can detect the corners of the squares in each image and solve for the projection matrix and distortion coefficients that minimize the error between the projected 3D points and the detected 2D corners.

Once calibrated, a camera becomes a powerful measurement tool. We can accurately determine the size and location of objects in the real world from their image, which is the foundation for countless computer vision applications.

Ready to test your knowledge?

Quiz Questions 1/6

What is the primary purpose of the Pinhole Camera Model in computer vision?

Quiz Questions 2/6

A camera's focal length ( $f_x, f_y$ ) and principal point ( $c_x, c_y$ ) are examples of what kind of parameters?

Understanding this bridge between the 3D world and the 2D image is the first step toward building systems that can perceive and interact with their environment.

Computer Vision Depth Estimation