Multi Camera Calibration Theory

In the Multi-Camera Calibration article, we focused on the practicalities of Multi-Camera Calibration. In this article, we will explain some of the theory behind.

Calibration object

The calibration object for multi-camera calibration must be such that it is possible to accurately determine its pose (position and orientation) relative to a camera.

Transformation matrix

Combines both rotation and translation required to transform a position vector in one coordinate frame to another coordinate frame via matrix multiplication.

The transformation between two cameras

Consider the scenario with two cameras.

Two cameras pointing at same object

We want to find the transformation matrix \(H^{Right}_{Left}\) which describes the transformation from the camera on the right to the camera on the left.

Left camera in right camera frame

What we have though, from either camera, is the pose of the calibration object in the camera reference frame \(H^{Obj}_{Left}\) and \(H^{Obj}_{Right}\). The inverse of these poses, (eq. 1) and (eq. 2), allows the transformation of the points in the point clouds to the calibration objects’ coordinate frame.

\[(H^{Obj}_{Left})^{-1} = H_{Obj}^{Left} \qquad (1)\]
\[(H^{Obj}_{Right})^{-1} = H_{Obj}^{Right} \qquad (2)\]
Object in left and right camera frame

We could have stopped there, and provided transformation matrices to the calibration object, as a final result. We do not want to have a dependency on the pose of the calibration object, as that is often arbitrary. Thus we want either \(H_{Left}^{Right}\) (eq. 3) or \(H^{Left}_{Right}\).

\[H_{Left}^{Right} = H_{Obj}^{Right} * H_{Left}^{Obj} = H_{Obj}^{Right} * (H_{Obj}^{Left})^{-1} \qquad (3)\]
Two cameras pointing at same object