Evaluating the Accuracy of Single Camera Calibration

Overview

Camera calibration is the process of estimating parameters of the camera using images of a special calibration pattern. The parameters include camera intrinsics, distortion coefficients, and camera extrinsics. Once you calibrate a camera, there are several ways to evaluate the accuracy of the estimated parameters:

Calibrate the Camera

Estimate camera parameters using a set of images of a checkerboard calibration pattern.

% Create a set of calibration images.
images = imageDatastore(fullfile(toolboxdir("vision"), "visiondata", ...
    "calibration", "mono"));
imageFileNames = images.Files;

% Detect calibration pattern.
[imagePoints, boardSize] = detectCheckerboardPoints(imageFileNames);

% Generate world coordinates of the corners of the squares.
squareSize = 29; % millimeters
worldPoints = patternWorldPoints("checkerboard", boardSize, squareSize);

% Calibrate the camera.
I = readimage(images, 1); 
imageSize = [size(I, 1), size(I, 2)];
[params, ~, estimationErrors] = estimateCameraParameters(imagePoints, worldPoints, ...
                                     ImageSize=imageSize);

Extrinsics

You can quickly discover obvious errors in your calibration by plotting relative locations of the camera and the calibration pattern. Use the showExtrinsics function to either plot the locations of the calibration pattern in the camera's coordinate system, or the locations of the camera in the pattern's coordinate system. Look for obvious problems, such as the pattern being behind the camera, or the camera being behind the pattern. Also check if a pattern is too far or too close to the camera.

figure; 
showExtrinsics(params, "CameraCentric");

figure; 
showExtrinsics(params, "PatternCentric");

Reprojection Errors

Reprojection errors provide a qualitative measure of accuracy. A reprojection error is the distance between a pattern keypoint detected in a calibration image, and a corresponding world point projected into the same image. The showReprojectionErrors function provides a useful visualization of the average reprojection error in each calibration image. If the overall mean reprojection error is too high, consider excluding the images with the highest error and recalibrating.

figure; 
showReprojectionErrors(params);

Estimation Errors

Estimation errors represent the uncertainty of each estimated parameter. The estimateCameraParameters function optionally returns estimationErrors output, containing the standard error corresponding to each estimated camera parameter. The returned standard error $$\sigma$$ (in the same units as the corresponding parameter) can be used to calculate confidence intervals. For example +/- $$1.96\sigma$$ corresponds to the 95% confidence interval. In other words, the probability that the actual value of a given parameter is within $$1.96\sigma$$ of its estimate is 95%.

displayErrors(estimationErrors, params);

			Standard Errors of Estimated Camera Parameters
			----------------------------------------------

Intrinsics
----------
Focal length (pixels):   [  714.1884 +/- 3.3219      710.3783 +/- 4.0579  ]
Principal point (pixels):[  563.6478 +/- 5.3967      355.7255 +/- 3.3036  ]
Radial distortion:       [   -0.3536 +/- 0.0091        0.1730 +/- 0.0488  ]

Extrinsics
----------
Rotation vectors:
                         [   -0.6096 +/- 0.0054       -0.1789 +/- 0.0073       -0.3835 +/- 0.0024  ]
                         [   -0.7283 +/- 0.0050       -0.0996 +/- 0.0072        0.1964 +/- 0.0027  ]
                         [   -0.6722 +/- 0.0051       -0.1444 +/- 0.0074       -0.1329 +/- 0.0026  ]
                         [   -0.5836 +/- 0.0056       -0.2901 +/- 0.0074       -0.5622 +/- 0.0025  ]
                         [   -0.3157 +/- 0.0065       -0.1441 +/- 0.0075       -0.1067 +/- 0.0011  ]
                         [   -0.7581 +/- 0.0052        0.1947 +/- 0.0072        0.4324 +/- 0.0030  ]
                         [   -0.7515 +/- 0.0051        0.0767 +/- 0.0072        0.2070 +/- 0.0029  ]
                         [   -0.6223 +/- 0.0053        0.0231 +/- 0.0073        0.3663 +/- 0.0024  ]
                         [    0.3443 +/- 0.0063       -0.2226 +/- 0.0073       -0.0437 +/- 0.0014  ]

Translation vectors (mm):
                         [ -146.0514 +/- 6.0391      -26.8688 +/- 3.7318      797.9024 +/- 3.9002  ]
                         [ -209.4354 +/- 6.9637      -59.4568 +/- 4.3578      921.8196 +/- 4.6295  ]
                         [ -129.3821 +/- 7.0907      -44.1033 +/- 4.3751      937.6829 +/- 4.4913  ]
                         [ -151.0046 +/- 6.6905      -27.3256 +/- 4.1339      884.2787 +/- 4.3925  ]
                         [ -174.9496 +/- 6.7056      -24.3502 +/- 4.1606      886.4959 +/- 4.6686  ]
                         [ -134.3092 +/- 7.8887     -103.4984 +/- 4.8925     1042.4551 +/- 4.8184  ]
                         [ -173.9842 +/- 7.6891      -73.1694 +/- 4.7812     1017.2383 +/- 4.8126  ]
                         [ -202.9443 +/- 7.4327      -87.9094 +/- 4.6482      983.6955 +/- 4.9072  ]
                         [ -319.8858 +/- 6.3213     -119.8901 +/- 4.0922      829.4580 +/- 4.9591  ]

How to Improve Calibration Accuracy

Whether or not a particular reprojection or estimation error is acceptable depends on the precision requirements of your particular application. However, if you have determined that your calibration accuracy is unacceptable, there are several ways to improve it:

Summary

This example showed how to obtain and interpret camera calibration errors.

References

[1] Z. Zhang. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330-1334, 2000.