主要内容

本页采用了机器翻译。点击此处可查看英文原文。

使用经标定的相机测量平面目标

本示例演示了如何使用一台经过标定的相机,以世界单位为单位测量硬币的直径。

概述

本示例演示了如何对相机进行标定,然后利用它测量平面物体(例如硬币)的尺寸。该方法的一个示例是,在传送带上对零件进行测量以进行质量控制。

标定相机

相机标定是指估算镜头和图像传感器参数的过程。这些参数是用于测量相机拍摄到的目标的。本示例演示了如何通过编程方式对相机进行标定。此外,您还可以使用Using the Single Camera Calibrator App对相机进行标定。

要对相机进行标定,我们首先需要从不同角度拍摄标定图案的多个图像。典型的标定图案是一个不对称的棋盘格,其中一侧包含偶数个方格(包括黑格和白格),另一侧则包含奇数个方格。

该图案必须固定在平坦的表面上,且其与相机的距离应与您要测量的对象大致相同。正方形的尺寸必须以世界单位(例如毫米)为单位进行测量,且精确率应尽可能高。例如,在此示例中,我们使用了 9 张图案图像,但在实际操作中,建议使用 10 至 20 个图像以确保标定准确。

准备标定图像

创建一组标定图像。

images = imageDatastore(fullfile(toolboxdir("vision"),"visiondata",...
    "calibration","slr"));
files = images.Files;

% Display one of the calibration images
I = imread(files{1});
figure
imshow(I)
title("One of the Calibration Images")

Figure contains an axes object. The hidden axes object with title One of the Calibration Images contains an object of type image.

估计相机参数

% Detect the checkerboard corners in the images.
[imagePoints, patternDims] = detectCheckerboardPoints(files);

% Generate the world coordinates of the checkerboard corners in the
% pattern-centric coordinate system, with the upper-left corner at (0,0).
squareSize = 29; % in millimeters
worldPoints = patternWorldPoints("checkerboard", patternDims, squareSize);

% Calibrate the camera.
imageSize = size(I,1:2);
cameraParams = estimateCameraParameters(imagePoints, worldPoints, ImageSize=imageSize);

% Evaluate calibration accuracy.
figure
showReprojectionErrors(cameraParams)
title("Reprojection Errors")

Figure contains an axes object. The axes object with title Reprojection Errors, xlabel Images, ylabel Mean Error in Pixels contains 3 objects of type bar, line. This object represents Overall Mean Error: 0.90 pixels.

该条形图显示了标定的准确度。每根柱状图显示了对应标定图像的平均重投影误差。重投影误差是指图像中检测到的角点与投影到该图像上的相应理想世界点之间的距离。

读取待测对象的图像

加载包含待测量对象的图像。该图像包含标定图案,且该图案与您要测量的对象位于同一平面上。在这个示例中,图案和硬币都放在同一张桌面上。

或者,您也可以使用两张独立的图像:一张包含图案,另一张包含待测对象。同样,对象和图案必须位于同一平面上。此外,图像必须从完全相同的视角拍摄,这意味着相机必须固定在原地。

imOrig = imread(fullfile(matlabroot, "toolbox", "vision", "visiondata", ...
        "calibration", "slr", "image9.jpg"));
figure
imshow(imOrig)
title("Input Image")

Figure contains an axes object. The hidden axes object with title Input Image contains an object of type image.

去畸变图像

使用 cameraParameters 对象来消除图像中的镜头畸变。这是确保测量准确所必需的。

% Since the lens introduced little distortion, use 'full' output view to illustrate that
% the image was undistorted. If we used the default 'same' option, it would be difficult
% to notice any difference when compared to the original image. Notice the small black borders.
[im, newIntrinsics] = undistortImage(imOrig, cameraParams, OutputView="full");
figure
imshow(im)
title("Undistorted Image")

Figure contains an axes object. The hidden axes object with title Undistorted Image contains an object of type image.

请注意,这张照片几乎没有镜头畸变。如果您使用广角镜头或低端网络相机,去畸变这一步就显得尤为重要。

分割硬币

在这种情况下,硬币呈彩色,背景为白色。利用图像 HSV 表示中的饱和度组件将其分割出来。

% Convert the image to the HSV color space.
imHSV = rgb2hsv(im);

% Get the saturation channel.
saturation = imHSV(:, :, 2);

% Threshold the image
t = graythresh(saturation);
imCoin = (saturation > t);

figure
imshow(imCoin)
title("Segmented Coins")

Figure contains an axes object. The hidden axes object with title Segmented Coins contains an object of type image.

检测硬币

我们可以假设,在分割后的图像中,两个最大的连通组件分别对应于这两枚硬币。

% Find connected components.
blobAnalysis = vision.BlobAnalysis(AreaOutputPort=true,...
    CentroidOutputPort=false,...
    BoundingBoxOutputPort=true,...
    MinimumBlobArea=200, ExcludeBorderBlobs=true);
[areas, boxes] = step(blobAnalysis, imCoin);

% Sort connected components in descending order by area
[~, idx] = sort(areas, "Descend");

% Get the two largest components.
boxes = double(boxes(idx(1:2), :));

% Reduce the size of the image for display.
scale = 0.25;
imDetectedCoins = imresize(im, scale);

% Insert labels for the coins.
imDetectedCoins = insertObjectAnnotation(imDetectedCoins, "rectangle", ...
    scale*boxes, "penny");
figure
imshow(imDetectedCoins)
title("Detected Coins")

Figure contains an axes object. The hidden axes object with title Detected Coins contains an object of type image.

计算外参

要将图像坐标系中的点映射到世界坐标系中的点,我们需要计算相机相对于标定图案的旋转和平移量。请注意,estimateExtrinsics 函数假设不存在透镜畸变。在此情况下,图像中的图像点是在已使用 undistortImage 进行去畸变校正的图像中检测到的。

% Detect the checkerboard.
[imagePoints, patternDims] = detectCheckerboardPoints(im);

% Extract camera intrinsics.
camIntrinsics = cameraParams.Intrinsics;

% Adjust the imagePoints so that they are expressed in the coordinate system
% used in the original image, before it was undistorted.  This adjustment
% makes it compatible with the cameraParameters object computed for the original image.
newOrigin = camIntrinsics.PrincipalPoint - newIntrinsics.PrincipalPoint;
imagePoints = imagePoints + newOrigin; % adds newOrigin to every row of imagePoints

% Compute extrinsic parameters of the camera.
camExtrinsics = estimateExtrinsics(imagePoints, worldPoints, camIntrinsics);

测量第一枚硬币

为了测量第一枚硬币,我们将边界框的左上角和右上角转换为世界坐标系。然后,我们计算它们之间的欧几里得距离(单位为毫米)。请注意,美国一分硬币的实际直径为 19.05 毫米。

% Adjust upper left corners of bounding boxes for coordinate system shift 
% caused by undistortImage with output view of 'full'. This would not be
% needed if the output was 'same'. The adjustment makes the points compatible
% with the cameraParameters of the original image.
boxes = boxes + [newOrigin, 0, 0]; % zero padding is added for width and height

% Get the top-left and the top-right corners.
box1 = double(boxes(1, :));
imagePoints1 = [box1(1:2); ...
                box1(1) + box1(3), box1(2)];

% Get the world coordinates of the corners            
worldPoints1 = img2world2d(imagePoints1, camExtrinsics, camIntrinsics);

% Compute the diameter of the coin in millimeters.
d = worldPoints1(2, :) - worldPoints1(1, :);
diameterInMillimeters = hypot(d(1), d(2));
fprintf("Measured diameter of one penny = %0.2f mm\n", diameterInMillimeters);
Measured diameter of one penny = 19.00 mm

测量第二枚硬币

按照与第一枚硬币相同的方法测量第二枚硬币。

% Get the top-left and the top-right corners.
box2 = double(boxes(2, :));
imagePoints2 = [box2(1:2); ...
                box2(1) + box2(3), box2(2)];

% Apply the inverse transformation from image to world            
worldPoints2 = img2world2d(imagePoints2, camExtrinsics, camIntrinsics);

% Compute the diameter of the coin in millimeters.
d = worldPoints2(2, :) - worldPoints2(1, :);
diameterInMillimeters = hypot(d(1), d(2));
fprintf("Measured diameter of the other penny = %0.2f mm\n", diameterInMillimeters);
Measured diameter of the other penny = 18.85 mm

测量到第一枚硬币的距离

除了测量硬币的大小外,我们还可以测量它与相机的距离。

% Compute the center of the first coin in the image.
center1_image = box1(1:2) + box1(3:4)/2;

% Convert to world coordinates.
center1_world  = img2world2d(center1_image, camExtrinsics, camIntrinsics);

% Remember to add the 0 z-coordinate.
center1_world = [center1_world 0];

% Compute the distance to the camera.
cameraPose = extr2pose(camExtrinsics);
cameraLocation = cameraPose.Translation;
distanceToCamera = norm(center1_world - cameraLocation);
fprintf("Distance from the camera to the first penny = %0.2f mm\n", ...
    distanceToCamera);
Distance from the camera to the first penny = 719.52 mm

总结

本示例演示了如何使用经过标定的相机来测量平面目标。请注意,测量误差在 0.2 毫米以内。

参考资料

[1] Z. Zhang.A flexible new technique for camera calibration.IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330-1334, 2000.