利用图像点特征在杂乱场景中定位对象

本示例演示了如何在杂乱的场景中，根据该目标的参考图像来检测特定目标。

概述

本示例介绍了一种基于查找参考图像与目标图像之间点对应关系的特定目标检测算法。即使目标发生比例变化或平面内旋转，它也能检测到这些目标。它对轻微的离平面旋转和遮挡也具有很强的稳健性。

这种目标检测方法最适用于具有非重复纹理图案的对象，因为这些图案能产生独特的特征匹配。对于颜色均匀的对象，或者带有重复图案的对象，这种技术可能效果不佳。请注意，该算法旨在检测特定对象（例如参考图像中的大象），而非任意的大象。

步骤 1：读取图像

读取包含对象的参考图像。

boxImage = imread("stapleRemover.jpg");
figure;
imshow(boxImage);
title("Image of a Box");

读取包含杂乱场景的目标图像。

sceneImage = imread("clutteredDesk.jpg");
figure;
imshow(sceneImage);
title("Image of a Cluttered Scene");

步骤 2：检测点特征

检测两幅图像中的点特征。

boxPoints = detectSIFTFeatures(boxImage);
scenePoints = detectSIFTFeatures(sceneImage);

将参考图像中检测到的最强的点特征可视化。

figure;
imshow(boxImage);
title("100 Strongest Point Features from Box Image");
hold on;
plot(selectStrongest(boxPoints, 100));

可视化目标图像中检测到的最强点特征。

figure;
imshow(sceneImage);
title("300 Strongest Point Features from Scene Image");
hold on;
plot(selectStrongest(scenePoints, 300));

步骤 3：提取特征描述符

从两幅图像中的兴趣点处提取特征描述符。

[boxFeatures, boxPoints] = extractFeatures(boxImage, boxPoints);
[sceneFeatures, scenePoints] = extractFeatures(sceneImage, scenePoints);

步骤 4：查找推测的点匹配

根据特征描述符将特征进行配对。

boxPairs = matchFeatures(boxFeatures, sceneFeatures, MatchThreshold=40,...
    MaxRatio=0.7, Unique=true);

显示推定匹配的特征。

matchedBoxPoints = boxPoints(boxPairs(:, 1), :);
matchedScenePoints = scenePoints(boxPairs(:, 2), :);
figure;
showMatchedFeatures(boxImage, sceneImage, matchedBoxPoints, ...
    matchedScenePoints, "montage");
title("Putatively Matched Points (Including Outliers)");

步骤 5：利用推测匹配在场景中定位目标

estgeotform2d 计算匹配点之间的变换关系，同时剔除异常值。通过这种变换，我们可以确定该目标在场景中的位置。

[tform, inlierIdx] = estgeotform2d(matchedBoxPoints, matchedScenePoints, "affine");
inlierBoxPoints   = matchedBoxPoints(inlierIdx, :);
inlierScenePoints = matchedScenePoints(inlierIdx, :);

显示剔除离群点后的匹配点对

figure;
showMatchedFeatures(boxImage, sceneImage, inlierBoxPoints, ...
    inlierScenePoints, "montage");
title("Matched Points (Inliers Only)");

获取参考图像的边界多边形。

boxPolygon = [1, 1;...                           % top-left
        size(boxImage, 2), 1;...                 % top-right
        size(boxImage, 2), size(boxImage, 1);... % bottom-right
        1, size(boxImage, 1);...                 % bottom-left
        1, 1];                   % top-left again to close the polygon

将多边形转换到目标图像的坐标系中。变换后多边形表示该目标在场景中的位置。

newBoxPolygon = transformPointsForward(tform, boxPolygon);

显示检测到的对象。

figure;
imshow(sceneImage);
hold on;
line(newBoxPolygon(:, 1), newBoxPolygon(:, 2), Color="y");
title("Detected Box");

步骤 6：检测另一个目标

按照与之前相同的步骤检测第二个目标。

读取包含第二个目标对象的图像。

elephantImage = imread("elephant.jpg");
figure;
imshow(elephantImage);
title("Image of an Elephant");

检测并可视化点特征。

elephantPoints = detectSIFTFeatures(elephantImage);
figure;
imshow(elephantImage);
hold on;
plot(selectStrongest(elephantPoints, 100));
title("100 Strongest Point Features from Elephant Image");

提取特征描述符。

[elephantFeatures, elephantPoints] = extractFeatures(elephantImage, elephantPoints);

匹配特征。

elephantPairs = matchFeatures(elephantFeatures, sceneFeatures, MatchThreshold=40,...
    MaxRatio=0.7, Unique=true);

显示推定匹配的特征。

matchedElephantPoints = elephantPoints(elephantPairs(:, 1), :);
matchedScenePoints = scenePoints(elephantPairs(:, 2), :);
figure;
showMatchedFeatures(elephantImage, sceneImage, matchedElephantPoints, ...
    matchedScenePoints, "montage");
title("Putatively Matched Points (Including Outliers)");

估计几何变换并剔除异常值。

[tform, inlierElephantPoints, inlierScenePoints] = ...
    estimateGeometricTransform(matchedElephantPoints, matchedScenePoints, "affine");
figure;
showMatchedFeatures(elephantImage, sceneImage, inlierElephantPoints, ...
    inlierScenePoints, "montage");
title("Matched Points (Inliers Only)");

同时显示这两个对象。

elephantPolygon = [1, 1;...                                 % top-left
        size(elephantImage, 2), 1;...                       % top-right
        size(elephantImage, 2), size(elephantImage, 1);...  % bottom-right
        1, size(elephantImage, 1);...                       % bottom-left
        1,1];                         % top-left again to close the polygon

newElephantPolygon = transformPointsForward(tform, elephantPolygon);

figure;
imshow(sceneImage);
hold on;
line(newBoxPolygon(:, 1), newBoxPolygon(:, 2), Color="y");
line(newElephantPolygon(:, 1), newElephantPolygon(:, 2), Color="g");
title("Detected Elephant and Box");