detectMultiPatternPoints

Detect keypoints of multiple calibration patterns

Since R2026a

collapse all in page

Syntax

imagePoints = detectMultiPatternPoints(imageFileNames,patternCount,"charuco-board",patternDims,markerFamily,checkerSize,markerSize)
imagePoints = detectMultiPatternPoints(imageFileNames,patternCount,"aprilgrid",patternDims,tagFamily)
imagePoints = detectMultiPatternPoints(imageFileNames,patternCount,patternDetector,numKeyPoints)
imagePoints = detectMultiPatternPoints(___,Name=Value)

Description

Natively Supported Patterns

imagePoints = detectMultiPatternPoints(imageFileNames,patternCount,"charuco-board",patternDims,markerFamily,checkerSize,markerSize) detects the keypoints of two or more ChArUco board calibration patterns in images imageFileNames, with dimensions specified by patternDims, markerFamily, and checkerSize. patternCount specifies the number of patterns to detect.

example

imagePoints = detectMultiPatternPoints(imageFileNames,patternCount,"aprilgrid",patternDims,tagFamily) detects the keypoints of two or more AprilGrid calibration patterns in images imageFileNames, with dimensions specified by patternDims.

Custom Pattern Detector

imagePoints = detectMultiPatternPoints(imageFileNames,patternCount,patternDetector,numKeyPoints) detects the specified number of uniquely detectable keypoints numKeyPoints of two or more custom calibration patterns of same type that can be detected in an image using the specified detector function patternDetector.

Optional Arguments

imagePoints = detectMultiPatternPoints(___,Name=Value) specifies options using one or more name-value arguments in addition to any combination of arguments from previous syntaxes. For example, Verbose=true displays the progress of the detection process.

Examples

collapse all

Open Live Script

Estimate the relative position and orientation of eight cameras with overlapping fields of view by using calibration images that contain multiple ChArUco boards.

Download the calibration images.

calibImagesURL = "https://www.mathworks.com/supportfiles/vision/data/overlapping-cameras-multiple-charuco.zip";
calibImagesDir = fullfile(pwd,"overlapping-cameras-multiple-charuco");
calibImagesZip = fullfile(pwd,"overlapping-cameras-multiple-charuco.zip");
if ~exist(calibImagesZip,"file")
    disp("Downloading calibration images (86 MB)...")
    websave(calibImagesZip,calibImagesURL);
end
if ~exist(calibImagesDir,"dir")
    unzip(calibImagesZip,pwd)
end

Specify calibration image filenames for each camera.

numCameras = 8;
camDirPrefix = "Cam00";
imageFiles = cell(1,numCameras);
for i = 1:numCameras
    camDir = fullfile(calibImagesDir,camDirPrefix+i);
    imds = imageDatastore(camDir);
    imageFiles{i} = imds.Files;
end
imageFiles = [imageFiles{:}];

Define the ChArUco board properties. Specify checker size and marker size in centimeters.

markerFamily = "DICT_6X6_1000";
patternDims = [5 5];
markerSize = 6.8;   % in cm
checkerSize = 9.15; % in cm

Detect the keypoints of the eight ChArUco boards captured in the calibration images using the detectMultiPatternPoints function. Specify the MinMarkerID for each pattern.

patternCount = 8;
patternIDs = 144:12:228;

imagePoints = detectMultiPatternPoints(imageFiles,patternCount,"charuco-board",patternDims,markerFamily, ...
    checkerSize,markerSize,MinMarkerID=patternIDs);
[==================================================] 100%
Elapsed time: 00:00:14
Estimated time remaining: 00:00:00

Display the detected keypoints in a couple of views and three of the eight cameras.

t = tiledlayout(2,3,TileSpacing="compact",Padding="compact");
colors = ["red","green","cyan","black","magenta","yellow","blue","white"];
for viewIdx = [19,24]
    for camIdx = [1,5,7]
        nexttile
        imshow(imageFiles{viewIdx,camIdx})
        title("Camera " + camIdx + " | View " + viewIdx)
        hold on
        for patternIdx = 1:patternCount
            plot(imagePoints(:,1,viewIdx,camIdx,patternIdx),imagePoints(:,2,viewIdx,camIdx,patternIdx),"yo",...
                MarkerSize=3,MarkerFaceColor=colors(patternIdx))
        end
    end
end
title(t,"Detected keypoints in multiple ChArUco boards")

Figure contains 6 axes objects. Hidden axes object 1 with title Camera 1 | View 19 contains 9 objects of type image, line. One or more of the lines displays its values using only markers Hidden axes object 2 with title Camera 5 | View 19 contains 9 objects of type image, line. One or more of the lines displays its values using only markers Hidden axes object 3 with title Camera 7 | View 19 contains 9 objects of type image, line. One or more of the lines displays its values using only markers Hidden axes object 4 with title Camera 1 | View 24 contains 9 objects of type image, line. One or more of the lines displays its values using only markers Hidden axes object 5 with title Camera 5 | View 24 contains 9 objects of type image, line. One or more of the lines displays its values using only markers Hidden axes object 6 with title Camera 7 | View 24 contains 9 objects of type image, line. One or more of the lines displays its values using only markers

Generate the world coordinates of each pattern points in their respective pattern coordinate system.

worldPoints = patternWorldPoints("charuco-board",patternDims,checkerSize);

Load the intrinsic parameters of the eight cameras. These parameters have been estimated using the Using the Single Camera Calibrator App.

ld = load("eightCameraIntrinsics.mat");

Perform multi-camera calibration.

params = estimateMultiCameraParameters(imagePoints,worldPoints,ld.intrinsics,WorldUnits="cm");

Plot the estimated camera poses.

figure
plotCamera(params.CameraPoses)
axis equal
grid on
view([0 1 0])

Figure contains an axes object. The axes object contains 80 objects of type line, text, patch.

Open Live Script

Calibrate multiple cameras that have some cameras sharing fields-of-view and some cameras not sharing any field-of-view.

Download the calibration images containing two ChArUco boards that were captured from six cameras as per the guidelines outlined in Prepare Cameras and Capture Images for Multi-Camera Calibration. The calibration images were captured by moving the multi-camera system around the stationary ChArUco boards.

calibImagesURL = "https://www.mathworks.com/supportfiles/vision/data/non-overlapping-cameras-charuco.zip";
calibImagesDir = fullfile(pwd,"non-overlapping-cameras-charuco");
calibImagesZip = fullfile(pwd,"non-overlapping-cameras-charuco.zip");
if ~exist(calibImagesZip,"file")
    disp("Downloading calibration images (68 MB)...")
    websave(calibImagesZip,calibImagesURL);
end
if ~exist(calibImagesDir,"dir")
    unzip(calibImagesZip,pwd)
end

Specify calibration image filenames for each camera.

numCameras = 6;
camDirPrefix = "Cam00";
imageFiles = cell(1,numCameras);
for i = 1:numCameras
    camDir = fullfile(calibImagesDir,camDirPrefix+i);
    imds = imageDatastore(camDir);
    imageFiles{i} = imds.Files;
end
imageFiles = [imageFiles{:}];

Display images captured by the six cameras from one of the views.

figure
t = tiledlayout(2,3,TileSpacing="tight", Padding="tight");
for i = 1:numCameras
    nexttile
    imshow(imageFiles{25,i})
    title("Camera " + i)
end

Define the ChArUco board properties. Specify checker size and marker size in centimeters.

markerFamily = "DICT_6X6_1000";
patternDims = [5 5];
markerSize = 2.55;   % in cm
checkerSize = 3.40; % in cm

Detect the key points of the two ChArUco boards captured in the calibration images using the detectMultiPatternPoints function.

patternCount = 2;
imagePoints = detectMultiPatternPoints(imageFiles,patternCount,"charuco-board",patternDims,markerFamily, ...
    checkerSize,markerSize);
[==================================================] 100%
Elapsed time: 00:00:09
Estimated time remaining: 00:00:00

Generate the world coordinates of each pattern points in their respective pattern coordinate system.

worldPoints = patternWorldPoints("charuco-board",patternDims,checkerSize);

Load the intrinsic parameters of the six cameras. These parameters have been estimated using the Using the Single Camera Calibrator App.

ld = load("sixCameraIntrinsicsNoOverlap.mat");

Perform multi-camera calibration. Enable verbose mode to track the progress of the calibration process.

params = estimateMultiCameraParameters(imagePoints,worldPoints,ld.intrinsics,WorldUnits="cm",Verbose=true);
Estimating multi-camera extrinsic parameters
--------------------------------------------
* Number of cameras: 6
* Number of patterns: 2
* Number of views: 30
* Number of images used: 110

* Undistorting image points...done.

* Grouping cameras with overlapping field-of-view....done.
* Group 1: camera 3 4 5 6
* Group 2: camera 1 2

* Refining camera poses of each group:
* Group 1...done.
* Group 2...done.

* Estimating relative pose between camera groups:
* [Pair 1/1] Group 1 and 2...done.

* Refining all camera poses and pattern poses...done.
* Finished calibrating cameras.

Visualize the camera extrinsic parameters along with the patterns from a couple of views.

figure
showExtrinsics(params,"CameraCentric",ViewIndex=21:22)

Figure contains an axes object. The axes object with title Extrinsic Parameters Visualization, xlabel X (cm), ylabel Z (cm) contains 24 objects of type patch, text, line.

Input Arguments

collapse all

Filenames of the camera calibration patterns, specified as a numViews-by-numCameras string or cell array. numViews is the number of calibration pattern views for each camera, and numCameras is the number of cameras. Because you must specify more than two cameras, numCameras must be greater than 2. Any missing views can be specified by an empty string in the array.

Organize your calibration data into folders that contain synchronized images for each camera, ensuring that the filenames are consistent across all camera folders. This arrangement aligns images from different cameras that correspond to the same scene or subject. By using identical filenames in the folder for each camera, you can ensure that the images are properly synchronized.

Number of unique calibration patterns captured in the calibration images, specified as a scalar. The patternCount value must be greater than 1. For single pattern, you can use the detectPatternPoints function to detect keypoints.

Pattern dimensions, specified as a two-element vector that represents the number of checkers in the rows (dim1) and columns (dim2) of the pattern. For an AprilGrid pattern, ensure that the pattern is oriented with the origin at the top-left as you count the rows and columns. For more information on patterns, see Calibration Patterns.

ChArUco board checker box side length, specified as a scalar in world units, such as millimeters or inches. You must specify checkerSize and markerSize in the same world units, and checkerSize must be a larger value than markerSize.

Size of ArUco marker, specified as a positive integer. This value sets the length of one side of a square ArUco marker in world units such as millimeters, centimeters, meters, or inches. The value of checkerSize must be greater than the value of markerSize.

ArUco marker family, specified as one of these options:

  • "DICT_ARUCO_ORIGINAL"

  • "DICT_4X4_1000"

  • "DICT_5X5_1000"

  • "DICT_6X6_1000"

  • "DICT_7X7_1000"

AprilTag family, specified as "tag36h11", "tag25h9", or "tag16h5".

Custom pattern detector, specified as a function handle with this syntax:

[imagePoints,hasDetected] = patternDetector(filename,patternID).

Input

  • filename — Name of the image to be processed, specified as a string scalar or character vector.

  • patternID — Scalar identifier assigned to the pattern to be detected. This serves the same purpose as the minMarkerID of a ChArUco board or an AprilGrid pattern.

Output

  • imagePoints — Coordinates of the detected keypoints in the image, returned as a numKeyPoints-by-2 matrix. N is the number of keypoints.

  • hasDetected — Logical that indicates whether the function detected the pattern, returned as true when the function detects the pattern and false when it does not.

Number of keypoints that can be detected in the custom pattern, specified as a positive integer.

Name-Value Arguments

collapse all

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Example: detectMultiPatternPoints(imageFileNames,patternCount,"charuco-board",patternDims,markerFamily,checkerSize,markerSize,Verbose=1) enables progress display to the screen.

Minimum marker ID to detect in each pattern, specified as an N-element vector of non-negative integers. The number of elements N is equal to the number of calibration patterns, patternCount.

Display detection progress information in the Command Window, specified as a numeric or logical 1 (true) or 0 (false). Set Verbose to true to display progress information.

Output Arguments

collapse all

Image coordinates for the keypoints found in each image, returned as a numKeyPoints-by-2-numViews-by-numCameras-by-patternCount array.

  • numKeyPoints is the number of keypoints found for each image.

  • numViews is the number of views with respect to the camera.

  • numCameras is the number of cameras. For example, imagePoints(:,:,i,j,k) contains the points from camera j when the kth pattern is located at the ith view.

  • patternCount is the number of unique calibration patterns captured in the calibration images.

For images where a camera fails to detect a pattern, the corresponding keypoint values are assigned as NaNs (Not a Number).

Extended Capabilities

expand all

Version History

Introduced in R2026a

See Also

Objects

Functions

Topics