Calibrate Multi-Sensor Systems

R2026b
Estimate the position and orientation between cameras, lidars, IMUs, and perform robot hand-eye calibration

The Multi‑Sensor Calibration Tools library provides algorithms, apps, and pipelines for estimating extrinsic parameters between cameras, lidars, and IMUs and performing robot hand-eye calibration. You can generate and detect calibration patterns, estimate intrinsic and extrinsic parameters for single and multiple sensors, and visualize calibration quality using 3‑D views, reprojection errors, and cross‑sensor overlays. The toolbox includes best‑practice guides and methodology pages that describe how to collect calibration data, set up sensors, and calibrate multi‑camera, lidar‑camera, camera-IMU, and hand‑eye systems. You can use the results, stored in parameter objects, to project data between sensor frames and integrate calibrated sensor poses into perception, mapping, and fusion pipelines.

To begin using the Multi‑Sensor Calibration Tools library, install it using the installMultiSensorCalibrationTools function.

Apps

Lidar Camera CalibratorFind rotation and translation between lidar and camera
Multi-Camera CalibratorCalibrate multiple cameras (Since R2026b)

Functions

expand all

installMultiSensorCalibrationToolsInstall Multi-Sensor Calibration Tools library in MATLAB (Since R2026b)
detectMultiPatternPointsDetect keypoints of multiple calibration patterns (Since R2026a)
detectPatternPointsDetect keypoints of calibration pattern in images from multiple cameras (Since R2025a)
estimateCameraRobotTransformEstimate pose of camera relative to robot using hand-eye calibration (Since R2025a)
estimateMultiCameraParametersCalibrate extrinsic parameters of multiple cameras with overlapping views (Since R2025a)
Multi-Camera CalibratorCalibrate multiple cameras (Since R2026b)

Pattern Generators

patternWorldPointsGenerate world point locations of a camera calibration pattern (Since R2024b)

Pattern Detectors

detectCheckerboardPointsDetect checkerboard pattern in image
detectAprilGridPointsDetect keypoints of AprilGrid pattern in images (Since R2024b)
detectCircleGridPointsDetect circle grid pattern in images
detectCharucoBoardPointsDetect ChArUco board pattern in images (Since R2024b)
detectPatternPointsDetect keypoints of calibration pattern in images from multiple cameras (Since R2025a)
detectMultiPatternPointsDetect keypoints of multiple calibration patterns (Since R2026a)

Estimate Multi-Camera Extrinsic Parameters

estimateMultiCameraParametersCalibrate extrinsic parameters of multiple cameras with overlapping views (Since R2025a)

Estimate Lidar-Camera Extrinsic Parameters

estimateBoardCornersCameraEstimate corners of calibration board in camera frame (Since R2026a)
estimateBoardCornersLidarEstimate corners of calibration board in lidar frame (Since R2026a)
estimateLidarCameraTransformEstimate rigid transformation from lidar sensor to camera

Estimate Robot-to-Camera Extrinsic Parameters

estimateCameraRobotTransformEstimate pose of camera relative to robot using hand-eye calibration (Since R2025a)

Estimate Camera-to-IMU Extrinsic Parameters

estimateCameraIMUTransformEstimate transformation from camera to IMU sensor using calibration data (Since R2024a)

Estimate Camera-to-Vehicle Extrinsic Parameters

estimateMonoCameraParametersEstimate extrinsic monocular camera parameters using calibration pattern
estimateMonoCameraFromSceneEstimate monocular camera parameters from scene geometry (Since R2022b)
bboxCameraToLidarEstimate 3-D bounding boxes in point cloud from 2-D bounding boxes in image
bboxLidarToCameraEstimate 2-D bounding boxes in image from 3-D bounding boxes in point cloud
multiSensorParametersStore multiple sensor mounting poses and intrinsic parameters (Since R2026b)
multiCameraParametersStore multi-camera system parameters (Since R2025a)
cameraIntrinsicsObject for storing intrinsic camera parameters
cameraIntrinsicsKBIntrinsic camera parameters based on Kannala-Brandt model (Since R2024a)
fisheyeIntrinsicsObject for storing intrinsic fisheye camera parameters
factorIMUParametersFactor IMU parameters (Since R2023a)
se3SE(3) homogeneous transformation (Since R2026a)
pcshowPlot 3-D point cloud
pcviewerVisualize and inspect large 3-D point clouds (Since R2023a)
pcplayerVisualize streaming 3-D point cloud data
plotCameraPlot camera in 3-D coordinates
pointCloudSparse 3-D scene point cloud reconstructed using structure-from-motion (SfM) (Since R2026b)
showExtrinsicsVisualize extrinsic camera parameters
showReprojectionErrorsVisualize calibration errors
projectLidarPointsOnImageProject lidar points onto an image frame
fuseCameraToLidarFuse image information to lidar point cloud
rotmat2vec3dConvert 3-D rotation matrix to rotation vector (Since R2022b)
rotvec2mat3dConvert 3-D rotation vector to rotation matrix (Since R2022b)
cameraIntrinsicsToOpenCVConvert camera intrinsic parameters from MATLAB to OpenCV
cameraIntrinsicsFromOpenCVConvert camera intrinsic parameters from OpenCV to MATLAB
stereoParametersToOpenCVConvert stereo camera parameters from MATLAB to OpenCV
stereoParametersFromOpenCVConvert stereo camera parameters from OpenCV to MATLAB

Topics

Get Started

Data Collection and Preparation

Calibrate Using Apps

Calibrate Multi-Sensor System

Featured Examples

3-D Motion Reconstruction Using Multiple Cameras

3-D Motion Reconstruction Using Multiple Cameras

Reconstruct 3-D motion of an object for use in a motion capture system consisting of multiple cameras.

Create 360° Bird's-Eye-View Image Around a Vehicle

Create 360° Bird's-Eye-View Image Around a Vehicle

Create a 360° bird's-eye-view image around a vehicle for use in a surround view monitoring system. It then shows how to generate code for the same bird's-eye-view image creation algorithm and verify the results.

(Automated Driving Toolbox)

Validate Calibration by Building a Colorized 3-D Map Using MUN-FRL Dataset

Validate Calibration by Building a Colorized 3-D Map Using MUN-FRL Dataset

Validate multi-sensor calibration by building a colorized 3-D map from flight data. The workflow fuses lidar, IMU, and camera data using the calibrated parameters, providing an end-to-end check that complements the static projection-based validation. While static validation checks pairwise extrinsics on calibration data, map building uses all parameters under real operating conditions including vibration and rapid motion.

Estimate Pose of Moving Camera Mounted on a Robot

Estimate Pose of Moving Camera Mounted on a Robot

Perform and verify hand-eye calibration for a robot arm equipped with a camera.

Estimate Pose of Fixed Camera Relative to Robot Base

Estimate Pose of Fixed Camera Relative to Robot Base

Estimate the camera intrinsics for a fixed camera using hand-to-eye calibration.

(Robotics System Toolbox)

  • Since R2024b
New
Track Objects Across Multiple Cameras Using Ground-Plane Fusion

Track Objects Across Multiple Cameras Using Ground-Plane Fusion

Track vehicles and pedestrians across calibrated cameras by fusing ground-plane projections into globally consistent track IDs.