Main Content

bboxLidarToCamera

Estimate 2-D bounding box in camera frame using 3-D bounding box in lidar frame

Since R2021a

collapse all in page

Syntax

bboxesCamera = bboxLidarToCamera(bboxesLidar,intrinsics,tform)
bboxesCamera = bboxLidarToCamera(bboxesLidar,intrinsics,tform,L)
[bboxesCamera,boxesUsed] = bboxLidarToCamera(___)
[___] = bboxLidarToCamera(___,'ProjectedCuboid',true)

Description

bboxesCamera = bboxLidarToCamera(bboxesLidar,intrinsics,tform) estimates 2-D bounding boxes in the camera frame from 3-D bounding boxes in the lidar frame bboxesLidar. The function uses the camera intrinsic parameters intrinsics and a lidar to camera transformation matrix tform.

bboxesCamera = bboxLidarToCamera(bboxesLidar,intrinsics,tform,L) further refines the 2-D bounding boxes to the edges of the object inside it using L. L is the corresponding labeled 2-D image of the 2-D bounding boxes, where the objects are labeled distinctively.

[bboxesCamera,boxesUsed] = bboxLidarToCamera(___) indicates for which of the specified 3-D bounding boxes the function detects a corresponding 2-D bounding box in the camera frame.

[___] = bboxLidarToCamera(___,'ProjectedCuboid',true) returns 3-D projected cuboids instead of 2-D bounding boxes.

example

Examples

collapse all

Open Live Script

Load ground truth data from a MAT file into the workspace. Extract the image, point cloud, and camera intrinsic parameters from the ground truth data. 

dataPath = fullfile(toolboxdir('lidar'),'lidardata','lcc','bboxGT.mat');
gt = load(dataPath);
im = gt.im;
pc = gt.pc;
intrinsics = gt.cameraParams;

Extract the lidar to camera transformation matrix from the ground truth data.

tform = gt.camToLidar.invert;

Extract the 3-D bounding box information.

bboxLidar = gt.cuboid1;

Estimate the 2-D bounding box on the image.

bboxesCamera = bboxLidarToCamera(bboxLidar,intrinsics,tform);

Display the 3-D bounding box overlaid on the point cloud.

pcshow(pc.Location,pc.Location(:,1))
showShape('cuboid',bboxLidar)

Figure contains an axes object. The axes object contains an object of type scatter.

Display the 2-D bounding box overlaid on the image.

J = undistortImage(im,intrinsics);
annotatedImage = insertObjectAnnotation(J,'Rectangle',bboxesCamera,'Vehicle');
imshow(annotatedImage)

Figure contains an axes object. The hidden axes object contains an object of type image.

Open Live Script

Load ground truth data from a MAT file into the workspace. Extract the image, point cloud, and camera intrinsic parameters from the ground truth data. 

dataPath = fullfile(toolboxdir('lidar'),'lidardata','lcc','bboxGT.mat');
gt = load(dataPath);
im = gt.im;
pc = gt.pc;
intrinsics = gt.cameraParams;

Extract the lidar to camera transformation matrix from the ground truth data.

tform = gt.camToLidar.invert;

Extract the 3-D bounding box information.

bboxLidar = gt.cuboid2;

Estimate the projected 3-D bounding box on the image.

bboxesCamera = bboxLidarToCamera(bboxLidar,intrinsics,tform, ...
        'ProjectedCuboid',true);

Display the 3-D bounding box overlaid on the point cloud.

figure
pcshow(pc.Location,pc.Location(:,1))
showShape('cuboid',bboxLidar)

Figure contains an axes object. The axes object contains an object of type scatter.

Display the 3-D projected bounding box overlaid on the image.

J = undistortImage(im,intrinsics);
h = imshow(J);
pcH = vision.roi.ProjectedCuboid;
pcH.Parent = h.Parent;
pcH.Position = bboxesCamera;
pcH.InteractionsAllowed = 'None';

Figure contains an axes object. The hidden axes object contains 2 objects of type image, vision.roi.projectedcuboid.

Input Arguments

collapse all

3-D bounding boxes in the lidar frame, specified as a cuboidModel object or an N-by-9 matrix of real values. N is the number of 3-D bounding boxes. Each row of the matrix has the form [xctr yctr zctr xlen ylen zlen xrot yrot zrot].

  • xctr, yctr, and zctr — These values specify the x-, y-, and z-axis coordinates, respectively, of the center of the cuboid bounding box.

  • xlen, ylen, and zlen — These values specify the length of the cuboid along the x-, y-, and z-axis, respectively, before it is rotated.

  • xrot, yrot, and zrot — These values specify the rotation angles of the cuboid around the x-, y-, and z-axis, respectively. These angles are clockwise-positive when you look in the forward direction of their corresponding axes.

This figure shows how these values determine the position of a cuboid.

Note

The function assumes that the point cloud data that corresponds to the 3-D bounding boxes and the image data are time synchronized.

Data Types: single | double

Camera intrinsic parameters, specified as a cameraIntrinsics object.

Camera to lidar rigid transformation, specified as a rigidtform3d object.

Labeled 2-D image, specified as a matrix of real values. The matrix size is the same as the ImageSize property of intrinsics.

Note

Labeled images are assumed to be undistorted.

Data Types: single | double | int8 | int16 | uint8 | uint16

Output Arguments

collapse all

2-D bounding boxes in the camera frame, returned as an M-by-4 matrix of real values. M is the number of detected bounding boxes. Each row of the matrix contains the location and size of a rectangular bounding box in the form [x y width height]. The x and y elements specify the x and y coordinates, respectively, for the upper-left corner of the rectangle. The width and height elements specify the size of the rectangle.

If 'ProjectedCuboid' is set to true, the 2-D bounding boxes are returned as an M-by-8 matrix of real values. The bounding boxes have a cuboid shape and enclose the object. Each row of the matrix contains the size and location of the cuboid bounding box in the form [frontFace backFace]. Both the faces are represented as 2-D bounding boxes.

Data Types: single | double

Bounding box detection flag, returned as an N-element row vector of logicals. 2 is the number of input 3-D bounding boxes. If the function detects a corresponding 2-D bounding box in the camera frame, then it returns a value of true for that input 3-D bounding box. If the function does not detect a corresponding 2-D bounding box, then it returns a value of false.

Data Types: logical

Version History

Introduced in R2021a

expand all

See Also

Functions