Detection and Tracking

Object detection, shape fitting, and tracking in lidar point cloud data

Object detection is a technique that identifies and locates objects in a scene. This enables you to detect 3-D objects in a point cloud. Lidar Toolbox™ includes functionality that enables you to detect objects using geometric shape fitting or deep learning with convolutional neural networks.

Geometric shape fitting — Detect the 3-D geometry of the objects in the point cloud by using ground segmentation and plane-fitting algorithms. You can detect the location, dimensions and direction of each object. You can use the detected objects for downstream workflows such as tracking, path planning and labeling.

Deep learning — A deep learning approach to object detection uses convolutional neural networks to perform object detection. Lidar Toolbox includes object detection workflows that use neural networks such as PointPillars and Complex-YOLO v4. You can train a custom object detection model, or use the available pretrained networks and further tune it for your application. The toolbox also supports CUDA^® MEX code generation for PointPillars and SqueezeSegV2 networks.

Object tracking is a technique that estimates and tracks the movement of objects across multiple scans of a scene. Object tracking consists of assigning a unique ID to detected objects and tracking their movement across point cloud frames. Lidar Toolbox includes detection and tracking workflows for vehicles, road lanes, and curbs. Most of these workflows use the joint probabilistic data association (JPDA) tracker.

Deep learning-based object detection in lidar point clouds.

Functions

expand all

Geometrical

Shape Fitting

`pcfitcuboid`	Fit cuboid over point cloud (Since R2020b)
`pcfitplane`	Fit plane to 3-D point cloud

Geometric Models

`planeModel`	Object for storing parametric plane model
`cuboidModel`	Parametric cuboid model (Since R2020b)

Deep Learning

Load Training Data

`groundTruth`	Ground truth label data
`combine`	Combine data from multiple datastores
`fileDatastore`	Datastore with custom file reader
`boxLabelDatastore`	Datastore for bounding box label data (Since R2019b)

Augment and Preprocess Training Data

`randomAffine3d`	Create randomized 3-D affine transformation (Since R2019b)
`bboxwarp`	Apply geometric transformation to bounding boxes (Since R2019b)
`pctransform`	Transform 3-D point cloud

Object Detection

`pointPillarsObjectDetector`	PointPillars object detector (Since R2021b)
`trainPointPillarsObjectDetector`	Train PointPillars object detector (Since R2021b)
`lidarLaneDetector`	Create lane detector for lidar data (Since R2023b)
`detect`	Detect objects using PointPillars object detector (Since R2021b)
`detectLOAMFeatures`	Detect LOAM feature points from 3-D lidar data (Since R2022a)
`lidarObjectDetectorTrainingData`	Create training data for lidar object detection (Since R2022a)

Visualize Results

`showShape`	Display shapes on image, video, or point cloud (Since R2020b)
`pcshow`	Plot 3-D point cloud

Evaluate Results

`evaluateObjectDetection`	Evaluate object detection data set against ground truth (Since R2023b)
`objectDetectionMetrics`	Object detection quality metrics (Since R2023b)
`bboxOverlapRatio`	Compute bounding box overlap ratio

Topics

Deep Learning with Point Clouds
Learn point cloud processing using deep learning.
Get Started with PointPillars
Define PointPillars network and learn how to perform object detection using the same.
Datastores for Deep Learning (Deep Learning Toolbox)
Learn how to use datastores in deep learning applications.
List of Deep Learning Layers (Deep Learning Toolbox)
Discover all the deep learning layers in MATLAB^®.

Featured Examples

Determine Safe Landing Area for Aerial Vehicles

Demonstrates how to determine a safe landing area for aerial vehicles, such as helicopters and UAVs, by using aerial lidar data.

Open Live Script

Lidar Object Detection Using Complex-YOLO v4 Network

Detect objects in point clouds using you only look once version 4 (YOLO v4) deep learning network. In this example, you will

Open Live Script

Code Generation for Lidar Object Detection Using SqueezeSegV2 Network

Generate CUDA® MEX code for a lidar object detection network. In the example, you first segment the point cloud with a pretrained network, then cluster the points and fit 3-D bounding boxes to each cluster. Finally, you generate MEX code for the network.

Open Live Script

Lidar 3-D Object Detection Using PointPillars Deep Learning

Train a PointPillars network for object detection in point clouds.

Open Live Script

Code Generation for Lidar Object Detection Using PointPillars Deep Learning

Generate CUDA® MEX for a PointPillars object detector. For more information, see Lidar 3-D Object Detection Using PointPillars Deep Learning example from the Lidar Toolbox™.

Open Live Script

Data Augmentations for Lidar Object Detection Using Deep Learning

Perform typical data augmentation techniques for 3-D object detection workflows with lidar data.

Open Live Script

Lane Detection in 3-D Lidar Point Cloud

Detect lanes in lidar point clouds. You can use the intensity values returned from lidar point clouds to detect ego vehicle lanes. You can further improve the lane detection by using a curve-fitting algorithm and tracking the curve parameters. Lidar lane detection enables you to build complex workflows like lane keep assist, lane departure warning, and adaptive cruise control for autonomous driving. A test vehicle collects the lidar data using a lidar sensor mounted on its rooftop.

Open Live Script

Curb Detection and Tracking in 3-D Lidar Point Cloud

Detect and track curbs in lidar point clouds. A curb is a line of stone or concrete, that connects the roadway to the sidewalk. Curbs act as delimiters for the drivable area of the road.

Open Live Script

Detect, Classify, and Track Vehicles Using Lidar

Detect, classify, and track vehicles by using lidar point cloud data captured by a lidar sensor mounted on an ego vehicle.

Open Live Script

Track Vehicles Using Lidar: From Point Cloud to Track List

Track vehicles using measurements from a lidar sensor mounted on top of an ego vehicle. Lidar sensors report measurements as a point cloud. The example illustrates the workflow in MATLAB® for processing the point cloud and tracking the objects. For a Simulink® version of the example, refer to Track Vehicles Using Lidar Data in Simulink (Sensor Fusion and Tracking Toolbox).The lidar data used in this example is recorded from a highway driving scenario. In this example, you use the recorded data to track vehicles with a joint probabilistic data association (JPDA) tracker and an interacting multiple model (IMM) approach.

Open Script

Track-Level Fusion of Radar and Lidar Data

Generate an object-level track list from measurements of a radar and a lidar sensor and further fuse them using a track-level fusion scheme. You process the radar measurements using an extended object tracker and the lidar measurements using a joint probabilistic data association (JPDA) tracker. You further fuse these tracks using a track-level fusion scheme. The schematic of the workflow is shown below.

Open Script