Tracking and Sensor Fusion

Object tracking and multisensor fusion, bird’s-eye plot of detections and object tracks

You can create a multi-object tracker to fuse information from radar and video camera sensors. The tracker uses Kalman filters that let you estimate the state of motion of a detected object. Use the sensor measurements made on a detected object to continuously solve for the position and velocity of that object. To track moving objects, you can use constant-velocity or constant-acceleration motion models, or you can define your own models.

Functions

expand all

Multi-Object Tracking

`multiObjectTracker`	Track objects using GNN assignment
`objectDetection`	Report for single object detection
`getTrackPositions`	Returns updated track positions and position covariance matrix
`getTrackVelocities`	Obtain updated track velocities and velocity covariance matrix
`confirmTrack`	Confirm tentative track (Since R2022b)
`objectTrack`	Single object track report (Since R2020a)
`trackHistoryLogic`	Confirm and delete tracks based on recent track history (Since R2020a)

Kalman Filters for Tracking

Alpha-Beta Filter

`trackingABF`	Alpha-beta filter for object tracking (Since R2020a)
`initcaabf`	Create constant acceleration alpha-beta tracking filter from detection report (Since R2020a)
`initcvabf`	Create constant velocity tracking alpha-beta filter from detection report (Since R2020a)

Linear Kalman Filter

`trackingKF`	Linear Kalman filter for object tracking
`initcakf`	Create constant-acceleration linear Kalman filter from detection report
`initcvkf`	Create constant-velocity linear Kalman filter from detection report

Extended Kalman Filter

`trackingEKF`	Extended Kalman filter for object tracking
`initcaekf`	Create constant-acceleration extended Kalman filter from detection report
`initctekf`	Create constant turn-rate extended Kalman filter from detection report
`initcvekf`	Create constant-velocity extended Kalman filter from detection report

Unscented Kalman Filter

`trackingUKF`	Unscented Kalman filter for object tracking
`initcaukf`	Create constant-acceleration unscented Kalman filter from detection report
`initctukf`	Create constant turn-rate unscented Kalman filter from detection report
`initcvukf`	Create constant-velocity unscented Kalman filter from detection report

Motion Models

Constant Velocity

`constvel`	State transition function for constant-velocity motion model
`constveljac`	Jacobian of state transition function based on constant-velocity motion model
`cvmeas`	Measurement function for constant-velocity motion model
`cvmeasjac`	Jacobian of measurement function for constant-velocity motion model

Constant Acceleration

`constacc`	State transition function for constant-acceleration motion model
`constaccjac`	Jacobian of state transition function based on constant-acceleration motion model
`cameas`	Measurement function for constant-acceleration motion model
`cameasjac`	Jacobian of measurement function for constant-acceleration motion model

Constant Turn-Rate

`constturn`	State transition function for constant turn-rate and velocity-magnitude motion model
`constturnjac`	Jacobian of state transition function based on constant turn-rate and velocity-magnitude motion
`ctmeas`	Measurement function for constant turn-rate and velocity-magnitude motion model
`ctmeasjac`	Jacobian of measurement function for constant turn-rate and velocity- magnitude motion model

Blocks

Multi-Object Tracker

Create and manage tracks of multiple objects

Topics

Multi-Object Tracking

Multiple Object Tracking Tutorial
Perform automatic detection and motion-based tracking of moving objects in a video by using a multi-object tracker.
Linear Kalman Filters
Estimate and predict object motion using a Linear Kalman filter.
Extended Kalman Filters
Estimate and predict object motion using an extended Kalman filter.
Convert Detections to objectDetection Format
These examples show how to convert actual detections in the native format of the sensor into objectDetection objects.

Sensor Fusion with Synthetic Data

Sensor Fusion Using Synthetic Radar and Vision Data
Generate a scenario, simulate sensor detections, and use sensor fusion to track simulated vehicles.
Sensor Fusion Using Synthetic Radar and Vision Data in Simulink
Implement a synthetic data simulation for tracking and sensor fusion in Simulink^® with Automated Driving Toolbox™.

Code Generation

Code Generation for Tracking and Sensor Fusion
Generate C code for a MATLAB^® function that processes data recorded from a test vehicle and tracks the objects around it.
Generate Code for a Track Fuser with Heterogeneous Source Tracks
Generate code for a track-level fusion algorithm where tracks originate from heterogeneous sources with different state definitions.

Featured Examples

Extended Object Tracking of Highway Vehicles with Radar and Camera

Track objects whose dimensions span multiple sensor resolution cells.

Open Script

Extended Object Tracking of Highway Vehicles with Radar and Camera in Simulink

Track highway vehicles around an ego vehicle in Simulink. In this example, you use multiple extended object tracking techniques to track highway vehicles and evaluate their tracking performance. This example closely follows the Extended Object Tracking of Highway Vehicles with Radar and Camera (Sensor Fusion and Tracking Toolbox) MATLAB® example.

Open Model

Grid-Based Tracking in Urban Environments Using Multiple Lidars

Track moving objects by using multiple lidar sensors and a grid-based tracker.

Open Live Script

Grid-based Tracking in Urban Environments Using Multiple Lidars in Simulink

Track moving objects with multiple lidars using a grid-based tracker in Simulink. You use the Grid-Based Multi Object Tracker (Sensor Fusion and Tracking Toolbox) Simulink block to define the grid-based tracker. This Grid-based tracker uses dynamic occupancy grid map as an intermediate representation of the environment. This example closely follows the Grid-Based Tracking in Urban Environments Using Multiple Lidars (Sensor Fusion and Tracking Toolbox) MATLAB® example.

Open Model

Track-to-Track Fusion for Automotive Safety Applications

Fuse tracks from two vehicles to provide a more comprehensive estimate of the environment than seen by each vehicle.

Open Live Script

Track-to-Track Fusion for Automotive Safety Applications in Simulink

Fuse tracks from multiple vehicles in the Simulink environment.

Open Model

Track Vehicles Using Lidar: From Point Cloud to Track List

Track vehicles from lidar data using a joint probabilistic data association (JPDA) tracker and an interacting multiple model (IMM) approach.

Open Script

Track Vehicles Using Lidar Data in Simulink

Track vehicles from lidar data using a JPDA tracker and IMM approach in the Simulink environment.

Open Script

Track Multiple Lane Boundaries with a Global Nearest Neighbor Tracker

Design and test a multiple-lane tracking algorithm.

Open Live Script

Track-Level Fusion of Radar and Lidar Data

Fuse tracks obtained by radar and lidar sensor measurements.

Open Script

Track-Level Fusion of Radar and Lidar Data in Simulink

Fuse tracks obtained by radar and lidar sensor measurements in Simulink.

Open Model

Object Tracking and Motion Planning Using Frenet Reference Path

Dynamically replan the motion of an autonomous vehicle based on the estimate of the surrounding environment. You use a Frenet reference path and a joint probabilistic data association (JPDA) tracker to estimate and predict the motion of other vehicles on the highway. Compared to the Highway Trajectory Planning Using Frenet Reference Path example, you use these estimated trajectories from the multi-object tracker in this example instead of ground truth for motion planning.

Open Live Script

Forward Vehicle Sensor Fusion

Simulate sensor fusion and tracking in a 3D simulation environment for automated driving applications.

Open Example

Automate Testing for Forward Vehicle Sensor Fusion

Automate the testing of a forward vehicle sensor fusion component and generated code.

Open Example

Automate Real-Time Testing for Forward Vehicle Sensor Fusion

Deploy and test forward vehicle sensor fusion component of highway lane following system in real-time.

Open Example

Surround Vehicle Sensor Fusion

Design sensor fusion and tracking component to detect vehicles using multiple vision and radar sensors, and generate fused tracks for surround view analysis.

Open Example

Asynchronous Sensor Fusion and Tracking with Retrodiction

Construct an event-based sensor fusion and tracking model in Simulink.

Open Model

Highway Vehicle Tracking with Multipath Radar Reflections

Assess and mitigate the impact of multipath radar reflections when tracking highway vehicles using an extended object tracker.

Open Live Script

Extended Target Tracking with Multipath Radar Reflections in Simulink

Model and mitigate multipath radar reflections in a highway driving scenario in Simulink®. It closely follows the Highway Vehicle Tracking with Multipath Radar Reflections MATLAB® example.

Open Live Script

Processor-in-the-Loop Verification of JPDA Tracker for Automotive Applications

Generate embedded code for a joint probabilistic data association (JPDA) tracker and verify it using processor-in-the-loop (PIL) simulations on a STM32 Nucleo board with 1 MB RAM and 2 MB flash memory. In this example, you configure the JPDA tracker to process detections from a camera and a radar sensor mounted in front of an ego vehicle in highway scenarios. For PIL simulations, you use simulated detections to verify the tracking and computational performance of the generated code.

Open Live Script