trackAssignmentMetrics

Track establishment, maintenance, and deletion metrics

Description

The trackAssignmentMetrics System object™ compares tracks from a multi-object tracking system against known truth by automatic assignment of tracks to the known truths at each track update. An assignment distance metric determines the maximum distance for which a track can be assigned to the truth object. A divergence distance metric determines when a previously assigned track can be reassigned to a different truth object when the distance exceeds another set threshold.

To generate track assignment metrics:

Create the trackAssignmentMetrics object and set its properties.
Call the object with arguments, as if it were a function.

To learn more about how System objects work, see What Are System Objects?

Creation

Syntax

assignmentMetrics = trackAssignmentMetrics

assignmentMetrics = trackAssignmentMetrics(Name,Value)

Description

assignmentMetrics = trackAssignmentMetrics creates a trackAssignmentMetrics System object, assignmentMetrics, with default property values.

assignmentMetrics = trackAssignmentMetrics(Name,Value) sets properties for the trackAssignmentMetrics object using one or more name-value pairs. For example, assignmentMetrics = trackAssignmentMetrics('AssignmentThreshold',5) creates a trackAssignmentMetrics object with an assignment threshold of 5. Enclose property names in quotes.

example

Properties

expand all

Unless otherwise indicated, properties are nontunable, which means you cannot change their values after calling the object. Objects lock when you call them, and the release function unlocks them.

If a property is tunable, you can change its value at any time.

For more information on changing property values, see System Design in MATLAB Using System Objects.

`AssignmentThreshold` — Maximum permitted assignment distance
`1` (default) | nonnegative scalar

Maximum permitted assignment distance between a newly encountered or divergent track and a truth object, specified as a nonnegative scalar. For distances beyond this value, assignments between the track and the truth cannot take place. Units are in normalized estimation error squared (NEES).

Data Types: single | double

`DivergenceThreshold` — Maximum permitted divergence distance
`2` (default) | nonnegative scalar

Maximum permitted divergence distance between a track state and the state of an assigned truth object, specified as a nonnegative scalar. For distances beyond this value, tracks are eligible for reassignment to a different truth object. Units are in NEES.

Data Types: single | double

`DistanceFunctionFormat` — Distance function format
`'built-in'` (default) | `'custom'`

Distance function format specified as 'built-in' or 'custom'.

'built-in' – Enable the MotionModel, AssignmentDistance, and DivergenceDistance properties. These properties are convenient interfaces when tracks are reported by any built-in multi-object tracker (such as trackerGNN), and truths reported by the platformPoses object function of a trackingScenario object.
'custom' – Enable custom properties: AssignmentDistanceFcn, DivergenceDistanceFcn, IsInsideCoverageAreaFcn, TruthIdentifierFcn, and TrackIdentifierFcns. You can use these properties to construct acceptance or divergence distances, coverage areas, and identifiers for arbitrary 'tracks' and 'truths' input arrays.

`MaxUnreportedPeriod` — Maximum time period before deleting tracks or truths that are not updated
`0` (default) | nonnegative scalar

Maximum time that must elapse before the metric deletes tracks or truths that were not updated, specified as a nonnegative scalar.

When specify 0, the metric does not check the time of the tracks or truths input arguments and deletes a track or a truth if it was not updated at any single time step.

When specified as a positive scalar,

The tracks input must have a property or field named UpdateTime. The truths input must have a field named Time.
The tracks and truths inputs for each update must be at the same time.
The metric deletes tracks or truths that were not updated within the time period specified in the MaxUnreportedPeriod property.
The track summary and truth summary outputs form the trackAssignmentMetric object, the trackMetricsTable object function, or the truthMetricsTable function have two additional fields: MeanTimeBetweenReports and MaxTimeBetweenReports.

Example: 0.5

Data Types: single | double

Built-in Distance Function Format Properties

`MotionModel` — Desired platform motion model
`'constvel'` (default) | `'constacc'` | `'constturn'` | `'singer'`

Desired platform motion model, specified as 'constvel', 'constacc', 'constturn', or 'singer'. This property selects the motion model used by the tracks input.

The motion models expect the 'State' field of the tracks to have a column vector containing these values:

'constvel' — Position is in elements [1 3 5], and velocity is in elements [2 4 6].
'constacc' — Position is in elements [1 4 7], velocity is in elements [2 5 8], and acceleration is in elements [3 6 9].
'constturn' — Position is in elements [1 3 6], velocity is in elements [2 4 7], and yaw rate is in element 5.
'singer' — Position is in elements [1 4 7], velocity is in elements [2 5 8], and acceleration is in elements [3 6 9].

The 'StateCovariance' field of the tracks input must have position, velocity, and turn-rate covariances in the rows and columns corresponding to the position, velocity, and turn-rate of the 'State' field of the tracks input.

`AssignmentDistance` — Type of assignment distance
`'posnees'` (default) | `'velnees'` | `'posabserr'` | `'velabserr'`

Type of assignment distance, specified as 'posnees', 'velnees', 'posabserr', or 'velabserr'. The type specifies the physical quantity used for assignment. When a new track is detected or a track becomes divergent, the track is compared against truth using this quantity. The assignment seeks the closest truth within the threshold defined by the AssignmentThreshold property.

'posnees' – NEES error of track position
'velnees' – NEES error in track velocity
'posabserr' – Absolute error of track position
'velabserr' – Absolute error of track velocity

Dependencies

To enable this property, set the DistanceFunctionFormat property to 'built-in'.

`DivergenceDistance` — Type of assignment distance
`'posnees'` (default) | `'velnees'` | `'posabserr'` | `'velabserr'`

Type of divergence distance, specified as 'posnees', 'velnees', 'posabserr', or 'velabserr'. The type specifies the physical quantity used for assessing divergence. When a track was previously assigned to truth, the distance between them is compared to this quantity on subsequent update steps. Any track whose divergence distance to its truth assignment exceeds the value of DivergenceThreshold is considered divergent and can be reassigned to a new truth.

'posnees' – NEES error of track position
'velnees' – NEES error in track velocity
'posabserr' – Absolute error of track position
'velabserr' – Absolute error of track velocity

Dependencies

To enable this property, set the DistanceFunctionFormat property to 'built-in'.

Custom Distance Function Format Properties

`AssignmentDistanceFcn` — Assignment distance function
function handle

Assignment distance function, specified as a function handle. This function determines the assignment distance between truths and tracks. Whenever a new track is detected or an existing track becomes divergent, the track needs to be compared against all truths at the current step. This function help to find the closest truth relative to the track within the threshold defined by the AssignmentThreshold property.

The function must have the following syntax:

dist = assignmentdistance(onetrack,onetruth)

The function must return a nonnegative assignment distance, dist, typically expressed in units of NEES. onetrack is an element of the tracks array input argument. onetruth is an element of the truths array input argument.

Dependencies

To enable this property, set the DistanceFunctionFormat property to 'custom'.

Data Types: function_handle

`DivergenceDistanceFcn` — Divergence distance function
function handle

Divergence distance function, specified as a function handle. This function determines the divergence distance between truths and tracks. If the divergence distance from a track to its truth assignment exceeds the DivergenceThreshold, the track is considered divergent and can be reassigned to a new truth.

The function must have the following syntax:

dist = divergencedistance(onetrack,onetruth)

The function must return a non-negative divergence distance, dist, typically expressed in units of NEES. onetrack is an element of the tracks array input argument. onetruth is an element of the truths array input argument.

Dependencies

To enable this property, set the DistanceFunctionFormat property to 'custom'.

Data Types: function_handle

`IsInsideCoverageAreaFcn` — Determine the time that a truth object is detectable
function handle

Function to determine the time that a truth object is detectable, specified as a function handle. This function determines the time that a truth object is inside the coverage area of the sensors and is therefore detectable.

The function must have the following syntax:

status = isinsidecoveragearea(truths)

and return a logical array, status. truths is an array of truth objects expected to be passed in on each step. status is a logical array with the same size as the truths input. An entry of status is true when the corresponding truth object specified by truths is within the coverage area of the sensors.

Dependencies

To enable this property, set the DistanceFunctionFormat property to 'custom'.

Data Types: function_handle

`TrackIdentifierFcn` — Track identifier function
function handle

Track identifier function for the tracks input, specified as a function handle. The track identifiers are unique strings or numeric values.

The function must have the following syntax

trackids = trackidentifier(tracks)

and return a numeric array, trackids. trackids must have the same size as tracks input argument. The default track identification function assumes Tracks is an array of struct or class with a TrackID field or property.

Dependencies

To enable this property, set the DistanceFunctionFormat property to 'custom'.

Data Types: function_handle

`TruthIdentifierFcn` — Truth identifier function
function handle

Truth identifier function for the truths input, specified as a function handle. The truth identifiers are unique strings or numeric values.

The function must have the following syntax

truthids = truthidentifier(truths)

and return a numeric array, truthids. truthids must have the same size as the truths input argument. The default truth identification function assumes truths is an array of struct or class with a PlatformID field or property.

Dependencies

To enable this property, set the DistanceFunctionFormat property to 'custom'.

Data Types: function_handle

`InvalidTrackIdentifier` — Track identifier for invalid assignment
`NaN` (default) | scalar | string

Track identifier for invalid assignment, specified as a scalar or string. This value is returned when the track assignment is invalid. The value must be of the same class as returned by the function handle specified in TrackIdentifierFcn.

Example: -1

Data Types: single | double | string

`InvalidTruthIdentifier` — Truth identifier for invalid assignment
`NaN` (default) | scalar | string

Truth identifier for invalid assignment, specified as a scalar or string. This value is returned when the truth assignment is invalid. The value must be of the same class as returned by the function handle specified in TruthIdentifierFcn.

Example: -1

Data Types: single | double | string

Usage

To compute metrics, call the track assignment metrics with arguments, as if it were a function (described here).

Syntax

[tracksummary,truthsummary] = assignmentMetrics(tracks,truths)

Description

[tracksummary,truthsummary] = assignmentMetrics(tracks,truths) returns structures, tracksummary and truthsummary, containing cumulative metrics across all tracks and truths, obtained from the previous object update.

Input Arguments

expand all

`tracks` — Track information
array of objects | array of structures

Track information, specified as an array of objects or an array of structures. When you set the MaxUnreportedPeriod property to a nonzero value, the object or structure must have an UpdateTime property or field, respectively.

If the DistanceFunctionFormat property is specified as 'built-in', then you must specify this argument as an array of objectTrack objects or as an array of track structures. Each track structure must contain the State, StateCovariance, and TrackID properties or fields. The track outputs from built-in trackers, such as trackerGNN, are compatible with the tracks input.

Data Types: struct

`truths` — Truth information
array of structures

Truth information, specified as an array of structures. When you set the MaxUnreportedPeriod property as a nonzero value, the structure must have a Time field.

If the DistanceFunctionFormat property is specified as 'built-in', then you must specify this argument as an array of truth structures. Each truth structure must contain the PlatformID, Position, and Velocity fields. You can obtain truth structures from the platformPoses function of the trackingScenario object.

Data Types: struct

Output Arguments

expand all

`tracksummary` — Cumulative track assignment metrics
structure

Cumulative metrics over all tracks, returned as a structure. The metrics are computed over all tracks since the last call to the reset object function. The structure has these fields:

Field	Description
`TotalNumTracks`	The total number of unique track identifiers encountered
`NumFalseTracks`	The number of tracks never assigned to any truth
`MaxSwapCount`	Maximum number of track swaps of each track. A track swap occurs whenever a track is assigned to a different truth.
`TotalSwapCount`	Total number of track swaps of each track. A track swap occurs whenever a track is assigned to a different truth.
`MaxDivergenceCount`	Maximum number of divergences. A track is divergent when the result of the `DivergenceDistanceFcn` is greater than the divergence threshold.
`TotalDivergenceCount`	Total number of divergences. A track is divergent when the result of the divergence distance function is greater than the divergence threshold.
`MaxDivergenceLength`	Maximum number of updates during which each track was in a divergent state
`TotalDivergenceLength`	Total number of updates during which each track was in a divergent state
`MaxRedundancyCount`	The maximum number of additional tracks assigned to the same truth
`TotalRedundancyCount`	The total number of additional tracks assigned to the same truth
`MaxRedundancyLength`	Maximum number of updates during which each track was in a redundant state
`TotalRedundancyLength`	Total number of updates during which each track was in a redundant state
`MaxTimeBetweenReports`	The maximum time interval between adjacent track updates to the metric across all tracks. To enable this field, set the `MaxUnreportedPeriod` property to a nonzero value. The field is returned as a `NaN` value if there was only one update to the metric.
`MeanTimeBetweenReports`	The average time interval between adjacent track updates to the metric across all tracks. To enable this field, set the `MaxUnreportedPeriod` property to a non-zero value. The field is returned as a `NaN` value if there was only one update to the metric.

Data Types: struct

`truthsummary` — Cumulative truth assignment metrics
structure

Cumulative assignment metrics over all truths, returned as a structure. The metrics are computed over all truths since the last call to the reset object function. The structure has these fields:

Field	Description
`TotalNumTruths`	The total number of unique truth identifiers encountered
`NumMissingTruths`	The number of truths never established with any track
`MaxEstablishmentLength`	Maximum number of updates before a truth was associated with any track while inside the coverage area. The lengths of missing truths do not count toward this summary metric.
`TotalEstablishmentLength`	Total number of updates before a truth was associated with any track while inside the coverage area. The lengths of missing truths do not count toward this summary metric.
`MaxBreakCount`	Maximum number of times each truth was unassociated by any track after being established.
`TotalBreakCount`	Total number of times each truth was unassociated by any track after being established.
`MaxBreakLength`	Maximum number of updates during which each truth was in a broken state
`TotalBreakLength`	Total number of updates during which each truth was in a broken state
`MaxTimeBetweenReports`	The maximum time interval between adjacent truth updates to the metric across all truths. To enable this field, set the `MaxUnreportedPeriod` property to a non-zero value. The field is returned as a `NaN` value if there was only one update to the metric.
`MeanTimeBetweenReports`	The average time interval between adjacent truth updates to the metric across all truths. To enable this field, set the `MaxUnreportedPeriod` property to a non-zero value. The field is returned as a `NaN` value if there was only one update to the metric.

Data Types: struct

Object Functions

To use an object function, specify the System object as the first input argument. For example, to release system resources of a System object named obj, use this syntax:

release(obj)

expand all

Specific to `trackAssignmentMetrics`

`currentAssignment`	Mapping of tracks to truth
`trackMetricsTable`	Compare tracks to truth
`truthMetricsTable`	Compare truth to tracks
`evaluateBatch`	Compute assignment metrics for recorded data

Common to All System Objects

`release`	Release resources and allow changes to System object property values and input characteristics
`reset`	Reset internal states of System object
`isLocked`	Determine if System object is in use
`clone`	Create duplicate System object

Examples

collapse all

Assignment and Error Metrics for Two Tracked Targets

Open Live Script

Examine the assignments and errors for a system tracking two targets.

First, load the stored track data.

load trackmetricex tracklog truthlog

Create objects to analyze assignment and error metrics.

tam = trackAssignmentMetrics;
tem = trackErrorMetrics;

Create the output variables.

posRMSE = zeros(numel(tracklog),1);
velRMSE = zeros(numel(tracklog),1);
posANEES = zeros(numel(tracklog),1);
velANEES = zeros(numel(tracklog),1);

Loop over all tracks to:

Extract the tracks and ground truth at the i th tracker update.
Analyze and retrieve the current track-to-truth assignment.
Analyze instantaneous error metrics over all tracks and truths.

for i=1:numel(tracklog)
    tracks = tracklog{i};
    truths = truthlog{i};
    [trackAM,truthAM] = tam(tracks, truths);
    [trackIDs,truthIDs] = currentAssignment(tam);
    [posRMSE(i),velRMSE(i),posANEES(i),velANEES(i)] = ...
        tem(tracks,trackIDs,truths,truthIDs);
end

Show the track metrics table.

trackMetricsTable(tam)

ans=4×16 table
    TrackID    AssignedTruthID    Surviving    TotalLength    UnreportedStatus    DeletionStatus    DeletionLength    DivergenceStatus    DivergenceCount    DivergenceLength    RedundancyStatus    RedundancyCount    RedundancyLength    FalseTrackStatus    FalseTrackLength    SwapCount
    _______    _______________    _________    ___________    ________________    ______________    ______________    ________________    _______________    ________________    ________________    _______________    ________________    ________________    ________________    _________

       1             NaN            false         1120             false              false               0                false                 3                   3                false                 0                   0                false                  0               0    
       2             NaN            false         1736             false              false               0                false                 8                  88                false                 0                   0                false                 28               3    
       6               3            true          1138             false              false               0                false                 4                 314                false                 1                  28                false                  0               2    
       8               2            true           662             false              false               0                false                 2                  29                false                 1                 169                false                 28               0

Show the truth metrics table.

truthMetricsTable(tam)

ans=2×11 table
    TruthID    AssociatedTrackID    DeletionStatus    TotalLength    UnreportedStatus    BreakStatus    BreakCount    BreakLength    InCoverageArea    EstablishmentStatus    EstablishmentLength
    _______    _________________    ______________    ___________    ________________    ___________    __________    ___________    ______________    ___________________    ___________________

       2               8                false            2678             false             false           4             168            true                 true                    56         
       3               6                false            2678             false             false           3             645            true                 true                    84

Plot the RMSE and ANEES error metrics.

subplot(2,2,1)
plot(posRMSE)
title('Position Error')
xlabel('tracker update')
ylabel('RMSE (m)')

subplot(2,2,2)
plot(velRMSE)
title('Velocity Error')
xlabel('tracker update')
ylabel('RMSE (m/s)')

subplot(2,2,3)
plot(posANEES)
title('Position Error')
xlabel('tracker update')
ylabel('ANEES')

subplot(2,2,4)
plot(velANEES)
title('Velocity Error')
xlabel('tracker update')
ylabel('ANEES')

Show the current error metrics for each individual recorded track.

currentTrackMetrics(tem)

ans=2×5 table
    TrackID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       6       44.712    20.988    0.05974     0.31325 
       8       129.26    12.739     1.6745      0.2453

Show the current error metrics for each individual recorded truth object.

currentTruthMetrics(tem)

ans=2×5 table
    TruthID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       2       129.26    12.739     1.6745      0.2453 
       3       44.712    20.988    0.05974     0.31325

Show the cumulative error metrics for each individual recorded track.

cumulativeTrackMetrics(tem)

ans=4×5 table
    TrackID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       1       117.69    43.951    0.58338     0.44127 
       2        129.7      42.8    0.81094     0.42509 
       6       371.35    87.083     4.5208      1.6952 
       8       130.45    53.914     1.0448     0.44813

Show the cumulative error metrics for each individual recorded truth object.

cumulativeTruthMetrics(tem)

ans=2×5 table
    TruthID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       2       258.21    65.078     2.2514     0.93359 
       3       134.41    48.253    0.96314     0.49183

Version History

Introduced in R2018b

expand all

R2023b: Evaluate track assignment metrics for recorded track and truth data

You can use the new evaluateBatch object function of the trackAssignmentMetrics object to evaluate track assignment metrics between a log of tracks and a log of truths.

Additionally, the trackAssignmentMetrics object has a new MaxUnreportedPeriod property, which represents the maximum time period that the object maintains a track or truth if the track or truth does not get any update. Previously, the object deleted a track or a truth if they were not updated at a step. When you set the MaxUnreportedPeriod property to a nonzero value, additional metrics, such as MeanTimeBetweenReports and MaxTimeBetweenReports, are reported.

R2023b: Specify `tracks` input using only one type of variable when `MaxUnreportPeriod` is nonzero

Where you specify the tracks input, you cannot switch between the objectTrack format and track structure format across multiple calls to the metric if the MaxUnreportedPeriod property is nonzero. To change the input format, call the release object function first.

trackAssignmentMetrics

Description

Creation

Syntax

Description

Properties

AssignmentThreshold — Maximum permitted assignment distance 1 (default) | nonnegative scalar

DivergenceThreshold — Maximum permitted divergence distance 2 (default) | nonnegative scalar

DistanceFunctionFormat — Distance function format 'built-in' (default) | 'custom'

MaxUnreportedPeriod — Maximum time period before deleting tracks or truths that are not updated 0 (default) | nonnegative scalar

MotionModel — Desired platform motion model 'constvel' (default) | 'constacc' | 'constturn' | 'singer'

AssignmentDistance — Type of assignment distance 'posnees' (default) | 'velnees' | 'posabserr' | 'velabserr'

Dependencies

DivergenceDistance — Type of assignment distance 'posnees' (default) | 'velnees' | 'posabserr' | 'velabserr'

Dependencies

AssignmentDistanceFcn — Assignment distance function function handle

Dependencies

DivergenceDistanceFcn — Divergence distance function function handle

Dependencies

IsInsideCoverageAreaFcn — Determine the time that a truth object is detectable function handle

Dependencies

TrackIdentifierFcn — Track identifier function function handle

Dependencies

TruthIdentifierFcn — Truth identifier function function handle

Dependencies

InvalidTrackIdentifier — Track identifier for invalid assignment NaN (default) | scalar | string

InvalidTruthIdentifier — Truth identifier for invalid assignment NaN (default) | scalar | string

Usage

Syntax

Description

Input Arguments

tracks — Track information array of objects | array of structures

truths — Truth information array of structures

Output Arguments

tracksummary — Cumulative track assignment metrics structure

truthsummary — Cumulative truth assignment metrics structure

Object Functions

Specific to trackAssignmentMetrics

Common to All System Objects

Examples

Assignment and Error Metrics for Two Tracked Targets

Version History

R2023b: Evaluate track assignment metrics for recorded track and truth data

R2023b: Specify tracks input using only one type of variable when MaxUnreportPeriod is nonzero

See Also

Objects

`AssignmentThreshold` — Maximum permitted assignment distance
`1` (default) | nonnegative scalar

`DivergenceThreshold` — Maximum permitted divergence distance
`2` (default) | nonnegative scalar

`DistanceFunctionFormat` — Distance function format
`'built-in'` (default) | `'custom'`

`MaxUnreportedPeriod` — Maximum time period before deleting tracks or truths that are not updated
`0` (default) | nonnegative scalar

`MotionModel` — Desired platform motion model
`'constvel'` (default) | `'constacc'` | `'constturn'` | `'singer'`

`AssignmentDistance` — Type of assignment distance
`'posnees'` (default) | `'velnees'` | `'posabserr'` | `'velabserr'`

`DivergenceDistance` — Type of assignment distance
`'posnees'` (default) | `'velnees'` | `'posabserr'` | `'velabserr'`

`AssignmentDistanceFcn` — Assignment distance function
function handle

`DivergenceDistanceFcn` — Divergence distance function
function handle

`IsInsideCoverageAreaFcn` — Determine the time that a truth object is detectable
function handle

`TrackIdentifierFcn` — Track identifier function
function handle

`TruthIdentifierFcn` — Truth identifier function
function handle

`InvalidTrackIdentifier` — Track identifier for invalid assignment
`NaN` (default) | scalar | string

`InvalidTruthIdentifier` — Truth identifier for invalid assignment
`NaN` (default) | scalar | string

`tracks` — Track information
array of objects | array of structures

`truths` — Truth information
array of structures

`tracksummary` — Cumulative track assignment metrics
structure

`truthsummary` — Cumulative truth assignment metrics
structure

Specific to `trackAssignmentMetrics`

R2023b: Specify `tracks` input using only one type of variable when `MaxUnreportPeriod` is nonzero