Main Content

trackErrorMetrics

Track error and NEES

expand all in page

Description

The trackErrorMetrics System object™ provides quantitative comparisons between tracks and known truth trajectories.

To generate track assignment metrics:

  1. Create the trackErrorMetrics object and set its properties.

  2. Call the object with arguments, as if it were a function.

To learn more about how System objects work, see What Are System Objects?

Creation

Syntax

errorMetrics = trackErrorMetrics
errorMetrics = trackErrorMetrics(Name,Value)

Description

errorMetrics = trackErrorMetrics creates a trackErrorMetrics System object with default property values.

errorMetrics = trackErrorMetrics(Name,Value) sets properties for the trackErrorMetrics object using one or more name-value pairs. For example, metrics = trackErrorMetrics('MotionModel','constvel') creates a trackErrorMetrics object with a constant velocity motion model. Enclose property names in quotes.

example

Properties

expand all

Unless otherwise indicated, properties are nontunable, which means you cannot change their values after calling the object. Objects lock when you call them, and the release function unlocks them.

If a property is tunable, you can change its value at any time.

For more information on changing property values, see System Design in MATLAB Using System Objects.

Error function format specified as 'built-in' or 'custom'.

  • 'built-in' – Enable the MotionModel property.

    This property is a convenient interface when using tracks reported by any built-in multi-object tracker, and truths reported by the platformPoses object function of a trackingScenario object. The default estimation error function assumes tracks and truths are arrays of structures or arrays of objects.

  • 'custom' – Enable custom properties: EstimationErrorLabels, EstimationErrorFcn, TruthIdentifierFcn, and TrackIdentifierFcns. These properties can be used to construct error functions for arbitrary tracks and truths input arrays.

Desired platform motion model, specified as 'constvel', 'constacc', 'constturn', or 'singer'. This property selects the motion model used by the tracks input.

The motion models expect the 'State' field of the tracks to have a column vector containing these values:

  • 'constvel' — Position is in elements [1 3 5], and velocity is in elements [2 4 6].

  • 'constacc' — Position is in elements [1 4 7], velocity is in elements [2 5 8], and acceleration is in elements [3 6 9].

  • 'constturn' — Position is in elements [1 3 6], velocity is in elements [2 4 7], and yaw rate is in element 5.

  • 'singer' — Position is in elements [1 4 7], velocity is in elements [2 5 8], and acceleration is in elements [3 6 9].

The 'StateCovariance' field of the tracks input must have position, velocity, and turn-rate covariances in the rows and columns corresponding to the position, velocity, and turn-rate of the 'State' field of the tracks input.

Labels for outputs of error estimation function, specified as an array of strings or cell array of character vectors. The number of labels must correspond to the number of outputs of the error estimation function. Specify the error estimation functions using the EstimationErrorFcn property.

Example: {'posMSE','velMSE'}

Dependencies

To enable this property, set the ErrorFunctionFormat property to 'custom'.

Data Types: char | string

Error estimation function, specified as a function handle. The function determines estimation errors of truths to tracks.

The error estimation function can have multiple scalar outputs and must have the following syntax.

 [out1,out2, ...,outN] = estimationerror(onetrack,onetruth)
The number of outputs must match the number of entries in the labels array specified in the EstimationErrorLabels property.

onetrack is an element of the tracks array passed in as input trackErrorMetric at object updates. onetruth is an element of the truths array passed in at object updates. The trackErrorMetrics object averages each output arithmetically when reporting across tracks or truths.

Example: @errorFunction

Dependencies

To enable this property, set the ErrorFunctionFormat property to 'custom'.

Data Types: function_handle

Track identifier function, specified as a function handle. Specifies the track identifiers for the tracks input at object update. The track identifiers are unique string or numeric values.

The track identifier function must have the following syntax:

 trackID = trackIDentifier(tracks)

tracks is the same as the tracks array passed as input for trackErrorMetric at object update. trackID is the same size as tracks. The default identification function handle, @defaultTrackIdentifier, assumes tracks is an array of structures or objects with a 'TrackID' field name or property.

Dependencies

To enable this property, set the ErrorFunctionFormat property to 'custom'.

Data Types: function_handle

Truth identifier function, specified as a function handle. Specifies the truth identifiers for the truths input at object update. The truth identifiers are unique string or numeric values.

The truth identifier function must have the following syntax:

 truthID = truthIDentifier(truths)

truths is the same as the truths array passed as input for trackErrorMetric updates. truthID must have the same size as truths. The default identification function handle, @defaultTruthIdentifier, assumes truths is an array or structures or objects with a 'PlatformID' field name or property.

Dependencies

To enable this property, set the ErrorFunctionFormat property to 'custom'.

Data Types: function_handle

Usage

To estimate errors, call the track error metrics object with arguments, as if it were a function (described here).

Syntax

[posRMSE,velRMSE,posANEES,velANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs)
[posRMSE,velRMSE,accRMSE,posANEES,velANEES,accANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs)
[posRMSE,velRMSE,yawRateRMSE,posANEES,velANEES,yawRateANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs)
[out1,out2, ... ,outN] = errorMetrics(tracks,trackIDs,truths,truthIDs)

Description

[posRMSE,velRMSE,posANEES,velANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs) returns the metrics

  • posRMSE – Position root mean squared error

  • velRMSE – Velocity root mean squared error

  • posANEES – Position average normalized-estimation error squared

  • velANEES – Velocity average normalized-estimation error squared

for constant velocity motion at the current time step. trackIDs is the set of track identifiers for all tracks. truthIDs is the set of truth identifiers. tracks are the set of tracks, and truths are the set of truths. trackIDs and truthIDs are each a vector whose corresponding elements match the track and truth identifiers found in tracks and truths, respectively.

The RMSE and ANEES values for different states are calculated by averaging the errors of all tracks at the current time step. For example, the position RMSE value, posRMSE, is defined as:

posRMSE=1M ∑i=1M‖Δpi‖2

where M is the total number of tracks with associated truth trajectories in the current time step, and

Δpi=ptrack,i−ptruth,i

is the position difference between the position of track i, ptrack,i, and the position of the corresponding truth, ptruth,i, at the current time step. The RMSE values for other states (vel, pos, acc, and yawRate) are defined similarly.

The position ANEES value, posANEES, is defined as:

posANEES=1M∑i=1MΔpiTCp,i−1Δpi

where Cp,i is the covariance matrix corresponding to the position of track i at the current time step. The ANEES values for other states (vel, pos, acc, and yawRate) are defined similarly.

To enable this syntax, set the ErrorFunctionFormat property to 'built-in' and the MotionModel property to 'constvel'.

example

[posRMSE,velRMSE,accRMSE,posANEES,velANEES,accANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs) also returns the metrics

  • accRMS – Acceleration root mean squared error

  • accANEES – acceleration average normalized-estimation error squared

for constant acceleration motion at the current time step.

To enable this syntax, set the ErrorFunctionFormat property to 'built-in' and the MotionModel property to 'constacc'.

[posRMSE,velRMSE,yawRateRMSE,posANEES,velANEES,yawRateANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs) also returns the metrics

  • yawRateRMSE – yaw rate root mean squared error

  • yawRateANEES – yaw rate average normalized-estimation error squared

for constant turn-rate motion at the current time step.

To enable this syntax, set the ErrorFunctionFormat property to 'built-in' and the MotionModel property to 'constturn'.

[out1,out2, ... ,outN] = errorMetrics(tracks,trackIDs,truths,truthIDs) returns the user-defined metrics out1, out2, ... , outN.

To enable this syntax, set the ErrorFunctionFormat property to 'custom'. The number of outputs corresponds to the number of elements listed in the EstimationErrorLabels property, and must match the number of outputs in the EstimationErrorFcn. The results of the estimation errors are averaged arithmetically over all track-to-truth assignments.

Tip

These usage syntaxes only calculate the RMSE and ANEES values of all tracks with associated truths at the current time step. To obtain the cumulative RMSE and ANEES values for each track and truth, use the cumulativeTrackMetrics and cumulativeTruthMetrics object functions, respectively. To obtain the current RMSE and ANEES values for each track and truth, use the currentTrackMetrics and currentTruthMetrics object functions, respectively.

Input Arguments

expand all

Track information, specified as an array of structures or objects. For built-in trackers such as trackerGNN or trackerTOMHT, the objectTrack output contains 'State', 'StateCovariance', and 'TrackID' information.

Data Types: struct

Track identifiers, specified as a real-valued vector. trackIDs elements match the tracks found in tracks.

Truth information, specified as an array of structures or objects. When using a trackingScenario, truth information can be obtained from the platformPoses object function.

Data Types: struct

Truth identifiers, specified as a real-valued vector. truthIDs elements match the truths found in truths.

Output Arguments

expand all

Position root mean squared error for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

Velocity root mean squared error for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

Acceleration root mean squared error for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

Yaw rate root mean squared error for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

Position average normalized estimation error squared for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

Velocity average normalized estimation error squared for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

Acceleration average normalized estimation error squared for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

Yaw rate average normalized estimation error squared for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

Custom error metric outputs, returned as scalars. These errors are the output of the error estimation function specified in the EstimationErrorFcn property.

Dependencies

To enable these arguments, set the ErrorFunctionFormat property to 'custom'.

Object Functions

To use an object function, specify the System object as the first input argument. For example, to release system resources of a System object named obj, use this syntax:

release(obj)

expand all

cumulativeTrackMetricsCumulative metrics for recent tracks
cumulativeTruthMetricsCumulative metrics for recent truths
currentTrackMetricsMetrics for recent tracks
currentTruthMetricsMetrics for recent truths
evaluateBatch Compute track error metrics for recorded data
releaseRelease resources and allow changes to System object property values and input characteristics
resetReset internal states of System object
isLockedDetermine if System object is in use
cloneCreate duplicate System object

Examples

collapse all

Open Live Script

Examine the assignments and errors for a system tracking two targets.

First, load the stored track data.

load trackmetricex tracklog truthlog

Create objects to analyze assignment and error metrics.

tam = trackAssignmentMetrics;
tem = trackErrorMetrics;

Create the output variables.

posRMSE = zeros(numel(tracklog),1);
velRMSE = zeros(numel(tracklog),1);
posANEES = zeros(numel(tracklog),1);
velANEES = zeros(numel(tracklog),1);

Loop over all tracks to:

  • Extract the tracks and ground truth at the i th tracker update.

  • Analyze and retrieve the current track-to-truth assignment.

  • Analyze instantaneous error metrics over all tracks and truths.

for i=1:numel(tracklog)
    tracks = tracklog{i};
    truths = truthlog{i};
    [trackAM,truthAM] = tam(tracks, truths);
    [trackIDs,truthIDs] = currentAssignment(tam);
    [posRMSE(i),velRMSE(i),posANEES(i),velANEES(i)] = ...
        tem(tracks,trackIDs,truths,truthIDs);
end

Show the track metrics table.

trackMetricsTable(tam)
ans=4×16 table
    TrackID    AssignedTruthID    Surviving    TotalLength    UnreportedStatus    DeletionStatus    DeletionLength    DivergenceStatus    DivergenceCount    DivergenceLength    RedundancyStatus    RedundancyCount    RedundancyLength    FalseTrackStatus    FalseTrackLength    SwapCount
    _______    _______________    _________    ___________    ________________    ______________    ______________    ________________    _______________    ________________    ________________    _______________    ________________    ________________    ________________    _________

       1             NaN            false         1120             false              false               0                false                 3                   3                false                 0                   0                false                  0               0    
       2             NaN            false         1736             false              false               0                false                 8                  88                false                 0                   0                false                 28               3    
       6               3            true          1138             false              false               0                false                 4                 314                false                 1                  28                false                  0               2    
       8               2            true           662             false              false               0                false                 2                  29                false                 1                 169                false                 28               0

Show the truth metrics table.

truthMetricsTable(tam)
ans=2×11 table
    TruthID    AssociatedTrackID    DeletionStatus    TotalLength    UnreportedStatus    BreakStatus    BreakCount    BreakLength    InCoverageArea    EstablishmentStatus    EstablishmentLength
    _______    _________________    ______________    ___________    ________________    ___________    __________    ___________    ______________    ___________________    ___________________

       2               8                false            2678             false             false           4             168            true                 true                    56         
       3               6                false            2678             false             false           3             645            true                 true                    84

Plot the RMSE and ANEES error metrics.

subplot(2,2,1)
plot(posRMSE)
title('Position Error')
xlabel('tracker update')
ylabel('RMSE (m)')

subplot(2,2,2)
plot(velRMSE)
title('Velocity Error')
xlabel('tracker update')
ylabel('RMSE (m/s)')

subplot(2,2,3)
plot(posANEES)
title('Position Error')
xlabel('tracker update')
ylabel('ANEES')

subplot(2,2,4)
plot(velANEES)
title('Velocity Error')
xlabel('tracker update')
ylabel('ANEES')

Figure contains 4 axes objects. Axes object 1 with title Position Error, xlabel tracker update, ylabel RMSE (m) contains an object of type line. Axes object 2 with title Velocity Error, xlabel tracker update, ylabel RMSE (m/s) contains an object of type line. Axes object 3 with title Position Error, xlabel tracker update, ylabel ANEES contains an object of type line. Axes object 4 with title Velocity Error, xlabel tracker update, ylabel ANEES contains an object of type line.

Show the current error metrics for each individual recorded track.

currentTrackMetrics(tem)
ans=2×5 table
    TrackID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       6       44.712    20.988    0.05974     0.31325 
       8       129.26    12.739     1.6745      0.2453

Show the current error metrics for each individual recorded truth object.

currentTruthMetrics(tem)
ans=2×5 table
    TruthID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       2       129.26    12.739     1.6745      0.2453 
       3       44.712    20.988    0.05974     0.31325

Show the cumulative error metrics for each individual recorded track.

cumulativeTrackMetrics(tem)
ans=4×5 table
    TrackID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       1       117.69    43.951    0.58338     0.44127 
       2        129.7      42.8    0.81094     0.42509 
       6       371.35    87.083     4.5208      1.6952 
       8       130.45    53.914     1.0448     0.44813

Show the cumulative error metrics for each individual recorded truth object.

cumulativeTruthMetrics(tem)
ans=2×5 table
    TruthID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       2       258.21    65.078     2.2514     0.93359 
       3       134.41    48.253    0.96314     0.49183

Version History

Introduced in R2018b

expand all

See Also

Objects