evaluateBatch

Compute assignment metrics for recorded data

Since R2023b

Syntax

[trackSummary,truthSummary,assignHistory] = evaluateBatch(assignmentMetric,tracklog,truthlog)

Description

[trackSummary,truthSummary,assignHistory] = evaluateBatch(assignmentMetric,tracklog,truthlog) returns the track summary, truth summary, and the assignment history based on a log of tracks and a log of truths using the trackingAssignmentMetric object. The function places tracks and truths of the same time in a group and performs assignment between the tracks and truths in each group.

Tip

This function ignores the MaxUnreportedPeriod property of the trackAssignmentMetrics object. The function computes a maximum unreported period value based on the tracklog and truthlog inputs and uses the calculated value to delete tracks and truths if they are not updated within the maximum period.

example

Examples

collapse all

Evaluate Track Metrics for Recorded Data

Open Live Script

First, load the stored track data.

load trackmetricex tracklog truthlog

Create a trackAssignmentMetric object.

tam = trackAssignmentMetrics;

Use the evaluateBatch object function to obtain the track summary, the truth summary, and the assignment history.

[trackSummary,truthSummary,assignHistory] = evaluateBatch(tam,tracklog,truthlog)

trackSummary = struct with fields:
            TotalNumTracks: 4
            NumFalseTracks: 0
              MaxSwapCount: 3
            TotalSwapCount: 5
        MaxDivergenceCount: 8
      TotalDivergenceCount: 17
       MaxDivergenceLength: 314
     TotalDivergenceLength: 434
        MaxRedundancyCount: 1
      TotalRedundancyCount: 3
       MaxRedundancyLength: 169
     TotalRedundancyLength: 199
     MaxTimeBetweenReports: 0.0187
    MeanTimeBetweenReports: 0.0187

truthSummary = struct with fields:
              TotalNumTruths: 2
            NumMissingTruths: 0
      MaxEstablishmentLength: 84
    TotalEstablishmentLength: 140
               MaxBreakCount: 4
             TotalBreakCount: 7
              MaxBreakLength: 645
            TotalBreakLength: 813
       MaxTimeBetweenReports: 0.0187
      MeanTimeBetweenReports: 0.0187

assignHistory=2678×1 struct array with fields:
    Time
    TrackIDs
    TruthIDs

Next, create a trackErrorMetrics object.

tem = trackErrorMetrics;

Use the evaluateBatch object function to obtain the cumulative track and truth error metric tables as well as the track and truth error history tables.

[cumTrackError,cumTruthError,trackErrorHistory,truthErrorHistory] = evaluateBatch(tem,tracklog,truthlog,assignHistory)

cumTrackError=4×5 table
    TrackID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       1       117.69    43.951    0.58338     0.44127 
       2        129.7      42.8    0.81094     0.42509 
       6       371.35    87.083     4.5208      1.6952 
       8       130.45    53.914     1.0448     0.44813

cumTruthError=2×5 table
    TruthID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       2       258.21    65.078     2.2514     0.93359 
       3       134.41    48.253    0.96314     0.49183

trackErrorHistory=4600×6 table
     Time     TrackID    posRMS    velRMS    posANEES    velANEES
    ______    _______    ______    ______    ________    ________

     1.064       1       30.104    12.092    0.049206    0.024983
    1.0827       1       29.996    12.092    0.035699    0.022217
    1.1013       1        29.89    12.092    0.035276    0.022217
      1.12       1       29.785    12.092    0.034862    0.022217
    1.1387       1       29.681    12.092    0.034458    0.022217
    1.1573       1       29.579    12.092    0.034065    0.022217
     1.176       1       29.478    12.092    0.033682    0.022217
    1.1947       1       29.379    12.092     0.03331    0.022217
    1.2133       1       29.281    12.092    0.032948    0.022217
     1.232       1       29.184    12.092    0.032597    0.022217
    1.2507       1       29.089    12.092    0.032257    0.022217
    1.2693       1       28.995    12.092    0.031927    0.022217
     1.288       1       28.902    12.092    0.031608    0.022217
    1.3067       1       28.812    12.092      0.0313    0.022217
    1.3253       1       28.722    12.092    0.031003    0.022217
     1.344       1       28.634    12.092    0.030715    0.022217
      ⋮

truthErrorHistory=4403×6 table
     Time     TruthID    posRMS    velRMS    posANEES    velANEES
    ______    _______    ______    ______    ________    ________

     1.064       2       30.104    12.092    0.049206    0.024983
    1.0827       2       29.996    12.092    0.035699    0.022217
    1.1013       2        29.89    12.092    0.035276    0.022217
      1.12       2       29.785    12.092    0.034862    0.022217
    1.1387       2       29.681    12.092    0.034458    0.022217
    1.1573       2       29.579    12.092    0.034065    0.022217
     1.176       2       29.478    12.092    0.033682    0.022217
    1.1947       2       29.379    12.092     0.03331    0.022217
    1.2133       2       29.281    12.092    0.032948    0.022217
     1.232       2       29.184    12.092    0.032597    0.022217
    1.2507       2       29.089    12.092    0.032257    0.022217
    1.2693       2       28.995    12.092    0.031927    0.022217
     1.288       2       28.902    12.092    0.031608    0.022217
    1.3067       2       28.812    12.092      0.0313    0.022217
    1.3253       2       28.722    12.092    0.031003    0.022217
     1.344       2       28.634    12.092    0.030715    0.022217
      ⋮

Input Arguments

collapse all

`assignmentMetric` — Track assignment metrics object
`trackAssignmentMetrics` System object

Track assignment metrics object, specified as a trackAssignmentMetrics System object™.

`tracklog` — Recorded track log
array of objects or structures | cell array of objects or structures

Recorded track log, specified as an array of objects or structures. Alternately, you can specify it as a cell array of objects or structures. Each object or structure must have an UpdateTime property or field, respectively.

If the DistanceFunctionFormat property is specified as 'built-in', each object must be an objectTrack object and each structure must be a track structure. The fields of the track structure should be the same as the properties of the objectTrack object. The track structure must at least have the State, StateCovariance, TrackID, and UpdateTime fields.

`truthlog` — Recorded truth log
array of structures | cell array of structures | `trackScenarioRecording` object

Recorded truth log, specified as an array of structures, a cell array of structures, or a trackingScenarioRecording object. Each structure must have a Time field.

If the DistanceFunctionFormat property is specified as 'built-in', each structure must have at least these fields.

Field	Description
`Time`	Time of the truth, specified as a nonnegative scalar.
`PlatformID`	Unique identifier for the platform, specified as a positive integer. This is a required field with no default value.
`Position`	Position of the platform in scenario coordinates, specified as a real-valued 1-by-3 row vector. If the `coordinateSystem` argument is specified as `'Cartesian'`, `Position` is the 3-element vector of Cartesian position coordinates in meters. If the `coordinateSystem` argument is specified as `'Geodetic'`, `Position` is the 3-element vector of geodetic coordinates: latitude in degrees, longitude in degrees, and altitude in meters.
`Velocity`	Velocity of the platform in scenario coordinates, specified as a real-valued 1-by-3 row vector. Units are meters per second. The default value is `[0 0 0]`.

You can use the platformPoses function of the trackingScenario object to generate this structure except the Time field, which you can obtain via the SimulationTime property of the trackingScenario object.

Output Arguments

collapse all

`trackSummary` — Track assignment summary
structure

Track assignment summary, returned as a structure. The structure has these fields.

Field	Description
`TotalNumTracks`	The total number of unique track identifiers encountered
`NumFalseTracks`	The number of tracks never assigned to any truth
`MaxSwapCount`	Maximum number of track swaps of each track. A track swap occurs whenever a track is assigned to a different truth.
`TotalSwapCount`	Total number of track swaps of each track. A track swap occurs whenever a track is assigned to a different truth.
`MaxDivergenceCount`	Maximum number of divergences. A track is divergent when the result of the `DivergenceDistanceFcn` is greater than the divergence threshold.
`TotalDivergenceCount`	Total number of divergences. A track is divergent when the result of the divergence distance function is greater than the divergence threshold.
`MaxDivergenceLength`	Maximum number of updates during which each track was in a divergent state
`TotalDivergenceLength`	Total number of updates during which each track was in a divergent state
`MaxRedundancyCount`	The maximum number of additional tracks assigned to the same truth
`TotalRedundancyCount`	The total number of additional tracks assigned to the same truth
`MaxRedundancyLength`	Maximum number of updates during which each track was in a redundant state
`TotalRedundancyLength`	Total number of updates during which each track was in a redundant state
`MaxTimeBetweenReports`	The maximum time interval between adjacent track updates to the metric across all tracks. The field is returned as `NaN` if there has been only one update to the metric.
`MeanTimeBetweenReports`	The average time interval between adjacent track updates to the metric across all tracks. The field is returned as `NaN` if there has been only one update to the metric.

`truthSummary` — Truth assignment summary
structure

Truth assignment summary, returned as a structure. The structure has these fields.

Field	Description
`TotalNumTruths`	The total number of unique truth identifiers encountered
`NumMissingTruths`	The number of truths never established with any track
`MaxEstablishmentLength`	Maximum number of updates before a truth was associated with any track while inside the coverage area. The lengths of missing truths do not count toward this summary metric.
`TotalEstablishmentLength`	Total number of updates before a truth was associated with any track while inside the coverage area. The lengths of missing truths do not count toward this summary metric.
`MaxBreakCount`	Maximum number of times each truth was unassociated with any track after being established.
`TotalBreakCount`	Total number of times each truth was unassociated with any track after being established.
`MaxBreakLength`	Maximum number of updates during which each truth was in a broken state
`TotalBreakLength`	Total number of updates during which each truth was in a broken state
`MaxTimeBetweenReports`	The maximum time interval between adjacent truth updates to the metric across all truths. The field is returned as `NaN` if there has been only one update to the metric.
`MeanTimeBetweenReports`	The average time interval between adjacent truth updates to the metric across all truths. The field is returned as `NaN` if there has been only one update to the metric.

`assignHistory` — Assignment history
array of structures

Assignment history, returned as an array of structures. Each structure has these fields.

Field	Description
`Time`	Time of assignment, returned as a nonnegative scalar.
`TrackIDs`	IDs of tracks, returned as a vector of track IDs.
`TruthIDs`	IDs of truths, returned as a vector of truth IDs.

The corresponding tracks and truths from the TrackIDs and TruthIDs fields are assigned to each other.

Data Types: struct

Version History

Introduced in R2023b

expand all

R2024a: Performance improvement

evaluateBatch shows improved performance for longer input logs with at least 1000 elements. For example, this code is about 5x faster than in the previous release:

function t = timingTest
[truthlog,tracklog] = createlonglogs(1000);
tam = trackAssignmentMetrics();
tic; evaluateBatch(tam,tracklog,truthlog); toc
end

function [truths,tracks]=createlonglogs(numSteps)
v = ones(1,3);
t1 = [struct("PlatformID",1,"Position",v,"Velocity",v,"Time",1);...
    struct("PlatformID",2,"Position",10*v,"Velocity",10*v,"Time",1)];
t2 = [objectTrack(TrackID=1,state=ones(6,1));...
    objectTrack(TrackID=2,state=10*ones(6,1))];
truths = cell(numSteps,1);
tracks = cell(numSteps,1);
for i=1:numSteps
    truths{i} = t1;
    truths{i}(1).Time = i;
    truths{i}(2).Time = i;
    tracks{i} = t2;
    tracks{i}(1).UpdateTime = i;
    tracks{i}(2).UpdateTime = i;
end
end

The approximate execution times are:

R2023b: 9.6 s
R2024a: 1.8 s

The code was timed on a Windows 11, AMD(R) EPYC 74F3 24-Core Processor @ 3.19 GHz test system by calling the timingTest function.

evaluateBatch

Syntax

Description

Examples

Evaluate Track Metrics for Recorded Data

Input Arguments

assignmentMetric — Track assignment metrics object trackAssignmentMetrics System object

tracklog — Recorded track log array of objects or structures | cell array of objects or structures

truthlog — Recorded truth log array of structures | cell array of structures | trackScenarioRecording object

Output Arguments

trackSummary — Track assignment summary structure

truthSummary — Truth assignment summary structure

assignHistory — Assignment history array of structures

Version History

R2024a: Performance improvement

`assignmentMetric` — Track assignment metrics object
`trackAssignmentMetrics` System object

`tracklog` — Recorded track log
array of objects or structures | cell array of objects or structures

`truthlog` — Recorded truth log
array of structures | cell array of structures | `trackScenarioRecording` object

`trackSummary` — Track assignment summary
structure

`truthSummary` — Truth assignment summary
structure

`assignHistory` — Assignment history
array of structures