Main Content

evaluate

Evaluate metrics on tracks and truths

Since R2023a

collapse all in page

Syntax

results = evaluate(metric,tracks,truths)

Description

results = evaluate(metric,tracks,truths) returns the Classification of Events, Activities, and Relationships (CLEAR), Mostly-Tracked, Partially-Tracked, and Mostly-Lost MOT metrics between tracks and truths.

example

Examples

collapse all

Open Live Script

Create a trackCLEARMetrics object and set the SimilarityThreshold property to "0.8".

metric = trackCLEARMetrics(SimilarityThreshold=0.8);

Load a tracking dataset consisting of truths and tracks.

load("trackCLEARData.mat","tracks","truths");

Visualize tracks in red and truths in blue.

figure
for t = 0:0.5:10
    % Extract tracks at a certain time stamp.
    tracks_t = tracks([tracks.Time] == t);
    % Extract turths at a certain time stamp.
    truths_t = truths([truths.Time] == t); 
    cla; % Clean plotting in axes if any.
    for j=1:numel(tracks_t)
        posTrack = tracks_t(j).BoundingBox;
        posTrack(2) = posTrack(2)-posTrack(4);
        rectangle(Position=posTrack,EdgeColor="r",Curvature=[0.2 1]);
    end
    for j=1:numel(truths_t)
        posTruth = truths_t(j).BoundingBox;
        posTruth(2) = posTruth(2)-posTruth(4);
        rectangle(Position=posTruth,EdgeColor="b");
    end
    xlabel("x (m)")
    ylabel("y (m)")
    pause(0.2) % Pause the animation 0.2 seconds for each time stamp.
end

Figure contains an axes object. The axes object with xlabel x (m), ylabel y (m) contains 19 objects of type rectangle.

Evaluate the CLEAR MOT metrics.

metricTable = evaluate(metric,tracks,truths)
metricTable=1×12 table
    MOTA (%)    MOTP (%)    Mostly Tracked (%)    Partially Tracked (%)    Mostly Lost (%)    False Positive    False Negative    Recall (%)    Precision (%)    False Track Rate    ID Switches    Fragmentations
    ________    ________    __________________    _____________________    _______________    ______________    ______________    __________    _____________    ________________    ___________    ______________

     27.737      85.492             50                     40                    10                 56                43            68.613         62.667             3.7333              0               16
Open Live Script

Create a trackCLEARMetrics object. Set the SimilarityMethod property to "Euclidean" and set the EuclideanScale property to 2.

metric = trackCLEARMetrics(SimilarityMethod="Euclidean",EuclideanScale=2);

Load a tracking dataset consisting of tracks and truths.

load("trackCLEAREuclideanData.mat","tracks","truths");

Visualize tracks in red circles and truths in blue diamonds.

figure
for t = 0:0.5:10
    % Extract tracks at a certain time stamp.
    tracks_t = tracks([tracks.Time] == t);
    % Extract truths at a certain time stamp.
    truths_t = truths([truths.Time] == t);
    hold off;
    for j=1:numel(tracks_t)
        position = tracks_t(j).Position;
        plot(position(1),position(2),"ro")
        hold on;
    end
    for j=1:numel(truths_t)
        position = truths_t(j).Position;
        plot(position(1),position(2),"bd")
    end
    xlabel("x (m)")
    ylabel("y (m)")
    pause(0.2) % Pause the animation 0.2 seconds for each time stamp.
end

Figure contains an axes object. The axes object with xlabel x (m), ylabel y (m) contains 4 objects of type line. One or more of the lines displays its values using only markers

Evaluate the CLEAR MOT metrics.

metricTable = evaluate(metric,tracks,truths)
metricTable=1×12 table
    MOTA (%)    MOTP (%)    Mostly Tracked (%)    Partially Tracked (%)    Mostly Lost (%)    False Positive    False Negative    Recall (%)    Precision (%)    False Track Rate    ID Switches    Fragmentations
    ________    ________    __________________    _____________________    _______________    ______________    ______________    __________    _____________    ________________    ___________    ______________

      57.5       80.934             50                     50                     0                 3                 14              65           89.655            0.14286              0               1

Input Arguments

collapse all

Track CLEAR metric, specified as a trackCLEARMetrics object.

Track list, specified as an array of track structures. Each structure must at least have these fields:

  • Time — Time of the track, specified as a nonnegative scalar.

  • TrackID — Track identifier, specified as a positive integer.

If you set the SimilarityMethod property of the trackCLEARMetrics object as "IoU2d", the structure must include a BoundingBox field.

  • BoundingBox — Bounding box of the track, specified as a four-element vector as [x, y, w, h]. x and y are the x- and y-coordinates of the top left corner of the bounding box, respectively. w and h are the width and height of the bounding box, respectively.

If you set the SimilarityMethod property of the trackCLEARMetrics object as "Euclidean", the structure must include a Position field.

  • Position — Position of the track, specified as a 1-D coordinate as x, a 2-D vector as [x, y], or a 3-D vector as [x, y,z]. x, y, and z are the x-, y-, and z-coordinates of the track, respectively.

Truth list, specified as an array of truth structures. Each structure must at least have these fields:

  • Time — Time of the truth, specified as a nonnegative scalar.

  • TruthID — Truth identifier, specified as a positive integer.

If you set the SimilarityMethod property of the trackCLEARMetrics object as "IoU2d", the structure must include a BoundingBox field.

  • BoundingBox — Bounding box of the truth, specified as a four-element vector as [x, y, w, h]. x and y are the x- and y-coordinates of the top left corner of the bounding box, respectively. w and h are the width and height of the bounding box, respectively.

If you set the SimilarityMethod property of the trackCLEARMetrics object as "Euclidean", the structure must include a Position field.

  • Position — Position of the truth, specified as a 1-D coordinate as x, a 2-D vector as [x, y], or a 3-D vector as [x, y,z]. x, y, and z are the x-, y-, and z-coordinates of the truth, respectively.

Output Arguments

collapse all

Metric results, returned as a table. The table contains the CLEAR, Mostly-Tracked, Partially-Tracked, and Mostly-Lost MOT metric values in this order:

  • MOTA — Multiple object tracking accuracy, returned as a percentage value. This metric represents the percentage of true matches (that are not misses, false positives, or switches) in all the matches between tracks and detections. A higher value indicates better tracking performance. For more details, see Algorithms.

  • MOTP — Multiple object tracking precision, returned as a percentage value. This metric represents the averaged similarity values over all the true matches (that are not misses, false positives, or switches) between tracks and truths. A higher value indicates better tracking performance. For more details, see Algorithms.

  • Mostly Tracked — Percentage of true trajectories that are tracked more than 80% of their lifetime. A higher value indicates better tracking performance.

  • Partially Tracked — Percentage of true trajectories that are tracked less than 80% but more than 20% of their lifetime.

  • Mostly Lost — Percentage of true trajectories that are tracked less than 20% of their lifetime. A lower value indicates better tracking performance.

  • False Positive — Total number of tracks that are not matched with any true object. A lower value indicates better tracking performance.

  • False Negative — Total number of truths that are not matched with any estimated object. A lower value indicates better tracking performance.

  • Recall — Percentage of true objects being tracked. A higher value indicates better tracking performance.

  • Precision — Percentage of estimated objects matching the corresponding true objects. A higher value indicates better tracking performance.

  • False Track Rate — Average number of false tracks or false positives per timestamp (frame). A lower value indicates better tracking performance.

  • ID Switches — Total number of identity switches. An identity switch occurs when the track associated to a true object changes from time t to time t+1. A lower value indicates better tracking performance.

  • Fragmentations — Total number of fragmentations. A fragmentation occurs when a true object is tracked again after being untracked for at least one timestamp. A lower value indicates better tracking performance.

Version History

Introduced in R2023a

See Also