Convert Ground Truth Labeling Data for Object Tracking

Overview

Use the trackCLEARMetrics (Sensor Fusion and Tracking Toolbox) object to implement the Classification of Events, Activities, and Relationships Multi-Object Tracking (CLEARMOT) metrics, which evaluate tracking performance by comparing tracks with ground truth. These metrics include identification measures such as MOTA (Multiple Object Tracking Accuracy) and MOTP (Multiple Object Tracking Precision), which capture the accuracy and precision of the tracking results. Additionally, the metrics quantify match conditions like false negatives, false positives, and ID switches, offering a comprehensive evaluation of tracking performance in complex scenarios. To measure this performance, you must first convert the ground truth labeling data to the trackCLEARmetrics truth format.

To use the CLEARMOT metrics to compare the tracks to the truth, you must provide truth information as an input, which you can obtain by labeling video data using the Ground Truth Labeler (Automated Driving Toolbox) app. To learn how to label data for object tracking, view the Automate Ground Truth Labeling for Object Tracking and Re-Identification example. In this example, you convert the output of the Ground Truth Labeler to the trackCLEARmetrics truth format required by the CLEARMOT metrics object.

Load Ground Truth Labeling Data

To convert ground truth data into the trackCLEARmetrics truth format, the groundTruth object must be in the correct format. The ground truth should have a rectangular ROI and a numeric attribute for the object ID. In this example, the ROI is labeled as Person.

Download the video containing the ground truth data, and load the groundTruth object.

helperDownloadLabelVideo();

Downloading Pedestrian Tracking Video (90 MB)

load("groundTruth.mat","gTruth");

Convert Ground Truth for Object Tracking

Convert the groundTruth object to the trackCLEARmetrics truth format. For the object tracking task each truth report is a structure with the fields Time, BoundingBox, and TruthID.

Obtain the label data from the groundTruth object.

labelTimeTable = gTruth.LabelData;

Calculate the total number of labels across all frames.

numLabels = sum(cellfun(@numel,labelTimeTable.Person));

Initialize the truth array in the trackCLEARmetrics format.

truthFormat = struct("Time",0,"BoundingBox",[0 0 0 0],"TruthID",0);
gTruthTracks = repmat(truthFormat,1,numLabels);

Convert from axis-aligned bounding boxes, labels, and timestamps to the trackCLEARmetrics format using the helper function helperGroundTruthToTrack.

gTruthTracks = helperGroundTruthToTrack(labelTimeTable,gTruthTracks);

Visualize the ground truth using the helperVisualizeTracks helper function, which annotates the video frames with the current and past truth record. Use this function to animate the true trajectories.

videoPath = gTruth.DataSource.Source;
helperVisualizeTracks(videoPath,gTruthTracks);

Next Steps

After you convert ground truth labeling data to the trackCLEARmetrics truth format, you can compare it to a tracking algorithm's result using the evaluate (Sensor Fusion and Tracking Toolbox) object function of the trackCLEARmetrics object. To learn how to perform multi-object tracking and evaluate results using trackCLEARmetrics, see the Multi-Object Tracking with DeepSORT (Sensor Fusion and Tracking Toolbox) example.

Supporting Functions

helperDownloadLabelVideo

Download the pedestrian labeling video.

function helperDownloadLabelVideo
videoURL = "https://ssd.mathworks.com/supportfiles/vision/data/PedestrianLabelingVideo.avi";
if ~exist("PPedestrianLabelingVideo.avi","file")
    disp("Downloading Pedestrian Tracking Video (90 MB)")
    websave("PedestrianLabelingVideo.avi",videoURL);
end
end

helperGroundTruthToTrack

Convert ground truth label data from the video labeler to the trackCLEARmetrics format.

function gTruthTracks = helperGroundTruthToTrack(labelTimeTable,gTruthTracks)
k = 1;
for i = 1:numel(labelTimeTable)
    videoTime = labelTimeTable.Time(i);
    frameLabels = labelTimeTable{i,1}{:};
    for j=1:numel(frameLabels)
        gTruthTracks(k).Time = seconds(videoTime);
        gTruthTracks(k).BoundingBox = frameLabels(j).Position;
        gTruthTracks(k).TruthID = frameLabels(j).ID;
        k = k+1;
    end
end
end

helperVisualizeTracks

Annotate video frames with current and past ground truth records.

function helperVisualizeTracks(videoPath,truths)
reader = VideoReader(videoPath);
pastTruths = [];
groundTruthHistoryDuration = 10;
for i=1:reader.NumFrames
    frame = readFrame(reader);

    % Read current truth from array.
    curTruths = truths(ismember([truths.Time], reader.CurrentTime-1));
    annotatedFrame = helperAnnotateGroundTruth(frame, curTruths, pastTruths);

    imshow(annotatedFrame);
    pause(0.1);

    % Add truth to pastTruths to display trajectory trails
    pastTruths = [pastTruths;curTruths(:)]; %#ok<AGROW>
    pastTruths([pastTruths.Time]<reader.CurrentTime-groundTruthHistoryDuration)=[];
end
end

helperAnnotateGroundTruth

Insert both current and past annotations into image frames.

function frame = helperAnnotateGroundTruth(frame, truths, pastTruths)
if ~isempty(truths)
    bbox = vertcat(truths.BoundingBox);
    truthlabels = [truths.TruthID];
    colors = helperGetColorByID(truths);
    frame = insertObjectAnnotation(frame, 'Rectangle',bbox, truthlabels,'Color',colors, 'TextBoxOpacity',0.8,'LineWidth',3,'FontSize',18);
end

if ~isempty(pastTruths)
    bbox = vertcat(pastTruths.BoundingBox);
    colors = helperGetColorByID(pastTruths);
    frame = insertShape(frame, 'filled-rectangle',bbox, 'Color',colors,'Opacity',0.2);
end
end

helperGetColorByID

Maintain and retrieve a specific color for each ID in the ground truth data.

function colors = helperGetColorByID(truths)
colors = zeros(numel(truths), 3);
coloroptions = 255*lines(7);
for i=1:numel(truths)
    colors(i,:) = coloroptions(mod(truths(i).TruthID, 7)+1,:);
end
end