Main Content

summarize

Summarize object detection performance metrics at data set and class level

Since R2024b

collapse all in page

Syntax

[summaryDataset,summaryClass] = summarize(metrics)
[summaryDataset,summaryClass] = summarize(metrics,MetricName=metricName)

Description

[summaryDataset,summaryClass] = summarize(metrics) summarizes the object detection evaluation metrics across all classes of the data set summaryDataset, and for each class in the data set summaryClass.

example

[summaryDataset,summaryClass] = summarize(metrics,MetricName=metricName) specifies the metrics category to return in the metrics summary.

Examples

collapse all

Open Live Script

Load a table containing image file names and ground truth bounding box labels into the workspace. The first column contains the image file names, and the remaining columns contain the labeled bounding boxes.

data = load("vehicleTrainingData.mat");
trainingData = data.vehicleTrainingData;

Set the value of the dataDir variable as the location where the vehicleTrainingData.mat file is located. Load the test data into a local vehicle data folder.

dataDir = fullfile(toolboxdir("vision"),"visiondata");
trainingData.imageFilename = fullfile(dataDir,trainingData.imageFilename);

Create an imageDatastore using the files from the table.

imds = imageDatastore(trainingData.imageFilename);

Create a boxLabelDatastore using the label column from the table.

blds = boxLabelDatastore(trainingData(:,2:end));

Load a pretrained YOLO v2 object detector, trained to detect vehicles, into the workspace.

vehicleDetector = load("yolov2VehicleDetector.mat");
detector = vehicleDetector.detector;

Run the detector on the test images. Set the detection threshold to a low value to detect as many objects as possible. This enables you to evaluate the detector precision across the full range of recall values.

results = detect(detector,imds,Threshold=0.01);

Compute metrics for evaluating the performance of the object detector.

metrics = evaluateObjectDetection(results,blds);

Summarize the object detection metrics for the vehicle class using the summarize object function.

[summaryDataset,summaryClass] = summarize(metrics);
summaryClass
summaryClass=1×3 table
               NumObjects    APOverlapAvg     AP0.5 
               __________    ____________    _______

    vehicle       336          0.99096       0.99096

Input Arguments

collapse all

Object detection performance metrics, specified as an objectDetectionMetrics object.

Metrics category to return in the metrics summary, specified as one of these options.

  • "all" – Returns the average precision and mean average precision metric values, aggregated over each overlap threshold, along with all additional metrics.

  • "AP" – Returns only the average precision across all classes at each overlap threshold as a numThresh-by-1 numeric vector, and the mean average precision (mAP) computed across all overlap thresholds. numThresh is the number of overlap thresholds specified by the OverlapThreshold property of the objectDetectionMetrics object.

  • "LAMR" – Returns only the log-average miss rate (LAMR) computed at each overlap threshold as a numThresh-by-1 numeric vector, and the mean of the log-average miss rate (mLAMR) computed across all overlap thresholds.

  • "AOS" – Returns only the average orientation similarity (AOS) computed at each overlap threshold as a numThresh-by-1 array, and the mean of the average orientation similarity (mAOS) computed across all overlap thresholds.

    To specify "AOS" as the metric to return, your input data must contain rotated bounding boxes.

Output Arguments

collapse all

Metric summary over the data set, returned as a 1-by-numMetrics table, where numMetrics is the number of specified metrics.

The columns of the table depend on the value of the metricName argument.

  • "all" —

    NumObjectsmAPOverlapAvgmAPmLAMROverlapAvgmLAMRmAOSOverlapAvgmAOS

    Number of objects in the ground truth data, returned as a positive integer.

    Mean average precision (mAP) averaged over all thresholds specified by the OverlapThreshold property of metrics, returned as a numeric scalar.

    Mean average precision (mAP), or average precision averaged over all classes, computed at all overlap thresholds specified by the OverlapThreshold property of metrics, returned as a numThresh-by-1 numeric vector. numThresh is the number of overlap thresholds.

    Mean log-average miss rate (mLAMR) averaged over all classes and all overlap thresholds specified by the OverlapThreshold property of metrics, returned as a numeric scalar.

    Mean log-average miss rate (mLAMR) averaged over all classes, computed at all overlap thresholds, specified by the OverlapThreshold property of metrics, returned as a numThresh-by-1 numeric vector.

    Mean average orientation similarity (mAOS) averaged over all classes, and averaged over all overlap thresholds specified by the OverlapThreshold property of metrics, returned as a numeric scalar.

    Mean average orientation similarity (mAOS) averaged over all classes, computed at all overlap thresholds specified by the OverlapThreshold property of metrics, returned as a numThresh-by-1 numeric vector.

    To return the "mAOSOverlapAvg" and "mAOS" columns, your input data must contain rotated bounding boxes.

  • "AP" —

    NumObjectsmAPOverlapAvgmAP

    Number of objects in the ground truth data, returned as a positive integer.

    Mean average precision (mAP) averaged over all thresholds specified by the OverlapThreshold property of metrics, returned as a numeric scalar.

    Mean average precision (mAP), or average precision averaged over all classes, computed at all overlap thresholds specified by the OverlapThreshold property of metrics, returned as a numThresh-by-1 numeric vector. numThresh is the number of overlap thresholds.

  • "LAMR" —

    mLAMROverlapAvgmLAMR

    Mean log-average miss rate (mLAMR) averaged over all classes and all overlap thresholds specified by the OverlapThreshold property of metrics, returned as a positive scalar.

    Mean log-average miss rate (mLAMR) averaged over all classes, computed at all overlap thresholds, specified by the OverlapThreshold property of metrics, returned as a numThresh-by-1 numeric vector.

  • "AOS" —

    mAOSOverlapAvgmAOS

    Mean average orientation similarity (mAOS) averaged over all classes and all overlap thresholds specified by the OverlapThreshold property of metrics, returned as a numeric scalar.

    Mean average orientation similarity (mAOS) averaged over all classes, computed at all overlap thresholds specified by the OverlapThreshold property of metrics, returned as a numThresh-by-1 numeric vector.

Metrics summary per class, returned as a numClasses-by-numMetrics table, where numClasses is the number of classes, and numMetrics is the number of specified metrics.

The columns of the table depend on the value of the metricName argument.

  • "all" —

    NumObjectsAPOverlapAvgAPLAMROverlapAvgLAMRAOSOverlapAvgAOS

    Number of objects in the ground truth data for each class, returned as a positive integer.

    Average precision (AP) for each class averaged over all thresholds specified by the OverlapThreshold property of metrics, returned as a numeric scalar.

    Average precision (AP) for each class, computed at all overlap thresholds specified by the OverlapThreshold property of metrics, returned as a numThresh-by-1 numeric vector. numThresh is the number of overlap thresholds.

    Log-average miss rate (LAMR) for each class, averaged over all overlap thresholds specified by the OverlapThreshold property of metrics.

    Log-average miss rate (LAMR) for each class, computed at all overlap thresholds specified by the OverlapThreshold property of metrics, returned as a numThresh-by-1 numeric vector.

    Average orientation similarity (AOS) for each class, averaged over all overlap thresholds specified by the OverlapThreshold property of metrics, returned as a numeric scalar.

    Average orientation similarity (AOS) for each class, computed at all overlap thresholds specified by the OverlapThreshold property of metrics, returned as a numThresh-by-1 numeric vector.

    To return the "AOSOverlapAvg" and "AOS" columns, your input data must contain rotated bounding boxes.

  • "AP" —

    NumObjectsAPOverlapAvgAP

    Number of objects in the ground truth data for each class, returned as a positive integer.

    Average precision (AP) for each class averaged over all thresholds specified by the OverlapThreshold property of metrics, returned as a numeric scalar.

    Average precision (AP) for each class, computed at all overlap thresholds specified by the OverlapThreshold property of metrics, returned as a numThresh-by-1 numeric vector. numThresh is the number of overlap thresholds.

  • "LAMR" —

    LAMROverlapAvgLAMR

    Log-average miss rate (LAMR) for each class, averaged over all overlap thresholds specified by the OverlapThreshold property of metrics.

    Log-average miss rate (LAMR) for each class, computed at all overlap thresholds specified by the OverlapThreshold property of metrics, returned as a numThresh-by-1 numeric vector.

  • "AOS" —

    AOSOverlapAvgAOS

    Average orientation similarity (AOS) for each class, averaged over all overlap thresholds specified by the OverlapThreshold property of metrics, returned as a numeric scalar.

    Average orientation similarity (AOS) for each class, computed at all overlap thresholds specified by the OverlapThreshold property of metrics, returned as a numThresh-by-1 numeric vector.

Version History

Introduced in R2024b

See Also

|