Main Content

anomalyDetectionMetrics

Anomaly detection metrics

expand all in page

    Description

    An anomalyDetectionMetrics object encapsulates instance segmentation quality metrics for a set of images.

    Note

    This object requires the Computer Vision Toolbox™ Automated Visual Inspection Library. You can install the Computer Vision Toolbox Automated Visual Inspection Library from Add-On Explorer. For more information about installing add-ons, see Get and Manage Add-Ons.

    Creation

    Create an anomalyDetectionMetrics object using the evaluateAnomalyDetection function.

    Properties

    expand all

    This property is read-only.

    Confusion matrix, specified as a 2-by-2 table. Each table element (i, j) is the count of pixels known to belong to class i but predicted to belong to class j.

    This property is read-only.

    Normalized confusion matrix, specified as a 2-by-2 table. NormalizedConfusionMatrix represents a confusion matrix normalized by the number of pixels known to belong to each class. Each table element (i, j) is the count of pixels known to belong to class i but predicted to belong to class j, divided by the total number of pixels predicted in class i. Elements are in the range [0, 1].

    This property is read-only.

    Metrics aggregated over the data set, specified as a 1-by-8 table with columns corresponding to these metrics:

    • GlobalAccuracy — Ratio of correctly classified pixels to total pixels, regardless of class.

    • MeanAccuracy — Ratio of correctly classified pixels in each class to total pixels, averaged over all classes.

    • Precison — Positive predictive value.

    • Recall — True positive (TP) rate.

    • Specificity — True negative (TN) rate.

    • F1Score — Harmonic mean of precision and recall.

    • FalsePositiveRate — False positive (FP) rate, or fallout.

    • FalseNegativeRate — False negative (FN) rate, or miss.

    This property is read-only.

    Anomaly detection metrics for each class, specified as a 2-by-2 table. The table has rows "Normal" and "Anomaly" and columns corresponding to these metrics:

    • Accuracy — Fraction of correctly classified images for each class ("Normal" or "Anomaly").

    • AccuracyPerSubClass — Fraction of correctly classified images for each subclass within the "Normal" and "Anomaly" class. The value of AccuracyPerSubClass is a table with C rows and one column, where C is the number of subclasses.

    Examples

    collapse all

    This example uses:

    Open Live Script

    Load test images and corresponding labels, then create a datastore that reads the test data. The data set consists of grayscale images of handwritten digits 0–9.

    [Xtest,catLabels] = digitTest4DArrayData;
dsTest = arrayDatastore(Xtest,IterationDimension=4);

    Load a pretrained FCDD anomaly detector. This detector has been trained to classify the digit 8 as normal and all other digits as anomalies. Therefore, specify the set of positive labels as the set of digits between 0 and 9, excluding 8.

    load("digit8AnomalyDetector.mat");
posLabels = setdiff(string(0:9),"8");

    Classify each test image as normal or anomalous.

    labelsPred = classify(detector,dsTest);

    Calculate metrics for the anomaly detection results.

    metrics = evaluateAnomalyDetection(labelsPred,catLabels,posLabels)
    Evaluating anomaly detection results
------------------------------------
* Finalizing... Done.
* Data set metrics:

    GlobalAccuracy    MeanAccuracy    Precision    Recall    Specificity    F1Score    FalsePositiveRate    FalseNegativeRate
    ______________    ____________    _________    ______    ___________    _______    _________________    _________________

        0.7662           0.839         0.98971     0.748        0.93        0.85204          0.07                 0.252      

    metrics = 
  anomalyDetectionMetrics with properties:

              ConfusionMatrix: [2x2 table]
    NormalizedConfusionMatrix: [2x2 table]
               DataSetMetrics: [1x8 table]
                 ClassMetrics: [2x2 table]

    Display the confusion matrix.

    cm = metrics.ConfusionMatrix
    cm=2×2 table
               Normal    Anomaly
               ______    _______

    Normal       465        35  
    Anomaly     1134      3366

    Version History

    Introduced in R2022b

    See Also