Verification

Train robust networks and verify network robustness

Deep learning verification is a set of techniques for assessing the properties of deep neural networks. For example, you can verify the robustness properties of a network, compute network output bounds, find adversarial examples, detect out-of-distribution data, and verify compliance with industry standards.

The Deep Learning Toolbox Verification Library support package enables testing robustness properties of deep learning networks.

Use the verifyNetworkRobustness function to verify network robustness to adversarial examples. A network is robust to adversarial inputs if the predicted class of the network does not change when the input is perturbed between specified lower and upper input bounds. For a set of input bounds, the function checks if the network is robust to adversarial examples between those input bounds and returns either verified, violated, or unproven.
Use the estimateNetworkOutputBounds function to estimate the range of output values that the network returns when the input is between specified lower and upper bounds. Use this function to estimate how sensitive the network predictions are to input perturbation.
Use the networkDistributionDiscriminator function to create a distribution discriminator that separates data into in- and out-of-distribution.

Functions

`estimateNetworkOutputBounds`	Estimate output bounds of deep learning network (Since R2022b)
`verifyNetworkRobustness`	Verify adversarial robustness of deep learning network (Since R2022b)
`networkDistributionDiscriminator`	Deep learning distribution discriminator (Since R2023a)
`isInNetworkDistribution`	Determine whether data is within the distribution of the network (Since R2023a)
`distributionScores`	Distribution confidence scores (Since R2023a)
`drise`	Explain object detection network predictions using D-RISE (Since R2024a)
`deep.gpu.deterministicAlgorithms`	Set determinism of deep learning operations on the GPU to get reproducible results (Since R2024b)

Objects

`BaselineDistributionDiscriminator`	Baseline distribution discriminator (Since R2023a)
`EnergyDistributionDiscriminator`	Energy distribution discriminator (Since R2023a)
`ODINDistributionDiscriminator`	ODIN distribution discriminator (Since R2023a)
`HBOSDistributionDiscriminator`	HBOS distribution discriminator (Since R2023a)

Topics

Verification of Neural Networks
Learn about verification of neural networks using Deep Learning Toolbox™ Verification Library.
Verify Robustness of Deep Learning Neural Network
This example shows how to verify the adversarial robustness of a deep learning neural network.
Verify Robustness of ONNX Network
This example shows how to verify the adversarial robustness of an imported ONNX™ deep neural network. (Since R2024a)
Deep Learning Visualization Methods
Learn about and compare deep learning visualization methods.
Out-of-Distribution Detection for Deep Neural Networks
This example shows how to detect out-of-distribution (OOD) data in deep neural networks.
Verify an Airborne Deep Learning System
This example shows how to verify a deep learning system for airborne applications and is based on the work in [5,6,7], which includes the development and verification activities required by DO-178C [1], ARP4754A [2], and prospective EASA and FAA guidelines [3,4]. (Since R2023b)

Featured Examples

Generate Untargeted and Targeted Adversarial Examples for Image Classification

Use the fast gradient sign method (FGSM) and the basic iterative method (BIM) to generate adversarial examples for a pretrained neural network.

Open Live Script

Train Image Classification Network Robust to Adversarial Examples

Train a neural network that is robust to adversarial examples using fast gradient sign method (FGSM) adversarial training.

Open Live Script

Train Robust Deep Learning Network with Jacobian Regularization

Train a neural network that is robust to adversarial examples using a Jacobian regularization scheme.

Open Live Script

Generate Adversarial Examples for Semantic Segmentation

Generate adversarial examples for a semantic segmentation network using the basic iterative method (BIM).

Open Live Script

Explore Semantic Segmentation Network Using Grad-CAM

Explore the predictions of a pretrained semantic segmentation network using Grad-CAM.

Open Live Script

Interpret Deep Network Predictions on Tabular Data Using LIME

Use the locally interpretable model-agnostic explanations (LIME) technique to understand the predictions of a deep neural network classifying tabular data. You can use the LIME technique to understand which predictors are most important to the classification decision of a network.