Deep Learning Layers Supported for Formal Verification
R2026bThis page lists the deep learning layers in MATLAB® that the verifyNetworkRobustness and estimateNetworkOutputBounds functions support. For a complete list of all deep learning
layers, see List of Deep Learning Layers.
The Recommendation column in the tables below indicates how well each layer works with formal verification:
✓✓ Strongly recommended — Produces tight verification bounds with efficient memory and computation.
✓ Recommended — Supported, but produces looser bounds or uses more resources. Consider replacing with a strongly recommended alternative.
Deep Learning Layers
Input Layers
| Layer | Description | Notes and Limitations | Recommendation |
|---|---|---|---|
| An image input layer inputs 2-D images to a neural network and applies data normalization. | Supported when Normalization is set to
"none"
(since R2022b) or
"zerocenter", "zscore",
"rescale-symmetric",
"rescale-zero-one"
(since R2023a). Custom normalization functions and split complex inputs are not
supported. | ✓✓ Strongly recommended | |
| A sequence input layer inputs sequence data to a neural network and applies data normalization. | Supported when | ✓✓ Strongly recommended |
| A feature input layer inputs feature data to a neural network and applies data normalization. Use this layer when you have a data set of numeric scalars representing features (data without spatial or time dimensions). | Supported when Normalization is set to
"none"
(since R2022b) or
"zerocenter", "zscore",
"rescale-symmetric",
"rescale-zero-one"
(since R2023a). Custom normalization functions and split complex inputs are not
supported. | ✓✓ Strongly recommended |
Convolution and Fully Connected Layers
| Layer | Description | Notes and Limitations | Recommendation |
|---|---|---|---|
| A 1-D convolutional layer applies sliding convolutional filters to 1-D input. | Supported when Dilation is set to
1 and PaddingValue is set to
0. Avoid padding, which can cause large memory
overheads during verification. | ✓✓ Strongly recommended |
| A 2-D convolutional layer applies sliding convolutional filters to 2-D input. | Supported when Dilation is set to [1
1] or 1 and
PaddingValue is set to 0. Avoid
padding, which can cause large memory overheads during
verification. | ✓✓ Strongly recommended |
| A 2-D grouped convolutional layer separates the input channels into groups and applies sliding convolutional filters. Use grouped convolutional layers for channel-wise separable (also known as depth-wise separable) convolution. | Supported when Dilation is set to [1
1] or 1 and
PaddingValue is set to 0. | ✓✓ Strongly recommended |
| A fully connected layer multiplies input vectors by a weight matrix and then adds a bias vector. | Supported when the operation dimension is set to
"spatial-channel" (default). | ✓✓ Strongly recommended |
Sequence Layers
| Layer | Description | Notes and Limitations | Recommendation |
|---|---|---|---|
| A sequence input layer inputs sequence data to a neural network and applies data normalization. | Supported when | ✓✓ Strongly recommended |
| An LSTM layer is an RNN layer that learns long-term dependencies between time steps in time-series and sequence data. | Recurrent layers have many nonlinearities that compound verification
error over time steps, producing increasingly loose bounds for long
sequences. Prefer convolution1dLayer with 1-D pooling layers. | ✓ Recommended |
| An LSTM projected layer is an RNN layer that learns long-term dependencies between time steps in time-series and sequence data using projected learnable weights. | Recurrent layers have many nonlinearities that compound verification
error over time steps, producing increasingly loose bounds for long
sequences. Prefer convolution1dLayer with 1-D pooling layers. | ✓ Recommended |
| A bidirectional LSTM (BiLSTM) layer is an RNN layer that learns bidirectional long-term dependencies between time steps of time-series or sequence data. These dependencies can be useful when you want the RNN to learn from the complete time series at each time step. | Recurrent layers have many nonlinearities that compound verification
error over time steps, producing increasingly loose bounds for long
sequences. Prefer convolution1dLayer with 1-D pooling layers. | ✓ Recommended |
| A GRU layer is an RNN layer that learns dependencies between time steps in time-series and sequence data. | Recurrent layers have many nonlinearities that compound verification
error over time steps, producing increasingly loose bounds for long
sequences. Prefer convolution1dLayer with 1-D pooling layers. | ✓ Recommended |
| A GRU projected layer is an RNN layer that learns dependencies between time steps in time-series and sequence data using projected learnable weights. | Recurrent layers have many nonlinearities that compound verification
error over time steps, producing increasingly loose bounds for long
sequences. Prefer convolution1dLayer with 1-D pooling layers. | ✓ Recommended |
| A 1-D convolutional layer applies sliding convolutional filters to 1-D input. | Supported when Dilation is set to
1 and PaddingValue is set to
0. | ✓✓ Strongly recommended |
| A 1-D max pooling layer performs downsampling by dividing the input into 1-D pooling regions, then computing the maximum of each region. | Consider replacing with averagePooling1dLayer. Verification of max pooling on GPU may
be nondeterministic. Set
deep.gpu.deterministicAlgorithms(true). | ✓ Recommended |
| A 1-D average pooling layer performs downsampling by dividing the input into 1-D pooling regions, then computing the average of each region. | Supported when PaddingValue is set to
0. | ✓✓ Strongly recommended |
| A 1-D global max pooling layer performs downsampling by outputting the maximum of the time or spatial dimensions of the input. | Consider replacing with globalAveragePooling1dLayer. | ✓ Recommended |
Activation Layers
| Layer | Description | Notes and Limitations | Recommendation |
|---|---|---|---|
| A ReLU layer performs a threshold operation to each element of the input, where any value less than zero is set to zero. | You can use the α-CROWN algorithm to produce tighter bounds for networks with this layer. | ✓✓ Strongly recommended | |
| A leaky ReLU layer performs a threshold operation, where any input value less than zero is multiplied by a fixed scalar. | You can use the α-CROWN algorithm to produce tighter bounds for networks with this layer. | ✓✓ Strongly recommended | |
| A clipped ReLU layer performs a threshold operation, where any input value less than zero is set to zero and any value above the clipping ceiling is set to that clipping ceiling. | Avoid as an intermediate layer. Recommended as the final layer in the network. | ✓ Recommended |
| A hyperbolic tangent (tanh) activation layer applies the tanh function on the layer inputs. | ✓✓ Strongly recommended | ||
| A swish activation layer applies the swish function on the layer inputs. | ✓✓ Strongly recommended | |
| A softplus layer applies the softplus activation function on the layer inputs. | ✓✓ Strongly recommended | |
| A softmax layer applies a softmax function to the input. | Avoid as an intermediate layer. Exponentiation can loosen bounds. Recommended as the final layer in the network. | ✓ Recommended |
| A sigmoid layer applies a sigmoid function to the input such that the output is bounded in the interval (0,1). | ✓✓ Strongly recommended | |
| A PReLU layer performs a threshold operation, where for each channel, any input value less than zero is multiplied by a scalar learned at training time. | You can use the α-CROWN algorithm
to produce tighter bounds for networks with this layer if the | ✓✓ Strongly recommended |
Normalization Layers
| Layer | Description | Notes and Limitations | Recommendation |
|---|---|---|---|
| A batch normalization layer normalizes a mini-batch of data across all observations for each channel independently. To speed up training of the convolutional neural network and reduce the sensitivity to network initialization, use batch normalization layers between convolutional layers and nonlinearities, such as ReLU layers. | ✓✓ Strongly recommended | |
| A layer normalization layer normalizes a mini-batch of data across all channels for each observation independently. To speed up training of recurrent and multilayer perceptron neural networks and reduce the sensitivity to network initialization, use layer normalization layers after the learnable layers, such as LSTM and fully connected layers. | Supported for real-valued statistics and when the operation dimension is
set to "channel-only" or when the normalization is taken
over the channel dimension. Consider replacing with batchNormalizationLayer, which is faster, uses less memory,
and produces tighter bounds. | ✓ Recommended |
| An inverse normalization layer transforms predictions for neural networks trained using normalized targets. | Supported for real-valued statistics and when the operation dimension is
set to "channel" (default). | ✓✓ Strongly recommended |
Utility Layers
| Layer | Description | Notes and Limitations | Recommendation |
|---|---|---|---|
| A dropout layer randomly sets input elements to zero with a given probability. | ✓✓ Strongly recommended | |
| A spatial dropout layer randomly selects input channels with a given probability, and sets all its elements to zero during training. | ✓✓ Strongly recommended | |
| An identity layer is a layer whose output is identical to its input. You can use an identity layer to create a skip connection, which allows the input to skip one or more layers in the main branch of a neural network. For more information about skip connections, see More About. | ✓✓ Strongly recommended | |
| A network layer contains a nested network. Use network layers to simplify building large networks that contain repeating components. | The network layer must be expandable. For more information, see expandLayers. Evaluate the verifiability of the layer contents
separately. | ✓✓ Strongly recommended |
| A scaling layer linearly scales and offsets the input data. | Scale and offset values must be real. | ✓✓ Strongly recommended |
| A projected layer is a compressed neural network layer resulting from projection. To compress a neural network using projection, use the | Evaluate the verifiability of the layer contents separately. | ✓✓ Strongly recommended |
Pooling and Unpooling Layers
| Layer | Description | Notes and Limitations | Recommendation |
|---|---|---|---|
| A 1-D average pooling layer performs downsampling by dividing the input into 1-D pooling regions, then computing the average of each region. | Supported when PaddingValue is set to
0. Avoid padding, which can cause large memory
overheads during verification. | ✓✓ Strongly recommended |
| A 2-D average pooling layer performs downsampling by dividing the input into rectangular pooling regions, then computing the average of each region. | Supported when PaddingValue is set to
0. Avoid padding, which can cause large memory
overheads during verification. | ✓✓ Strongly recommended |
| A 1-D global average pooling layer performs downsampling by outputting the average of the time or spatial dimensions of the input. | ✓✓ Strongly recommended | |
| A 2-D global average pooling layer performs downsampling by computing the mean of the height and width dimensions of the input. | ✓✓ Strongly recommended | |
| A 1-D max pooling layer performs downsampling by dividing the input into 1-D pooling regions, then computing the maximum of each region. | Consider replacing with averagePooling1dLayer. Verification of max pooling on GPU may
be nondeterministic. Set
deep.gpu.deterministicAlgorithms(true). | ✓ Recommended |
| A 2-D max pooling layer performs downsampling by dividing the input into rectangular pooling regions, then computing the maximum of each region. | Consider replacing with averagePooling2dLayer. Verification of max pooling on GPU may
be nondeterministic. Set
deep.gpu.deterministicAlgorithms(true). | ✓ Recommended |
| A 1-D global max pooling layer performs downsampling by outputting the maximum of the time or spatial dimensions of the input. | Consider replacing with globalAveragePooling1dLayer. | ✓ Recommended |
| A 2-D global max pooling layer performs downsampling by computing the maximum of the height and width dimensions of the input. | Consider replacing with globalAveragePooling2dLayer. | ✓ Recommended |
| A flatten layer collapses the spatial dimensions of the input into the channel dimension. | ✓✓ Strongly recommended |
Combination Layers
| Layer | Description | Notes and Limitations | Recommendation |
|---|---|---|---|
| An addition layer adds inputs from multiple neural network layers element-wise. | ✓✓ Strongly recommended | |
| A depth concatenation layer takes inputs that have the same height and width and concatenates them along the channel dimension. | ✓✓ Strongly recommended | |
| A concatenation layer takes inputs and concatenates them along a specified dimension. The inputs must have the same size in all dimensions except the concatenation dimension. | Supported when concatenation is over the channel dimension. | ✓✓ Strongly recommended |
| A multiplication layer multiplies inputs from multiple neural network layers element-wise. | Number of inputs must be 2. Consider replacing with additionLayer. | ✓ Recommended |
PyTorch, ONNX, and TensorFlow Layers
| Layer | Description | Notes and Limitations | Recommendation |
|---|---|---|---|
| Built-in PyTorch® layer. | ✓✓ Strongly recommended | |
| Built-in ONNX™ layer. | Supported when DataFormat is set to
"CB" or "SSCB". The data format is
commonly set by the InputDataFormats and
OutputDataFormats options of the importNetworkFromONNX function. | ✓✓ Strongly recommended |
| Built-in ONNX layer. | ✓✓ Strongly recommended | |
| Built-in ONNX layer. | ✓✓ Strongly recommended | |
| Built-in ONNX layer. | ✓✓ Strongly recommended | |
| Built-in ONNX layer. | ✓✓ Strongly recommended | |
| Built-in ONNX layer. | ✓✓ Strongly recommended | |
| Built-in TensorFlow™-Keras layer. | ✓✓ Strongly recommended | |
| Built-in TensorFlow-Keras layer. | ✓✓ Strongly recommended |
See Also
verifyNetworkRobustness | estimateNetworkOutputBounds
