Custom Layers

Define custom layers for deep learning

For most tasks, you can use built-in layers. If there is not a built-in layer that you need for your task, then you can define your own custom layer. You can define custom layers with learnable and state parameters. After defining a custom layer, you can check that the layer is valid, GPU compatible, and outputs correctly defined gradients. For a list of supported layers, see List of Deep Learning Layers.

Functions

expand all

Custom Layers

`functionLayer`	Function layer (Since R2021b)
`checkLayer`	Check validity of custom or function layer
`setLearnRateFactor`	Set learn rate factor of layer learnable parameter
`setL2Factor`	Set L₂ regularization factor of layer learnable parameter
`getLearnRateFactor`	Get learn rate factor of layer learnable parameter
`getL2Factor`	Get L₂ regularization factor of layer learnable parameter
`networkDataLayout`	Deep learning network data layout for learnable parameter initialization (Since R2022b)

Network Assembly

`dlnetwork`	Deep learning neural network
`findPlaceholderLayers`	Find placeholder layers in network architecture imported from Keras or ONNX
`replaceLayer`	Replace layer in neural network
`PlaceholderLayer`	Layer replacing an unsupported Keras or ONNX layer

Topics

Custom Layers Overview

Define Custom Deep Learning Layers
Learn how to define custom deep learning layers.
Check Custom Layer Validity
Learn how to check the validity of custom deep learning layers.

Define Custom Layers

Define Custom Deep Learning Layer with Learnable Parameters
This example shows how to define a SReLU layer and use it in a convolutional neural network.
Define Custom Deep Learning Layer with Multiple Inputs
This example shows how to define a custom weighted addition layer and use it in a convolutional neural network.
Define Custom Deep Learning Layer with Formatted Inputs
This example shows how to define a custom layer with formatted dlarray inputs.
Define Custom Recurrent Deep Learning Layer
This example shows how to define a peephole LSTM layer and use it in a neural network.
Specify Custom Layer Backward Function
This example shows how to define a SReLU layer and specify a custom backward function.
Custom Layer Function Acceleration
Accelerate custom layer forward and predict functions by caching and reusing traces.
Define Custom Deep Learning Layer for Code Generation
This example shows how to define a SReLU layer that supports code generation.

Network Composition and Nested Layers

Deep Learning Network Composition
Define custom layers that contain neural networks.
Define Nested Deep Learning Layer Using Network Composition
This example shows how to define a nested custom deep learning layer.
Train Network with Custom Nested Layers
This example shows how to create and train a network with nested layers defined using network composition.
Weight Tying Using Nested Layer
This example shows how to implement weight tying using a nested layer.