identityLayer

Define Block Architecture

Write a function that returns a network layer representing a residual block.

function resblock = residualBlockLayer(name)

reslayers = dlnetwork;

layers = [
    identityLayer(Name="split")
    convolution2dLayer(3,32,Padding="same")
    batchNormalizationLayer
    reluLayer
    convolution2dLayer(3,32,Padding="same")
    batchNormalizationLayer
    additionLayer(2,Name="add")
    reluLayer];

reslayers = addLayers(reslayers,layers);
reslayers = connectLayers(reslayers,"split","add/in2");

resblock = networkLayer(reslayers,Name=name);

end

The residualBlockLayer function accepts a layer name and returns a networkLayer object representing a residual block. The function uses identityLayer with the connectLayers function to create a skip connection between the input of the residual block and the second input of its final layer.

Create and Analyze Residual Network

Create a residual network using residualBlockLayer to generate the residual blocks.

layers = [
    imageInputLayer([224 224 3])
    convolution2dLayer(7,32,Stride=2,Padding="same")
    batchNormalizationLayer
    reluLayer
    maxPooling2dLayer(3,Stride=2)
    residualBlockLayer("resBlock1")
    residualBlockLayer("resBlock2")
    globalAveragePooling2dLayer
    fullyConnectedLayer(5)
    softmaxLayer];

resnet = dlnetwork(layers)

resnet = 
  dlnetwork with properties:

         Layers: [10x1 nnet.cnn.layer.Layer]
    Connections: [9x2 table]
     Learnables: [22x3 table]
          State: [10x3 table]
     InputNames: {'imageinput'}
    OutputNames: {'softmax'}
    Initialized: 1

  View summary with summary.

resnet is a dlnetwork object that represents a residual network. resnet contains two networkLayer objects corresponding to two residual blocks.

Inspect Network

Use the expandLayers and analyzeNetwork functions to inspect the residual block layers in the Deep Network Designer app. The leftmost pane in Deep Network Designer shows the network layers and their connections.

xresnet = expandLayers(resnet);
analyzeNetwork(xresnet)

The names of the layers inside the first and second residual blocks begin with resBlock1: and resBlock2:, respectively. The first layer inside each block is an identity layer that accepts input from outside the residual block. Each identity layer has a skip connection to an addition layer. The names of the identity layers are resBlock1:split and resBlock2:split, and the names of the addition layers are resBlock1:add and resBlock2:add.