Transition trainNetwork, SeriesNetwork, and DAGNetwork Code to dlnetwork Workflows

Recommended Workflow Examples

In most cases, taking code from examples and adapting it for your task is the easiest option. These examples show the recommended workflows for common deep learning tasks:

For greater flexibility or to reproduce algorithms exactly, you can write a custom training loop. For more information, see Custom Training Loops.

Update Training Code

The trainNetwork function uses output layers (such as classificationLayer and regressionLayer objects) to determine the loss function. The trainnet function instead takes a loss function as an input argument, so you do not need an output layer.

When updating your training code, note these key differences:

This table shows how to update typical trainNetwork code to use the trainnet function.

layers = [
    imageInputLayer([28 28 1])
    convolution2dLayer(5,20)
    reluLayer
    fullyConnectedLayer(10)
    softmaxLayer
    classificationLayer];

net = trainNetwork(data,layers,options);

layers = [
    imageInputLayer([28 28 1])
    convolution2dLayer(5,20)
    reluLayer
    fullyConnectedLayer(10)
    softmaxLayer];

net = trainnet(data,layers,"crossentropy",options);

layers = [
    featureInputLayer(12)
    fullyConnectedLayer(25)
    reluLayer
    fullyConnectedLayer(1)
    regressionLayer];

net = trainNetwork(X,T,layers,options);

layers = [
    featureInputLayer(12)
    fullyConnectedLayer(25)
    reluLayer
    fullyConnectedLayer(1)];

net = trainnet(X,T,layers,"mse",options);

layers = [
    imageInputLayer([28 28 1])
    convolution2dLayer(5,20)
    reluLayer
    fullyConnectedLayer(10)
    softmaxLayer
    classificationLayer(ClassWeights=weights)];

net = trainNetwork(data,layers,options);

layers = [
    imageInputLayer([28 28 1])
    convolution2dLayer(5,20)
    reluLayer
    fullyConnectedLayer(10)
    softmaxLayer];

weights = dlarray(weights,"C");
lossFcn = @(Y,T) crossentropy(Y,T,weights);
net = trainnet(data,layers,lossFcn,options);

Update Prediction Code

The classify and predict functions for SeriesNetwork and DAGNetwork objects are not recommended. For dlnetwork objects, use the minibatchpredict function for multiple observations, or the predict function for a single observation.

This table shows how to update prediction code.

Y = classify(net,X);

scores = minibatchpredict(net,X);
Y = scores2label(scores,classNames);

Y = predict(net,X);

Y = minibatchpredict(net,X);

Y = activations(net,X,layerName);

Y = minibatchpredict(net,X,Outputs=layerName);

% Or, for a single observation:
Y = predict(net,X,Outputs=layerName);

[net,Y] = classifyAndUpdateState(net,X);

[scores,state] = predict(net,X);
Y = scores2label(scores,classNames);
net.State = state;

[net,Y] = predictAndUpdateState(net,X);

[Y,state] = predict(net,X);
net.State = state;

Convert Existing Networks

If you have existing trained SeriesNetwork or DAGNetwork objects, then you can convert them to dlnetwork objects using the dag2dlnetwork function. This function converts any SeriesNetwork or DAGNetwork object to a dlnetwork object and removes the output layer.

net = dag2dlnetwork(trainedNet);

Update Network Architecture Code

If you construct network architectures using layerGraph objects, then you can build dlnetwork objects directly instead. The dlnetwork object supports the same network building functions (such as addLayers and connectLayers), so you can reuse the same function calls to edit the network.

layers = ...;
lgraph = layerGraph(layers);

layers = ...;
lgraph = addLayers(lgraph,layers);

lgraph = connectLayers(lgraph,"in","conv_skip");
lgraph = connectLayers(lgraph,"relu_skip","add/in2");

net = dlnetwork;

layers = ...;
net = addLayers(net,layers);

layers = ...;
net = addLayers(net,layers);

net = connectLayers(net,"in","conv_skip");
net = connectLayers(net,"relu_skip","add/in2");

When you create a neural network with branching, such as neural networks with skip connections, start with an empty dlnetwork object and build on it. This approach prevents the software from initializing the learnable parameters before you finish building the network. If the neural network does not have an input layer, is misconfigured, or is not complete, then the layer array input syntax dlnetwork(layers) fails because the software cannot initialize the learnable parameters of the network. Alternatively, to convert a layer array to a dlnetwork object without initializing the learnable parameters, use net = dlnetwork(layers,Initialize=false).

Working with Pretrained Image Networks

If you use pretrained network functions that return SeriesNetwork or DAGNetwork objects (such as alexnet, vgg16, or resnet50), then use the imagePretrainedNetwork function instead. This function returns a dlnetwork object and automatically adjusts the network for transfer learning workflows, so you do not need to edit the network layers.

net = googlenet;
Y = classify(net,X);

net = imagePretrainedNetwork("googlenet");
scores = minibatchpredict(net,X);
Y = scores2label(scores,classNames);

net = resnet50;
lgraph = layerGraph(net);

layer = fullyConnectedLayer(numClasses,Name="fc_new");
lgraph = replaceLayer(lgraph,"fc1000",layer);

layer = softmaxLayer(Name="softmax_new");
lgraph = replaceLayer(lgraph,"fc1000_softmax",layer);

layer = classificationLayer(Name="output_new");
lgraph = replaceLayer(lgraph,"ClassificationLayer_fc1000",layer);

net = trainNetwork(data,lgraph,options);

net = imagePretrainedNetwork("resnet50", ...
    NumClasses=numClasses);
net = trainnet(data,net,"crossentropy",options);

Summary of Recommendations

This table summarizes the functions and objects that are not recommended and their recommended replacements.

For more specific information, refer to the reference page of the affected functionality.