Transfer Learning

Transfer learning is commonly used in deep learning applications. You can take a pretrained network and use it as a starting point to learn a new task. Fine-tuning a network with transfer learning is usually much faster and easier than training a network with randomly initialized weights from scratch. You can quickly transfer learned features to a new task using a smaller number of training images.

Deep Network Designer has a selection of pretrained networks suitable for transfer learning with image data.

Image Classification

You can build an image classification network using Deep Network Designer by dragging layers from the Layer Library and connecting them. You can also create the network at the command line and then import the network into Deep Network Designer.

For example, create a network to train for image classification on a data set of 28-by-28 images divided into 10 classes.

inputSize = [28 28 1];
numClasses = 10;

layers = [
    imageInputLayer(inputSize)
    convolution2dLayer(5,20)
    batchNormalizationLayer
    reluLayer
    fullyConnectedLayer(numClasses)
    softmaxLayer];

deepNetworkDesigner(layers)

To adapt this network to your own data, set the InputSize of the image input layer to match your image input size and set the OutputSize of the fully connected layer to the number of classes in your data. For more complex classification tasks, create a deeper network. For more information, see Deep Networks.

For an example showing how to create and train an image classification network, see Get Started with Image Classification.

Sequence Classification

You can use Deep Network Designer to build a sequence network from scratch, or you can use one of the prebuilt untrained networks from the start page. Open the Deep Network Designer start page. Pause on Sequence-to-Label and click Open. Doing so opens a prebuilt network suitable for sequence classification problems.

You can adapt this sequence network for training with your data. Suppose you have data with 10 features and 5 classes. To adapt this network, select sequenceInputLayer and set the InputSize to 10.

Then, select the fullyConnectedLayer and set the OutputSize to 5, the number of classes.

For an example showing how to create and train a sequence classification network, see Get Started with Deep Network Designer.

Numeric Data Classification

If you have a data set of numeric features (for example, a collection of numeric data without spatial or time dimensions), then you can train a deep learning network using a feature input layer. For more information about the feature input layer, see featureInputLayer.

You can construct a suitable network using Deep Network Designer, or you can create the network at the command line and import the network into Deep Network Designer.

For example, create a network for numeric data with 10 classes, where each observation consists of 20 features.

inputSize = 20;
numClasses = 10;

layers = [
featureInputLayer(inputSize,Normalization="zscore")
fullyConnectedLayer(50)
batchNormalizationLayer
reluLayer
fullyConnectedLayer(numClasses)
softmaxLayer];

deepNetworkDesigner(layers)

To adapt this network to your own data, set the InputSize of the feature input layer to match the number of features in your data and set the OutputSize of the fully connected layer to the number of classes in your data. For more complex classification tasks, create a deeper network. For more information, see Deep Networks.

Convert Classification Network into Regression Network

You can convert a classification network into a regression network by adapting the final layers of the network. Conversion is useful when you want to take a pretrained classification network and retrain it for regression tasks.

For example, suppose you have a GoogLeNet pretrained network. To convert this network into a regression network with a single response, remove the softmax layer and set the OutputSize of the final fully connected layer to 1 (the number of responses).

If your output has multiple responses, change the OutputSize value of the fully connected layer to the number of responses.

Multiple-Input and Multiple-Output Networks

Deep Networks

Building large networks can be difficult, but you can use Deep Network Designer to speed up construction. You can work with blocks of layers at a time. Select multiple layers, then copy and paste or delete. For example, you can use blocks of layers to create multiple copies of groups of convolution, batch normalization, and ReLU layers.

For trained networks, copying layers also copies the weights and the biases.

You can also copy sub-networks from the workspace to connect up easily using the app. To import a network or layers into the app, click New > Import from workspace. Click Add to add the layers to the current network.

Check Network

To check your network and examine the layers in further detail, click Analyze. Investigate problems and examine the layer properties to resolve size mismatches in the network. Return to Deep Network Designer to edit layers, then check the results by clicking Analyze again. If Deep Learning Network Analyzer reports zero errors, then the edited network is ready for training.