Ebook

Chapter 3

Build and Train a Deep Learning Network

For the hex nuts inspection example, we want to train a convolutional neural network (CNN).

A CNN is a type of “deep” neural network that can work directly with structured data, such as images. It consists of an input layer, several hidden layers, and an output layer. The layers are interconnected via nodes, or neurons.

Here is a simple architecture for a CNN.

Figure 1

As the data travels through the network, some of the layers (convolution, ReLU, and pooling) learn features from the image, while the softmax layer provides the classification.

Two Ways to Train a Network

You can train a deep learning network from scratch or fine-tune a pretrained network.

Training from Scratch

This method produces the most accurate results, but it is also the most challenging. It is good for new applications, or ones that will have a large number of output categories. But it requires huge amounts of data and expertise in deep learning, which is why most people try the second approach.

Fine-Tuning a Pretrained Network

This method uses transfer learning. The idea is that you take an existing network, like AlexNet or GoogLeNet, and feed in new data containing previously unknown classes. After adjusting the network, you can then perform a new task.

Transfer learning is much quicker and easier than training from scratch. It also has the advantage of needing much less data (thousands rather than millions of images).

  • Q. With so many pretrained networks to choose from, how do I decide which one will work best, and how do I know what parameters will provide best results?

  • A. Use the Experiment Manager app.

Experiment Manager

The Experiment Manager app enables you to create a deep learning experiment to train networks under various initial conditions and compare the results. For example, you can use deep learning experiments to:

  • Sweep through a range of hyperparameter values
  • Compare the results of using different datasets to train a network
  • Test different architectures by reusing the same set of training data on several networks

The Experiment Manager provides visualization tools such as training plots and confusion matrices, as well as filters to refine your experiment results.