Main Content

Resolve GPU Memory Issues

Training deep neural networks often requires significant memory resources. For best performance, take full advantage of your available GPU memory.

Out of GPU Memory Errors

Running out of GPU memory is a common issue when you train a network. When you run out of GPU memory, the software throws an error:

GPU out of memory. Try reducing 'MiniBatchSize' using the trainingOptions function.

Possible Solutions

If you run out of GPU memory during training, try these solutions and workarounds:

  • Reduce your mini-batch size until you no longer run out of memory during training. For built-in training, use the trainingOptions function to set the MiniBatchSize name-value argument. For custom training workflows, set the MiniBatchSize property of your minibatchqueue object.

  • Avoid using your GPU for other tasks. For example, avoid using one GPU across multiple instances of MATLAB®. Some GPU devices support the Tesla Compute Cluster (TCC) operating model on Windows® systems, which ensures your system only uses your GPU for computing and not for graphics. For more information, see Configure Your Hardware for GPU Performance (Parallel Computing Toolbox).

  • Reduce the size of your training data. For example, if you are training a network using image data, use an augmentedImageDatastore object to downscale or extract patches of the original images.

  • Reduce the size of your network. To reduce the size of your network, reduce the number of layers in your network or the number of learnable parameters in the layers. For example, if your network contains a convolution layer, reduce the filterSize or numFilters property.

  • Buy a replacement or an additional GPU, or rent a cloud instance with sufficient resources. For deep learning workflows in MATLAB, the most important GPU characteristics are memory and single-precision (FP32) processing power. For a comprehensive performance overview of NVIDIA® GPU cards, including single-precision processing power, see https://en.wikipedia.org/wiki/List_of_Nvidia_graphics_processing_units. For more information about connecting to cloud GPU resources, see Deep Learning in the Cloud.

For more information about how to deal with out of memory errors, not specific to GPU memory, see Resolve “Out of Memory” Errors.

See Also

| (Parallel Computing Toolbox)

Related Topics