深度学习自定义训练循环

自定义深度学习训练循环和损失函数

如果 trainingOptions 函数不提供任务所需的训练选项，或者自定义输出层不支持所需的损失函数，则您可以定义自定义训练循环。对于无法使用层图创建的网络，可以将自定义网络定义为函数。要了解详细信息，请参阅定义自定义训练循环、损失函数和网络。

函数

全部展开

自定义网络

`dlnetwork`	Deep learning network for custom training loops
`resetState`	Reset state parameters of neural network
`plot`	Plot neural network architecture
`addInputLayer`	Add input layer to network
`addLayers`	Add layers to layer graph or network
`removeLayers`	Remove layers from layer graph or network
`connectLayers`	Connect layers in layer graph or network
`disconnectLayers`	Disconnect layers in layer graph or network
`replaceLayer`	Replace layer in layer graph or network
`summary`	Print network summary
`initialize`	Initialize learnable and state parameters of a `dlnetwork`
`networkDataLayout`	Deep learning network data layout for learnable parameter initialization

自定义训练循环

`forward`	Compute deep learning network output for training
`predict`	Compute deep learning network output for inference
`adamupdate`	Update parameters using adaptive moment estimation (Adam)
`rmspropupdate`	Update parameters using root mean squared propagation (RMSProp)
`sgdmupdate`	Update parameters using stochastic gradient descent with momentum (SGDM)
`dlupdate`	Update parameters using custom function
`minibatchqueue`	Create mini-batches for deep learning
`onehotencode`	Encode data labels into one-hot vectors
`onehotdecode`	Decode probability vectors into class labels
`padsequences`	Pad or truncate sequence data to same length
`trainingProgressMonitor`	Monitor and plot training progress for deep learning custom training loops

自动微分

`dlarray`	Deep learning array for customization
`dlgradient`	Compute gradients for custom training loops using automatic differentiation
`dlfeval`	Evaluate deep learning model for custom training loops
`dims`	Dimension labels of `dlarray`
`finddim`	Find dimensions with specified label
`stripdims`	Remove `dlarray` data format
`extractdata`	Extract data from `dlarray`
`isdlarray`	Check if object is `dlarray`
`functionToLayerGraph`	(To be removed) Convert deep learning model function to a layer graph

深度学习运算

`dlconv`	Deep learning convolution
`dltranspconv`	Deep learning transposed convolution
`lstm`	Long short-term memory
`gru`	Gated recurrent unit
`attention`	Dot-product attention
`embed`	Embed discrete data
`fullyconnect`	Sum all weighted input data and apply a bias
`dlode45`	Deep learning solution of nonstiff ordinary differential equation (ODE)
`relu`	Apply rectified linear unit activation
`leakyrelu`	Apply leaky rectified linear unit activation
`gelu`	Apply Gaussian error linear unit (GELU) activation
`batchnorm`	Normalize data across all observations for each channel independently
`crosschannelnorm`	Cross channel square-normalize using local responses
`groupnorm`	Normalize data across grouped subsets of channels for each observation independently
`instancenorm`	Normalize across each channel for each observation independently
`layernorm`	Normalize data across all channels for each observation independently
`avgpool`	Pool data to average values over spatial dimensions
`maxpool`	Pool data to maximum value
`maxunpool`	Unpool the output of a maximum pooling operation
`softmax`	Apply softmax activation to channel dimension
`sigmoid`	应用 sigmoid 激活

深度学习损失运算

`sigmoid`	应用 sigmoid 激活
`crossentropy`	Cross-entropy loss for classification tasks
`l1loss`	L₁ loss for regression tasks
`l2loss`	L₂ loss for regression tasks
`huber`	Huber loss for regression tasks
`mse`	Half mean squared error
`ctc`	Connectionist temporal classification (CTC) loss for unaligned sequence classification

深度学习函数加速

`dlaccelerate`	Accelerate deep learning function for custom training loops
`AcceleratedFunction`	Accelerated deep learning function
`clearCache`	Clear accelerated deep learning function trace cache

主题

自定义训练循环

Train Deep Learning Model in MATLAB
Learn how to training deep learning models in MATLAB^®.
定义自定义训练循环、损失函数和网络
了解如何使用自动微分来定义和自定义深度学习训练循环、损失函数和网络。
Train Network Using Custom Training Loop
This example shows how to train a network that classifies handwritten digits with a custom learning rate schedule.
Specify Training Options in Custom Training Loop
Learn how to specify common training options in a custom training loop.
Define Model Loss Function for Custom Training Loop
Learn how to define a model loss function for a custom training loop.
Update Batch Normalization Statistics in Custom Training Loop
This example shows how to update the network state in a custom training loop.
Make Predictions Using dlnetwork Object
This example shows how to make predictions using a dlnetwork object by splitting data into mini-batches.
Monitor Custom Training Loop Progress
Track and plot custom training loop progress.
Train Network with Multiple Outputs
This example shows how to train a deep learning network with multiple outputs that predict both labels and angles of rotations of handwritten digits.
Classify Videos Using Deep Learning with Custom Training Loop
This example shows how to create a network for video classification by combining a pretrained image classification model and a sequence classification network.
Train Image Classification Network Robust to Adversarial Examples
This example shows how to train a neural network that is robust to adversarial examples using fast gradient sign method (FGSM) adversarial training.
Train Neural ODE Network
This example shows how to train an augmented neural ordinary differential equation (ODE) network.
Train Robust Deep Learning Network with Jacobian Regularization
This example shows how to train a neural network that is robust to adversarial examples using a Jacobian regularization scheme [1].
Solve Ordinary Differential Equation Using Neural Network
This example shows how to solve an ordinary differential equation (ODE) using a neural network.
Assemble Multiple-Output Network for Prediction
This example shows how to assemble a multiple output network for prediction.
Run Custom Training Loops on a GPU and in Parallel
Speed up custom training loops by running on a GPU, in parallel using multiple GPUs, or on a cluster.

模型函数

Train Network Using Model Function
This example shows how to create and train a deep learning network by using functions rather than a layer graph or a dlnetwork.
Update Batch Normalization Statistics Using Model Function
This example shows how to update the network state in a network defined as a function.
Make Predictions Using Model Function
This example shows how to make predictions using a model function by splitting data into mini-batches.
Initialize Learnable Parameters for Model Function
Learn how to initialize learnable parameters for custom training loops using a model function.
Train Latent ODE Network with Irregularly Sampled Time-Series Data
This example shows how to train a latent ordinary differential equation (ODE) autoencoder with time-series data that is sampled at irregular time intervals.
Multivariate Time Series Anomaly Detection Using Graph Neural Network
This example shows how to detect anomalies in multivariate time series data using a graph neural network (GNN).

自动微分

List of Functions with dlarray Support
View the list of functions that support dlarray objects.
Automatic Differentiation Background
Learn how automatic differentiation works.
Use Automatic Differentiation In Deep Learning Toolbox
How to use automatic differentiation in deep learning.

深度学习函数加速

Deep Learning Function Acceleration for Custom Training Loops
Accelerate model functions and model loss functions for custom training loops by caching and reusing traces.
Accelerate Custom Training Loop Functions
This example shows how to accelerate deep learning custom training loop and prediction functions.
Check Accelerated Deep Learning Function Outputs
This example shows how to check that the outputs of accelerated functions match the outputs of the underlying function.
Evaluate Performance of Accelerated Deep Learning Function
This example shows how to evaluate the performance gains of using an accelerated function.

精选示例

训练生成对抗网络 (GAN)

此示例说明如何训练生成对抗网络来生成图像。

深度学习自定义训练循环

函数

自定义网络

自定义训练循环

自动微分

深度学习运算

深度学习损失运算

深度学习函数加速

主题

自定义训练循环

模型函数

自动微分

深度学习函数加速

精选示例

训练生成对抗网络 (GAN)

Train Conditional Generative Adversarial Network (CGAN)

Train Wasserstein GAN with Gradient Penalty (WGAN-GP)

Train Fast Style Transfer Network

Train Network Using Custom Training Loop

Train Network Using Model Function

Train a Siamese Network for Dimensionality Reduction

Train a Siamese Network to Compare Images

Image Captioning Using Attention

Multilabel Text Classification Using Deep Learning

Sequence-to-Sequence Translation Using Attention

Train Network in Parallel with Custom Training Loop

训练变分自编码器 (VAE) 以生成图像

Train Network Using Cyclical Learning Rate for Snapshot Ensembling

Solve Partial Differential Equations Using Deep Learning

Solve Partial Differential Equation with L-BFGS Method and Deep Learning

Train Neural ODE Network

Dynamical System Modeling Using Neural ODE

Node Classification Using Graph Convolutional Network

Multilabel Graph Classification Using Graph Attention Networks

Train Network Using Federated Learning

WeChat