自定义训练循环

自定义深度学习训练循环和损失函数

如果 trainingOptions 函数不提供任务所需的训练选项，或者自定义输出层不支持所需的损失函数，则您可以定义自定义训练循环。对于无法指定为由层组成的网络的模型，可以将模型定义为函数。要了解详细信息，请参阅定义自定义训练循环、损失函数和网络。

函数

全部展开

网络构建

`dlnetwork`	Deep learning neural network
`imagePretrainedNetwork`	Pretrained neural network for images (自 R2024a 起)
`resnetNetwork`	2-D residual neural network (自 R2024a 起)
`resnet3dNetwork`	3-D residual neural network (自 R2024a 起)
`addLayers`	向神经网络添加层
`removeLayers`	Remove layers from neural network
`replaceLayer`	Replace layer in neural network
`connectLayers`	在神经网络中连接各层
`disconnectLayers`	Disconnect layers in neural network
`addInputLayer`	Add input layer to network (自 R2022b 起)
`initialize`	Initialize learnable and state parameters of a `dlnetwork` (自 R2021a 起)
`networkDataLayout`	Deep learning network data layout for learnable parameter initialization (自 R2022b 起)
`setL2Factor`	Set L₂ regularization factor of layer learnable parameter
`getL2Factor`	Get L₂ regularization factor of layer learnable parameter
`setLearnRateFactor`	Set learn rate factor of layer learnable parameter
`getLearnRateFactor`	Get learn rate factor of layer learnable parameter

网络信息

`plot`	绘制神经网络架构
`summary`	打印网络摘要 (自 R2022b 起)
`analyzeNetwork`	Analyze deep learning network architecture
`checkLayer`	Check validity of custom or function layer
`isequal`	Check equality of neural networks (自 R2021a 起)
`isequaln`	Check equality of neural networks ignoring `NaN` values (自 R2021a 起)

自定义训练循环

`forward`	Compute deep learning network output for training
`predict`	Compute deep learning network output for inference
`adamupdate`	Update parameters using adaptive moment estimation (Adam)
`rmspropupdate`	Update parameters using root mean squared propagation (RMSProp)
`sgdmupdate`	Update parameters using stochastic gradient descent with momentum (SGDM)
`lbfgsupdate`	Update parameters using limited-memory BFGS (L-BFGS) (自 R2023a 起)
`lbfgsState`	State of limited-memory BFGS (L-BFGS) solver (自 R2023a 起)
`dlupdate`	Update parameters using custom function
`trainingProgressMonitor`	Monitor and plot training progress for deep learning custom training loops (自 R2022b 起)
`updateInfo`	Update information values for custom training loops (自 R2022b 起)
`recordMetrics`	Record metric values for custom training loops (自 R2022b 起)
`groupSubPlot`	Group metrics in training plot (自 R2022b 起)
`deep.gpu.deterministicAlgorithms`	Set determinism of deep learning operations on the GPU to get reproducible results (自 R2024b 起)

数据处理

`padsequences`	Pad or truncate sequence data to same length (自 R2021a 起)
`minibatchqueue`	Create mini-batches for deep learning (自 R2020b 起)
`onehotencode`	Encode data labels into one-hot vectors (自 R2020b 起)
`onehotdecode`	Decode probability vectors into class labels (自 R2020b 起)
`next`	Obtain next mini-batch of data from minibatchqueue (自 R2020b 起)
`reset`	Reset minibatchqueue to start of data (自 R2020b 起)
`shuffle`	Shuffle data in minibatchqueue (自 R2020b 起)
`hasdata`	Determine if minibatchqueue can return mini-batch (自 R2020b 起)
`partition`	Partition minibatchqueue (自 R2020b 起)

自动微分

`dlarray`	用于自定义的深度学习数组
`dlgradient`	Compute gradients for custom training loops using automatic differentiation
`dljacobian`	Jacobian matrix deep learning operation (自 R2024b 起)
`dldivergence`	Divergence of deep learning data (自 R2024b 起)
`dllaplacian`	Laplacian of deep learning data (自 R2024b 起)
`dlfeval`	Evaluate deep learning model for custom training loops
`dims`	`dlarray` 对象的数据格式
`finddim`	Find dimensions with specified label
`stripdims`	Remove `dlarray` data format
`extractdata`	从 `dlarray` 中提取数据
`isdlarray`	Check if object is `dlarray` (自 R2020b 起)

损失运算

`crossentropy`	Cross-entropy loss for classification tasks
`indexcrossentropy`	Index cross-entropy loss for classification tasks (自 R2024b 起)
`l1loss`	L₁ loss for regression tasks (自 R2021b 起)
`l2loss`	L₂ loss for regression tasks (自 R2021b 起)
`huber`	Huber loss for regression tasks (自 R2021a 起)
`ctc`	Connectionist temporal classification (CTC) loss for unaligned sequence classification (自 R2021a 起)
`mse`	Half mean squared error

函数加速

`dlaccelerate`	Accelerate deep learning function for custom training loops (自 R2021a 起)
`AcceleratedFunction`	Accelerated deep learning function (自 R2021a 起)
`clearCache`	Clear accelerated deep learning function trace cache (自 R2021a 起)

主题

自定义训练循环

Train Deep Learning Model in MATLAB
Learn how to training deep learning models in MATLAB^®.
定义自定义训练循环、损失函数和网络
了解如何定义和自定义深度学习训练循环、损失函数和模型。
Train Network Using Custom Training Loop
This example shows how to train a network that classifies handwritten digits with a custom learning rate schedule.
Train Sequence Classification Network Using Custom Training Loop
This example shows how to train a network that classifies sequences with a custom learning rate schedule.
Specify Training Options in Custom Training Loop
Learn how to specify common training options in a custom training loop.
Define Model Loss Function for Custom Training Loop
Learn how to define a model loss function for a custom training loop.
Update Batch Normalization Statistics in Custom Training Loop
This example shows how to update the network state in a custom training loop.
Make Predictions Using dlnetwork Object
This example shows how to make predictions using a dlnetwork object by looping over mini-batches.
Monitor Custom Training Loop Progress
Track and plot custom training loop progress.
Multiple-Input and Multiple-Output Networks
Learn how to define and train deep learning networks with multiple inputs or multiple outputs.
训练具有多个输出的网络
此示例说明如何训练具有多个输出的深度学习网络，来预测手写数字的标签和旋转角度。
Classify Videos Using Deep Learning with Custom Training Loop
This example shows how to create a network for video classification by combining a pretrained image classification model and a sequence classification network.
Train Image Classification Network Robust to Adversarial Examples
This example shows how to train a neural network that is robust to adversarial examples using fast gradient sign method (FGSM) adversarial training.
Train Robust Deep Learning Network with Jacobian Regularization
Train a neural network that is robust to adversarial examples using a Jacobian regularization scheme.
Solve Ordinary Differential Equation Using Neural Network
This example shows how to solve an ordinary differential equation (ODE) using a neural network.
Train Network in Parallel with Custom Training Loop
This example shows how to set up a custom training loop to train a network in parallel.
Run Custom Training Loops on a GPU and in Parallel
Speed up custom training loops by running on a GPU, in parallel using multiple GPUs, or on a cluster.
Speed Up Deep Neural Network Training
Learn how to accelerate deep neural network training.

自动微分

Deep Learning Data Formats
Learn about deep learning data formats.
List of Functions with dlarray Support
View the list of functions that support dlarray objects.
Automatic Differentiation Background
Learn how automatic differentiation works.
Use Automatic Differentiation In Deep Learning Toolbox
How to use automatic differentiation in deep learning.

深度学习函数加速

Deep Learning Function Acceleration for Custom Training Loops
Accelerate model functions and model loss functions for custom training loops by caching and reusing traces.
Accelerate Custom Training Loop Functions
This example shows how to accelerate deep learning custom training loop and prediction functions.
Check Accelerated Deep Learning Function Outputs
This example shows how to check that the outputs of accelerated functions match the outputs of the underlying function.
Evaluate Performance of Accelerated Deep Learning Function
This example shows how to evaluate the performance gains of using an accelerated function.

精选示例

Train Wasserstein GAN with Gradient Penalty (WGAN-GP)

Train a Wasserstein generative adversarial network with a gradient penalty (WGAN-GP) to generate images.

打开实时脚本

Generate Images Using Diffusion

Generate new images using a diffusion model.

自 R2023b 起
打开实时脚本

Train Fast Style Transfer Network

Train a network to transfer the style of an image to a second image. It is based on the architecture defined in [1].

打开实时脚本

Train a Twin Network for Dimensionality Reduction

Train a twin neural network with shared weights to compare handwritten digits using dimensionality reduction.

打开实时脚本

Train a Twin Neural Network to Compare Images

Train a twin neural network with shared weights to identify similar images of handwritten characters.

打开实时脚本

Image Captioning Using Attention

Train a deep learning model for image captioning using attention.

打开实时脚本

Sequence-to-Sequence Translation Using Attention

Convert decimal strings to Roman numerals using a recurrent sequence-to-sequence encoder-decoder model with attention.

打开实时脚本

Train Network in Parallel with Custom Training Loop

Set up a custom training loop to train a network in parallel.

打开实时脚本

Train Network Using Cyclical Learning Rate for Snapshot Ensembling

Train a network to classify images of objects using a cyclical learning rate schedule and snapshot ensembling for better test accuracy. In the example, you learn how to use a cosine function for the learning rate schedule, take snapshots of the network during training to create a model ensemble, and add L2-norm regularization (weight decay) to the training loss.

打开实时脚本

Solve PDE Using Physics-Informed Neural Network

Train a physics-informed neural network (PINN) to predict the solutions of the Burger's equation.

打开实时脚本

Dynamical System Modeling Using Neural ODE

Train a neural network with neural ordinary differential equations (ODEs) to learn the dynamics of a physical system.

打开实时脚本

Reduced Order Modeling Using Continuous-Time Echo State Network

Train a continuous-time echo state network (CTESN) model to solve Robertson's equation.

打开实时脚本

Node Classification Using Graph Convolutional Network

Classify nodes in a graph using a graph convolutional network (GCN).

打开实时脚本

Multilabel Graph Classification Using Graph Attention Networks

Classify graphs that have multiple independent labels using graph attention networks (GATs).

打开实时脚本

Train Network Using Federated Learning

Train a network using federated learning. Federated learning is a technique that enables you to train a network in a distributed, decentralized way [1].

打开实时脚本

自定义训练循环

函数

网络构建

网络信息

自定义训练循环

数据处理

自动微分

损失运算

函数加速

主题

自定义训练循环

自动微分

深度学习函数加速

相关信息

精选示例

训练生成对抗网络 (GAN)

训练条件生成对抗网络 (CGAN)

Train Wasserstein GAN with Gradient Penalty (WGAN-GP)

Generate Images Using Diffusion

Train Fast Style Transfer Network

Train a Twin Network for Dimensionality Reduction

Train a Twin Neural Network to Compare Images

Image Captioning Using Attention

Sequence-to-Sequence Translation Using Attention

Train Network in Parallel with Custom Training Loop

训练变分自编码器 (VAE) 以生成图像

Train Network Using Cyclical Learning Rate for Snapshot Ensembling

Solve PDE Using Physics-Informed Neural Network

Dynamical System Modeling Using Neural ODE

Reduced Order Modeling Using Continuous-Time Echo State Network

Node Classification Using Graph Convolutional Network

Multilabel Graph Classification Using Graph Attention Networks

Train Network Using Federated Learning

自定义训练循环

函数

网络构建

网络信息

自定义训练循环

数据处理

自动微分

损失运算

函数加速

主题

自定义训练循环

自动微分

深度学习函数加速

相关信息

精选示例

WeChat