How do I train only one layer of a neural network?

5 次查看（过去 30 天）
Wout Haksteen
Wout Haksteen 2022-9-16
回答： Shreyansh Mehra 2022-9-20
I have a pre trained neural network, and I want to apply a temperature scaling calibration to it.
This means that I want to add one layer in front of the softmax layer, and train only the weights of this layer on the validation set.
However, I cannot seem to edit the network such that only this layer 'trains'. I have tried setting all the learning rates to zero, but even then some things change. Like; Batch Normalization layers and the input layers still update.
I even tried setting the overall learnrate to some value very small (1e-12) and the learn rate of my layer to something very large (1e8) but this does not work either. Other parts of the model still change.
This should be a relatively easy task, but I cannot seem to find a way to solve it. Any tips/ideas?

请先登录，再回答此问题。

回答（1 个）

Shreyansh Mehra
Shreyansh Mehra 2022-9-20
Hello,
The MATLAB Answers link given below explains that to freeze some weights of a neural network, learning parameters 'BiasLearnRateFactor' and/or 'WeightLearnRateFactor' can be set to zero, and might be of help. Please have a look.
https://www.mathworks.com/matlabcentral/answers/548100-how-can-i-freeze-specific-weights-of-neural-network-model

类别

在 Help Center 和 File Exchange 中查找有关 Image Data Workflows 的更多信息

标签

产品


版本

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by