How can I learn the final learning rate of a deep neural network?

15 次查看(过去 30 天)
After completing the training procedure of a deep neural network over a spesific dataset, how can we learn the last, final, optimum learning rate? We specify a certain learning rate at the beginning, it is OK, but it reaches to an optimum value? What is it? How can I learn it?
Could you please refer a sample test case below:
clc;clear;
[XTrain,YTrain] = japaneseVowelsTrainData;
figure
plot(XTrain{1}')
title("Training Observation 1")
numFeatures = size(XTrain{1},1);
legend("Feature " + string(1:numFeatures),'Location','northeastoutside')
inputSize = 12;
numHiddenUnits = 100;
numClasses = 9;
layers = [ sequenceInputLayer(inputSize)
lstmLayer(numHiddenUnits,'OutputMode','last')
fullyConnectedLayer(numClasses)
softmaxLayer
classificationLayer]
maxEpochs = 70;
miniBatchSize = 27;
options = trainingOptions('sgdm', ...
'ExecutionEnvironment','cpu', ...
'MaxEpochs',maxEpochs, ...
'MiniBatchSize',miniBatchSize, ...
'GradientThreshold',1, ...
'Verbose',false, ...
'Plots','training-progress');
net = trainNetwork(XTrain,YTrain,layers,options);
[XTest,YTest] = japaneseVowelsTestData;
YPred = classify(net,XTest,'MiniBatchSize',miniBatchSize);
acc = sum(YPred == YTest)./numel(YTest)

回答(1 个)

Meet
Meet 2024-8-23,8:50
Hi Faruk,
From my understanding of your question, you want to know the final learning rate at the end of training the deep neural network.
There are two ways to do so, let me provide both the ways,
  1. Training Progress Window: You can find the final learning rate which was used by the network. Since, the learning rate schedule is constant it would remain 0.01, even at the end of the training.
  2. Using ‘info’ object: ‘trainNetwork’ method return two objects one is ‘net’ and other is ‘info’ which is a struct and contains an array of learning rates named ‘BaseLearnRate’, you can access the last element of this array to know the final learning rate the model used­­­­. In your case you could replace the line
net = trainNetwork(XTrain,YTrain,layers,options);
to
[net,info] = trainNetwork(XTrain,YTrain,layers,options);
finalLearningRate = info.BaseLearnRate(end);
I hope these suggestions prove helpful for your task!
You can refer to this article for more detailed information:

类别

Help CenterFile Exchange 中查找有关 Deep Learning Toolbox 的更多信息

产品


版本

R2020b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by