How to change learning rate and momentum in 'trainlm' neural network?

Question

Avanthi Saumyamala 2018-11-2

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/427646-how-to-change-learning-rate-and-momentum-in-trainlm-neural-network

评论： Katy 2023-7-19

I have a feed forward neural network trained with trainlm function.Now i want to change the learning rate and momentum.Is there a specific default range for learning rate or Do I need to find the optimum range for learning rate?

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

GW 2018-11-2

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/427646-how-to-change-learning-rate-and-momentum-in-trainlm-neural-network#answer_344840

编辑：GW 2018-11-2

在 MATLAB Online 中打开

use

options = trainingOptions('sgdm',...
    'MaxEpochs',60, ...
    'LearnRateDropFactor',0.1,...
    'LearnRateDropPeriod',1,...
    'LearnRateSchedule','piecewise',...
    'ValidationData',{valImages,valLabels},...
    'ValidationFrequency',50,...
    'Momentum',0.85,...
    'ValidationPatience',5,...
    'InitialLearnRate',0.1,...
    'MiniBatchSize',128,...
    'Verbose',true,...
    'executionenvironment','cpu',...
    'Shuffle','every-epoch',...
    'Plots','training-progress');
net = trainNetwork(...,...,...,options);

and choose the parameters accordingly.

For default values, see the docs of trainingOptions.

In terms of the learning rate and momentum, I typically start with a large one just to test the general behaviour and then I drop the learning rate to get more accurate results. Its difficult to give you some concrete advice because the optimal learning rate depends on your specific problem.