change learning rate trainlm

Question

Pendar Arzandeh 2022-5-25

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1727135-change-learning-rate-trainlm

回答： Krishna 2024-8-25

How to change learning rate in trainlm? matlab allows changing mu and Gradient

tnx

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Phuong Huynh 2022-5-26

I also wonder about this problem. I think trainlm function (optimizer) auto choose LR for us (http://matlab.izmiran.ru/help/toolbox/nnet/trainlm.html). Because other function allows us to change LR (for example :traingdx) when we change parameter net.trainParam.lr

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Krishna 2024-8-25

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1727135-change-learning-rate-trainlm#answer_1504719

Hello,

From what I understand you want to change the learning rate of ‘trainlm’ optimiser. If you go through the documentation of ‘trainlm’ function you can see that there is no way to set a learning rate of this optimiser, but you can control the learning dynamics via mu, the adaptive parameter that adjusts the learning process in Levenberg-Marquardt as it acts similarly to the learning rate. A smaller mu allows smaller steps during learning, which can lead to finer adjustments but slower convergence. A larger mu takes bigger steps, which may speed up convergence but risks overshooting.

Set the hyper-paramter ‘net.trainParam.mu_dec’ to specify how much to decrease ‘mu’ when the error decreases after an iteration. Also set parameter ‘net.trainParam.mu_inc’ how much to increase ‘mu’ when the error increase after an iteration.

Please go through the following documentation to learn more regarding the hyper-parameters of Levenberg-Marquardt algorithm and also the mathematical update function being used in the following algorithm,

https://in.mathworks.com/help/deeplearning/ref/trainlm.html

Also please go through the following documentation to learn more about how to ask question on MATLAB answer and get a fast response,

https://in.mathworks.com/matlabcentral/answers/6200-tutorial-how-to-ask-a-question-on-answers-and-get-a-fast-answer

Hope this helps.