Standard learning rate and momentum term trainlm NAR

14 次查看(过去 30 天)
Hello,
I'm looking for the standard learning rate and momentum term that is used by the Levenberg-Marquardt training algorithm. I found the standard mu factor, but as far as I can see this is not the same as the learning / momentum rate. Can someone help me with this?
Thanks in advance!

回答(2 个)

Nick Durkee
Nick Durkee 2018-6-18
There is not really a simple answer to this question. The only thing I have found is by going into each of the layers and adjusting the learning rate individually.
net = feedforwardnet([5 5 5]);
net.inputWeights{1}.learnParam.lr = .0001;
for ii = 1:3
net.layerWeights{1+ii,ii}.learnParam.lr = .0001;
end
for jj = 1:4
net.biases{jj}.learnParam.lr = .0001;
end
I'm not entirely sure if this is correct as it doesn't seem to have a large impact on my training time.
Other than that, there is no documentation that I can find on how to change the learning rate.
  1 个评论
Lucas Ferreira-Correia
I don't think it has an impact at all.
I'm using the Levenberg-Marquardt (trainlm) solver and after trying learning rates 0, 0.0001, 1 I was not able to observe a change in my MSE.
My theory is that this solver simply doesn't support any changes in learning rate.
Perhaps it only works on Gradient descent with momentum and adaptive learning rate backpropagation (traingdx)?
Hope this helps!

请先登录,再进行评论。


Greg Heath
Greg Heath 2017-8-4
There are 3 commands to try before posting
help trainlm
doc trainlm
type trainlm
Hope this helps.
Thank you for accepting my answer
Greg
  1 个评论
Nick Durkee
Nick Durkee 2018-6-18
There are 3 commands to try before giving a snarky answer
help trainlm
doc trainlm
type trainlm
Hope this helps.
The question stands, there is no simple way to change the learning rate.

请先登录,再进行评论。

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by