MATLAB Answers

Standard learning rate and momentum term trainlm NAR

21 views (last 30 days)
Timen
Timen on 10 Feb 2015
Hello,
I'm looking for the standard learning rate and momentum term that is used by the Levenberg-Marquardt training algorithm. I found the standard mu factor, but as far as I can see this is not the same as the learning / momentum rate. Can someone help me with this?
Thanks in advance!

  0 Comments

Sign in to comment.

Answers (2)

Greg Heath
Greg Heath on 4 Aug 2017
There are 3 commands to try before posting
help trainlm
doc trainlm
type trainlm
Hope this helps.
Thank you for accepting my answer
Greg

  1 Comment

Nick Durkee
Nick Durkee on 18 Jun 2018
There are 3 commands to try before giving a snarky answer
help trainlm
doc trainlm
type trainlm
Hope this helps.
The question stands, there is no simple way to change the learning rate.

Sign in to comment.


Nick Durkee
Nick Durkee on 18 Jun 2018
There is not really a simple answer to this question. The only thing I have found is by going into each of the layers and adjusting the learning rate individually.
net = feedforwardnet([5 5 5]);
net.inputWeights{1}.learnParam.lr = .0001;
for ii = 1:3
net.layerWeights{1+ii,ii}.learnParam.lr = .0001;
end
for jj = 1:4
net.biases{jj}.learnParam.lr = .0001;
end
I'm not entirely sure if this is correct as it doesn't seem to have a large impact on my training time.
Other than that, there is no documentation that I can find on how to change the learning rate.

  1 Comment

Lucas Ferreira-Correia
Lucas Ferreira-Correia on 17 Jul 2019
I don't think it has an impact at all.
I'm using the Levenberg-Marquardt (trainlm) solver and after trying learning rates 0, 0.0001, 1 I was not able to observe a change in my MSE.
My theory is that this solver simply doesn't support any changes in learning rate.
Perhaps it only works on Gradient descent with momentum and adaptive learning rate backpropagation (traingdx)?
Hope this helps!

Sign in to comment.


Translated by