MATLAB Answers

Timen
0

Standard learning rate and momentum term trainlm NAR

Asked by Timen
on 10 Feb 2015
Latest activity Commented on by Lucas Ferreira-Correia on 17 Jul 2019
Hello,
I'm looking for the standard learning rate and momentum term that is used by the Levenberg-Marquardt training algorithm. I found the standard mu factor, but as far as I can see this is not the same as the learning / momentum rate. Can someone help me with this?
Thanks in advance!

  0 Comments

Sign in to comment.

2 Answers

Answer by Greg Heath
on 4 Aug 2017

There are 3 commands to try before posting
help trainlm
doc trainlm
type trainlm
Hope this helps.
Thank you for accepting my answer
Greg

  1 Comment

There are 3 commands to try before giving a snarky answer
help trainlm
doc trainlm
type trainlm
Hope this helps.
The question stands, there is no simple way to change the learning rate.

Sign in to comment.


Answer by Nick Durkee on 18 Jun 2018

There is not really a simple answer to this question. The only thing I have found is by going into each of the layers and adjusting the learning rate individually.
net = feedforwardnet([5 5 5]);
net.inputWeights{1}.learnParam.lr = .0001;
for ii = 1:3
net.layerWeights{1+ii,ii}.learnParam.lr = .0001;
end
for jj = 1:4
net.biases{jj}.learnParam.lr = .0001;
end
I'm not entirely sure if this is correct as it doesn't seem to have a large impact on my training time.
Other than that, there is no documentation that I can find on how to change the learning rate.

  1 Comment

I don't think it has an impact at all.
I'm using the Levenberg-Marquardt (trainlm) solver and after trying learning rates 0, 0.0001, 1 I was not able to observe a change in my MSE.
My theory is that this solver simply doesn't support any changes in learning rate.
Perhaps it only works on Gradient descent with momentum and adaptive learning rate backpropagation (traingdx)?
Hope this helps!

Sign in to comment.