Standard learning rate and momentum term trainlm NAR
5 次查看(过去 30 天)
显示 更早的评论
Hello,
I'm looking for the standard learning rate and momentum term that is used by the Levenberg-Marquardt training algorithm. I found the standard mu factor, but as far as I can see this is not the same as the learning / momentum rate. Can someone help me with this?
Thanks in advance!
0 个评论
回答(2 个)
Nick Durkee
2018-6-18
There is not really a simple answer to this question. The only thing I have found is by going into each of the layers and adjusting the learning rate individually.
net = feedforwardnet([5 5 5]);
net.inputWeights{1}.learnParam.lr = .0001;
for ii = 1:3
net.layerWeights{1+ii,ii}.learnParam.lr = .0001;
end
for jj = 1:4
net.biases{jj}.learnParam.lr = .0001;
end
I'm not entirely sure if this is correct as it doesn't seem to have a large impact on my training time.
Other than that, there is no documentation that I can find on how to change the learning rate.
1 个评论
Lucas Ferreira-Correia
2019-7-17
I don't think it has an impact at all.
I'm using the Levenberg-Marquardt (trainlm) solver and after trying learning rates 0, 0.0001, 1 I was not able to observe a change in my MSE.
My theory is that this solver simply doesn't support any changes in learning rate.
Perhaps it only works on Gradient descent with momentum and adaptive learning rate backpropagation (traingdx)?
Hope this helps!
Greg Heath
2017-8-4
There are 3 commands to try before posting
help trainlm
doc trainlm
type trainlm
Hope this helps.
Thank you for accepting my answer
Greg
1 个评论
Nick Durkee
2018-6-18
There are 3 commands to try before giving a snarky answer
help trainlm
doc trainlm
type trainlm
Hope this helps.
The question stands, there is no simple way to change the learning rate.
另请参阅
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!