There is not really a simple answer to this question. The only thing I have found is by going into each of the layers and adjusting the learning rate individually.
net = feedforwardnet([5 5 5]);
net.inputWeights{1}.learnParam.lr = .0001;
for ii = 1:3
net.layerWeights{1+ii,ii}.learnParam.lr = .0001;
end
for jj = 1:4
net.biases{jj}.learnParam.lr = .0001;
end
I'm not entirely sure if this is correct as it doesn't seem to have a large impact on my training time.
Other than that, there is no documentation that I can find on how to change the learning rate.
