Can someone explain to me the meaning of these numbers is this line of code?

Question

Nathan Reilly 2018-4-29

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/398045-can-someone-explain-to-me-the-meaning-of-these-numbers-is-this-line-of-code

编辑： Walter Roberson 2018-5-2

In relation to deep learning setting the weight and bias learn rate factors both to 20 means learn faster in the new layers not the updated layers. Does the 20 represent faster learning from layer 20 onward or something else?

Code:

fullyConnectedLayer(numClasses,'Name','fc','WeightLearnRateFactor',20,'BiasLearnRateFactor',20)

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Prajit T R 2018-5-2

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/398045-can-someone-explain-to-me-the-meaning-of-these-numbers-is-this-line-of-code#answer_318232

Hi Nathan

I think the documentation of the function 'fullyConnectedLayer' explains this accurately.

'WeightLearnRateFactor' is the factor by which the learning rate of the weights in the layer is greater than that of the global learning rate.

'BiasLearnRateFactor' is the factor by which the learning rate of the biases in the layer is greater than that of the global learning rate.

Let us take your code as an example to explain this: The 'WeightLearnRateFactor' in this layer you are creating is 20 times that of the global learning rate. That would mean that this layer hugely influences the model as the learning rate is much higher. Similar is the case for the 'BiasLearnRateFactor'.

As you suggested, this does represent faster learning, but has nothing to do with layer 20. The faster learning corresponds to the fully connected layer that you currently generated.

Hope this helps

Cheers