L2 regularization in sparse stacked autoencoders not clear to me

Question

Lukas Vareka 2018-3-21

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/389685-l2-regularization-in-sparse-stacked-autoencoders-not-clear-to-me

回答： Sara Sharifzadeh 2020-6-6

Dear Matlab users,

on https://www.mathworks.com/help/nnet/ref/trainautoencoder.html , there is a bit of theory behind L2 regularization used in stacked autoencoders. However, the definition of L2 regularization is not clear to me. First, why is the sum running through all hidden layers l = 1..L but not through all neurons in each hidden layer? Second, I do not understand "k is the number of variables in the training data". Does it mean that k corresponds to the dimensionality of feature vectors?

Thanks for any clarification.

Regards, Lukas Vareka

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Sara Sharifzadeh 2020-6-6

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/389685-l2-regularization-in-sparse-stacked-autoencoders-not-clear-to-me#answer_447194

As this L2 regularisation aims to constraints all the weights, The first Sigma should sum all hidden layers+final layer and the two inner Sigma must count on the number of neurons in previous layer e.g. (l-1) and the current hidden layer (l) that in total will include all weights of the network to be constraint.