Correct weight Initialization in CNN
6 次查看(过去 30 天)
显示 更早的评论
When a very deep DAG network is built from scratch, the initialization of the weights made by matlab is not very good since it presents a vanishing gradient problem which causes the CNN not to learn.
What is the function with which Matlab does the initiation of CNN weights?
Why do you implement initialization functions in Matlab such as XAVIER or RELU AWARE SCALALED?
Thank you for your answers.
2 个评论
Greg Heath
2018-7-31
I do not understand
"Why do you implement initialization functions in Matlab such as XAVIER or RELU AWARE SCALALED?"
Please explain.
Greg
采纳的回答
Maria Duarte Rosa
2019-7-5
编辑:Maria Duarte Rosa
2019-7-5
In R2019a, the following weight initializers are available (including a custom initializer via a function handle):
'glorot' (default) | 'he' | 'orthogonal' | 'narrow-normal' | 'zeros' | 'ones' | function handle
Glorot is also know as Xavier initializer.
Here is a page comparing 3 initializers when training LSTMs:
I hope this helps,
Maria
0 个评论
更多回答(2 个)
Andres Ramirez
2018-7-31
1 个评论
Greg Heath
2018-8-1
编辑:Greg Heath
2018-8-1
Do you have a reference for
RELA AWARE SCALALED
I have no idea what this is.
Thanks
Greg
fareed jamaluddin
2018-8-4
I think you can take a look at this example https://www.mathworks.com/help/images/single-image-super-resolution-using-deep-learning.html
I am also looking for a way on weight initialization options, you can see in the example it create the initialization with He method for every conv layer.
0 个评论
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Image Data Workflows 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!