Issue with batch normalization layer of saved CNN

1 次查看（过去 30 天）
Kjetil Kveim
Kjetil Kveim 2019-3-21
回答： Sam Leeney 2022-12-15
When loading a previously trained CNN, I got an issue with the batch normalization layers. When looking into the loaded CNN model the trainable mean and variance are empty.
Name: 'batchnorm_1'
TrainedMean: []
TrainedVariance: []
So the checkpoint doesn't seem to save these parameters. Are there any workarounds for this issue? I am using Matlab R2018b.
  1 个评论
Wes Baldwin
Wes Baldwin 2020-7-29
Doesn't this mean using checkpoints on networks with a batchnorm layer is useless??? Kinda a big deal for long training!!! You could potentially lose days or weeks of training with no option but to start from the beginning.

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Vivek Singh
Vivek Singh 2019-3-22
We were able to reproduce the issue. We will inform you once the issue is fixed.
Since TrainedMean and TrainedVariance are calculated after the training is finished, therefore as a workaround you can use the below mentioned codes to explicitly save and load the Model.
%To save model with name "demoModel", assuming your network is in "net"
save('demoModel','net')
%To load model to variable net1
net1=load('demoModel.mat','net');
net1.net.Layers(n).TrainedMean %where n is the batch normalization layer
  4 个评论
显示 2更早的评论
ramin nateghi
ramin nateghi 2020-11-5
编辑：ramin nateghi 2020-11-5
Hello,
I also faced this problem. When a model is saved by "save" function, it is ok and the model contains all information (TrainedMean and TrainedVariance) of the trained batch normalization layers. But, when the model is saved by the"checkpoint" during training, both of the TrainedMean and TrainedVariance params became empty. This is a bug for "checkpoint".
Yi Wei
Yi Wei 2020-11-30
Does it mean the saving process will cost too much time if a network contains normalization layers and the training data volume is large(e.g.8T)?

请先登录，再进行评论。

更多回答（1 个）

Sam Leeney
Sam Leeney 2022-12-15
For anyone else stuck, there is a fix here; https://uk.mathworks.com/matlabcentral/answers/423588-how-to-classify-with-dag-network-from-checkpoint

类别

在 Help Center 和 File Exchange 中查找有关 Deep Learning Toolbox 的更多信息

标签

产品


版本

R2018b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by