Issue with batch normalization layer of saved CNN
显示 更早的评论
When loading a previously trained CNN, I got an issue with the batch normalization layers. When looking into the loaded CNN model the trainable mean and variance are empty.
Name: 'batchnorm_1'
TrainedMean: []
TrainedVariance: []
So the checkpoint doesn't seem to save these parameters. Are there any workarounds for this issue? I am using Matlab R2018b.
1 个评论
Wes Baldwin
2020-7-29
Doesn't this mean using checkpoints on networks with a batchnorm layer is useless??? Kinda a big deal for long training!!! You could potentially lose days or weeks of training with no option but to start from the beginning.
采纳的回答
Vivek Singh
2019-3-22
0 个投票
We were able to reproduce the issue. We will inform you once the issue is fixed.
Since TrainedMean and TrainedVariance are calculated after the training is finished, therefore as a workaround you can use the below mentioned codes to explicitly save and load the Model.
%To save model with name "demoModel", assuming your network is in "net"
save('demoModel','net')
%To load model to variable net1
net1=load('demoModel.mat','net');
net1.net.Layers(n).TrainedMean %where n is the batch normalization layer
4 个评论
Kjetil Kveim
2019-3-25
ytzhak goussha
2020-9-24
Hello,
Was this issue fixed? I have encounterd the same problem when I stoped training and tried to predict with the saved checkpoint.
ramin nateghi
2020-11-5
编辑：ramin nateghi
2020-11-5
Hello,
I also faced this problem. When a model is saved by "save" function, it is ok and the model contains all information (TrainedMean and TrainedVariance) of the trained batch normalization layers. But, when the model is saved by the"checkpoint" during training, both of the TrainedMean and TrainedVariance params became empty. This is a bug for "checkpoint".
Yi Wei
2020-11-30
Does it mean the saving process will cost too much time if a network contains normalization layers and the training data volume is large(e.g.8T)?
更多回答（1 个）
Sam Leeney
2022-12-15
0 个投票
For anyone else stuck, there is a fix here; https://uk.mathworks.com/matlabcentral/answers/423588-how-to-classify-with-dag-network-from-checkpoint
类别
在 帮助中心 和 File Exchange 中查找有关 Deep Learning Toolbox 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
