How can we save neural network with best validation loss

Question

Arjun Desai 2018-7-9

2
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/409357-how-can-we-save-neural-network-with-best-validation-loss

评论： Nethtrick 2020-9-22

Currently I am using the trainNetwork command to train my network model. I want to save the model with the best running validation loss. For example, let us say at epoch 10, my validation loss is 0.2 and that is the lowest validation loss up to that point, then I would save that network model. Then, we reach epoch 11, where the validation loss reaches 0.1, we would also save this model (i.e. running best validation loss model).

My network contains batchNormalization layers, and as a result, I cannot use the models saved at checkpoints as the batchNormalization layers are not initialized.

Is there a work around for this? I know that tensorflow/Keras supports saving models with the best validation loss that do contain batchNormalization layers.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Pablo Rivas 2019-6-2

2
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/409357-how-can-we-save-neural-network-with-best-validation-loss#answer_377524

I don;t think this is possible yet.

** feature request **

It seems like, for now, you will have to save checkpoints of your network in every epoch, and at the end, in the training summary, you can see which epoch gave you the best validation accuracy/error and go back and find on what file corresponds to the chekpoint for that epoch.

However, this can be space-consuming and not ideal at all. It would be realy nice to have this feature. Right?

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Nethtrick 2020-9-22

Unfortunately the checkpoint approach does not work with the batch normalization layers. I am running into the same issue. It's an oversight not to have this built in because "training" itself is defined as minimizing the loss function.

I posted this question also before I found your post:

https://www.mathworks.com/matlabcentral/answers/598039-can-i-tell-trainnetwork-to-return-the-net-with-the-lowest-validation-or-training-loss-as-opposed

请先登录，再进行评论。