Inconsistent training loss curve when training UNet with and without convergence criterion

Question

Memo Remo 2023-4-22

1
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1951273-inconsistent-training-loss-curve-when-training-unet-with-and-without-convergence-criterion

编辑： Matt J 2023-4-24

I want to train a semantic segmentation model using UNet layers. I trained the model one time without any convergence criterion and max epoch of 500 and the other time with a convergence criterion (validation patience of 10 and validation frequency of 25). The secnd training process converged at a lower epoch number compared to the first one. The results of the application of the second model is significantly less accurate than the first one on the test slices. I noticed that the training loss in the second training process is different from the first one. My question is that why defining a convergence criterion changes the training loss curve? I would be thankful if any one can help me understand why this is happening. The learning rate is constant and equal to 1e-05.

Training curves without convergence criterion:

Training curves with convergence criterion:

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Matt J 2023-4-22

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1951273-inconsistent-training-loss-curve-when-training-unet-with-and-without-convergence-criterion#answer_1221083

编辑：Matt J 2023-4-24

There should be a change in the training loss curve - it should be shorter. By loosening the stopping criteria, you cause fewer iterations to be run.

There will also be changes in the training loss curves due to the stochastic nature of the SGD algorithm and the parameter initialization.

3 个评论
显示 1更早的评论隐藏 1更早的评论

Matt J 2023-4-22

编辑：Matt J 2023-4-22

It's not clear to me how strong the differences are. The axes limits are not the same in the 2 plots.

Also, we have not been shown the 3rd plot (on the same axes as the first two) with the training curve you get after reverting back to default settings.

Memo Remo 2023-4-24