CNN overfits when trained too long on low dataset
4 次查看(过去 30 天)
显示 更早的评论
Hi!
As you can seen below I have an overfitting problem. I am facing this problem because I have a very small dataset: 3 classes of each 20 1D images. Therefore, I am using a very simple architecture so the model will be robust, and cannot be trained 'too well' to the training data. However, it seems if I train it for too long, the model will eventually still be specific to the training data, and not robust. However, at around 0.5 iterations the model works quite okay and robustly. There is literally only one convolutional layer containing 8 filters in my architecture.
At 0.5 iterations the model seems to be robust. A solution would be to stop the training here. However, this feels wrong, is there another way? Or is this the way to go...
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/1125670/image.png)
0 个评论
回答(1 个)
Antoni Woss
2022-9-16
As you mentioned, this looks like an overfitting problem. Terminating the training early is one way that you can avoid overfitting. There are also training options that you can specify to automate this.
For example, see the validation section on the following documentation page:
In particular, setting the option OutputNetwork as best-validation-loss will return the network at the iteration with the minimal validation loss. From your plot, this looks to be around 0.5 * 10^4 (although there is a spike around 1.5 * 10^4 that looks sporadically lower). You might also want to set the ValidationPatience. This specifies the number of times that the loss on the validation set can be larger than or equal to the previously smallest loss before network training stops. This should help terminate the training before the upward drift in the validation loss. You may also want to increasing the spacing between validation loss evaluation to remove the oscillations and help isolate the upward drift. This can be done by setting the ValidationFrequency value, which is the number of iterations between evaluations of validation metrics.
0 个评论
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Image Data Workflows 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!