Early Stopping for Deep Networks
9 次查看(过去 30 天)
显示 更早的评论
Hi everyone,
just a quick question.
How can I stop the training of a deep network (LSTM for instance) in order to have weights and biases set accordingly with the minimum of the validation loss?
In other words what's the reason of having a validation set if the final network is NOT the one that minimize the validation loss because it's overtrained in any case?
Validation Patience parameter is not useful in this sense because it stops the training when it's too late and setting it too small could result in being stuck in a local minimum.
The only way I found is repeating the training with max epochs set where the minimum of validation loss in the first training is reached but it's a crazy solution...
Any idea?
Thanks
0 个评论
回答(1 个)
Greg Heath
2019-1-16
编辑:Greg Heath
2019-1-16
It is not clear to me that, based on a random 15% of the data, this is a better choice. It would be interesting to make a formal comparison based on multiple runs with different random number seeds using multiple data sets.
I believe that a more important point is to try to minimize the number of hidden nodes subject to an upper bound on the training set error rate.
Hope this helps
Thank you for formally accepting tmy answer
Greg
2 个评论
Greg Heath
2019-1-19
编辑:Greg Heath
2019-1-19
No. That is not what I meant.
HOWEVER
Any decent method will outperform others depending on the data set.
My shallow net double loop procedure (MANY examples in NEWSGROUP and ANSWERS) has been successfull for decades.
- Single hidden layer
- Outer loop over number of hidden nodes H = 0:dH:Hmax
- Inner loop over random intial weights
I have not tried it on deep nets but am interested if anyone else has.
Greg.
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Image Data Workflows 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!