How can we recover the network state at iteration T

Question

Arjun Desai 2018-5-25

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/402634-how-can-we-recover-the-network-state-at-iteration-t

评论： Torsten K 2020-10-16

I am using a validation set to determine stop training when the validation loss stops decreasing. I have my validation patience as 3. Assuming that my networks stops training when it has surpassed this patience threshold, during the final 3 validation steps of my network would have been overfitting.

As a result, I want to recover the network state at the step that produced the minimum validation loss. Is there a way to do this?

2 个评论
显示无隐藏无

Roberto 2019-1-15

Hi Arjun. I have the same problem. Did you find a solution?

My current solution is keeping track of the best iterations and then re-train the network with the same rng seed and max epoch set in order to stop the training wher the validation loss reached the minimum in the previous training. It's an horrible solution but it's the only way I've found.

Torsten K 2020-10-16

Hi Roberto,

I also have the same problem. Did you find a solution yet? If so, I am very interested how you solved the problem!

Regards

Torsten

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Greg Heath 2018-5-26

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/402634-how-can-we-recover-the-network-state-at-iteration-t#answer_322010

In the distant past I'm pretty sure that I have checked, by using the error plot, that it is done automatically.

Thank you for formally accepting this answer

Greg.

2 个评论
显示无隐藏无

Arjun Desai 2018-5-31

Hi Greg, I checked with the Mathworks team, and it seems that this is not the case. In fact, the state pf the network that is accepted is the last state before the training stops.

Greg Heath 2019-1-16

编辑：Greg Heath 2019-1-16

I have not verified this but I have the feeling that, in the long run, the difference between stopping at minval and minval + 6 for 15% of the data is not significant w.r.t. performance on the entire dataset.

Greg

请先登录，再进行评论。