What happend if the validation performance was greater than the test performance?

Question

Rita 2017-2-22

1
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/326435-what-happend-if-the-validation-performance-was-greater-than-the-test-performance

评论： Rita 2017-3-3

I have trained several networks with different hidden neurons. Now my question is what happens if I select a network with the validation performance higher than the test performance.I have read somewhere that in this situation the distribution of data between the training set and test set is not correct.Thanks for any suggestion.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Greg Heath 2017-2-28

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/326435-what-happend-if-the-validation-performance-was-greater-than-the-test-performance#answer_256713

There is no rule governing the order of the val and tst performances. That is why it is worthwhile to design a number of nets differing only by the random initial weights and/or if not a time series, the order of the input/target pairs.

Thank you for formally accepting my answer

Greg

3 个评论
显示 1更早的评论隐藏 1更早的评论

Greg Heath 2017-3-1

Sometimes it is dangerous to rely on the results of just a few designs. That is why, for challenging cases, I typically design at least 10 or 15 nets for each candidate value of H, the number of hidden nodes.

Then by tabulating the 3 NMSE values with the column for NMSEval monotonically decreasing, you can detect how well trends in NMSEval can indicate the acceptability of NMSEtst.

Of course there are other tabulations and/or plots that reveal design trends( e.g., NMSE vs H). The important thing is to design enough nets so that you are convinced your choice is reliable.

Rita 2017-3-3