Inconsistent test-results with neural network

Question

Detlef Preis 2015-7-13

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/229657-inconsistent-test-results-with-neural-network

评论： Greg Heath 2015-7-13

Hello,

I have just used the GUI-tool of the neural network toolbox for fitting. This app automatically divides the input data in training-, validation- and testdatasets. After training, I get the performance of these 3 stages which are all fine (MSE~50). Especially the included test-setting shows good results. -> the network generalised Unfortunately these good results vanish when I test the neural network manuelly. When I load an unused testdata in the trained network, I get really bad performance values(~1000). -> the network does not generalise at all This confuses me, because this 2 test-options, 'included in tool' and 'manually', should bring almost similar performance values!

Can anyone please tell me, why there is such a huge difference in the results? The tool tells me, that the neural network is good, but when I use it, it sucks. Why?

I am grateful for every answer!

Kind regards, Detlef

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Walter Roberson 2015-7-13

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/229657-inconsistent-test-results-with-neural-network#answer_185903

How are you initializing the weights? By default NN initialize the weights randomly.

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Detlef Preis 2015-7-13

Default initialization, therefore random weights at the beginning. But the issue are not the weights. Both testing options that I described happen after the ANN training, therefore with the same weight values.

请先登录，再进行评论。

Answer 2

Greg Heath 2015-7-13

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/229657-inconsistent-test-results-with-neural-network#answer_185927

在 MATLAB Online 中打开

The only time that should happen is when the 2 sets do not appear to come from the same probability distribution.

You don't give enough information. Assuming the data are 1-Dimensional

For subset 1.training 2. validation 3. test1 4. test2:

 a. size(subset) =
 b. mean(subset) =
 c. var(subset)  = 
 d. (mean(test1)-mean(test2))/sqrt(var(test1) + var(test2))

Hope this helps

Greg

2 个评论
显示无隐藏无

Detlef Preis 2015-7-13

I am only talking from the testing, therefore after training. The included test delivers much better results than the manual test(manual test is the same as just using the trained network, but calculating the error).