fitnet Why test results are unbiased?

3 次查看(过去 30 天)
I've read form Greg Heath:
Data = Design + Non-design
Design = Train(estimate weights) + Validation(Stop training when MSEval goes thru a minimum).
Non-design = Test(Obtain UNBIASED generalization estimate of performance on unseen non-design data).
I like to know why and how test results are calculated on unbiased net despite the fact that training is made on biased network and how to obtain the same results using :
output=net(input)
I must use :
net.biasConnect = [0; 0];
after training and before using net for new outputs?
There is a contradiction between training net and the changing made after?
What do you mean when you say in:
"The test set error is unbiased because it is completely independent of design (training and validation)." ?

采纳的回答

Greg Heath
Greg Heath 2015-5-22
This discussion has absolutely nothing to do with bias weights.
Theoretically, training, validation and test data are all assumed to be random samples from the same probability distribution function.
Both training and validation data are used to design the net with the goal of minimizing a performance function (such as mse (fitnet) or crossentropy (patternnet) for net performance on ANY data (seen and unseen) obtained from the same probability distribution.
Therefore, their performance cannot be used to obtain an unbiased (i.e., honest) estimate of performance on unseen data.
On the other hand, the test data is in no way involved in the design of the net. Therefore it is a valid representative of "unseen" data and it's performance is considered an unbiased estimate of net performance on unseen data.
Hope this helps.
Greg
PS Not sure why I never saw this post before.

更多回答(0 个)

类别

Help CenterFile Exchange 中查找有关 Statistics and Machine Learning Toolbox 的更多信息

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by