% I've been reading in documentation that validation samples are used to measure network generalization; i.e found out how my network would perform on data it has never seen before.
NO. See Below.
%There's also testing samples which are used to give an independent measure of network performance during and after training; used to determing when to stop training.
1. total = design + test
2. design = training + validation
3. training:
a. used to obtain weight values given training parameters.
b. training error estimates tend to be extremely biased as the number
of unknown weights, Nw, increases toward the number of training
equations, Ntrneq.
c. Ndof = Ntrneq-Nw is the number of estimation degrees of freedom
(See Wikipedia). As long as Ndof is sufficiently positive, the bias
of estimating error with training data can be mitigated, somewhat,
by using the degree of freedom adjustment of dividing SSEtrn by Ndof
instead of Ntrneq.
4. validation:
a. used repeatedly with the training set to determine a good set of training parameters (especially the stopping epoch) via choosing the best of multiple random initial weight designs.
b. Validation set error tends to be much less biased than training set error, especially if training doesn't stop because of validation error convergence.
5. test:
a.used once, and only once to obtain an unbiased error estimate of nontraining data.
b. if performance is unsatisfactory and more designs are necessary, the data should be repatitioned into new tr/val/tst subsets.
%I would still like to use these. A work around to this is to combine my training, validation, and testing vectors into one matrix.
No. This not a work around. 'dividerand' is the default.
MATLAB uses
Ntst = round(tstratio*N)
Nval = round(valratio*N)
Ntrn = N - Nval - Ntst.
The training record, tr, contains the indices for each subset.
[ net tr Y E ] = train(net,x,t);
Y is the output and E is the error E = t-Y.
If you want , you can use dividerand anytime before training to obtain the indices then assign the indices to the net by using divideind.
Hope this helps.
*Thank you for formally accepting my answer
Greg
