By default, the data is divided into three subsets (trn/val/tst):
total = training + validation + testing
You can think of it two different ways:
1. total = designing + testing
designing = training + validation
2. total = training + nontraining
nontraining = validation + testing
TRAINING SUBSET: Design data used to directly determine the weights and thresholds of the model given inputs and targets (desired outputs). It's performance estimates are, obviously, highly biased.
VALIDATION SUBSET: Nontraining design data used two ways:
1. Validation stopping: Training is stopped
when the error on the validation subset
increases for 6(default) consecutive epochs.
2. Ranking multiple designs with slightly
biased performance estimates that typically
differ by
a. number of hidden nodes
b. initial random weight assignments.
TESTING SUBSET: Once designs are ranked to be satisfactory, UNBIASED PERFORMANCE ESTIMATES are obtained using the test subset.
If no unbiased performance estimate is
satisfactory, the process can be repeated with
the data repartitioned and/or design parameters
changed.
MATLAB yields separate confusion matrices for the three data division subsets in addition to the one for all of the data .
Hope this helps.
Greg
