1. Use original trn/val/tst data and 'divideblock' to create and evaluate the open loop net.
2. If val and tst performance is unsatisfactory, try to design better nets via changing delays, numHidden, random initial weights and/or data-split ratios. Oherwise ...
3. Close the loop and evaluate using the ORIGINAL data. Unbiased val and tst performances are much more important than the biased trn performance (even if the latter bias is mitigated by a degree-of-freedom-adjustment).
4. Predict closed loop performance using NEW test input data. If corresponding NEW target data is available, evaluate the performance.
I think data are insufficient to achieve the desired accuracy. Besides the variables are correlated but do not show much correlation (0.35 or so).
Are you referring to zero lag correlations obtained from corrcoef or dynamic correlations obtained from nncorr, xcorr or crosscorr? Nonzero lag correlations are usually significant above 0.14 or so.
In openloop get accuracy R2a = 0.82 with MSE = 0.0016 ...
Relatively irrelevant. What are R2val and R2tst?
and performing the analysis of R2 in training / validation / test none of them made me overfitting.
Overfitting is usually detected by high R2trn but low R2val, R2tst and possibly R2trna.
You think that you should use more data?
If Nw << Ntrneq, you probably have enough training data.
If std(MSEtst) << MSEtst you probably have enough test data and similarly for val.
Look up the expression for std(MSE) assuming MSE is Chi-square distributed in Wikipedia (I only remember it for binomially distributed classifications).
Hope this helps.
Thank you for formally accepting my answer.
Greg
