Precision network analysis NarX

2013 2 26
1 个回答
回答已采纳
4 次查看（30 天）

0 个投票

Well I think I know how to do it: in the matrix of new inputs and new targets I can go poking above the predicted outputs and by looping multiple outputs could get ahead, because as I've seen in a code, to perform 300 iterations of a one time I need to give the 300 inputs (question that is impossible because of the future inputs do not know). Now the dilemma is this: I have done an analysis of details, I've closed the loop and I checked with more data if more or less fulfilled the prediction, but unfortunately I have not been successful. For this I used about 2100 but I think data are insufficient to achieve the desired accuracy. Besides the variables are correlated but do not show much correlation (0.35 or so). In openloop get accuracy R2a = 0.82 with MSE = 0.0016 and performing the analysis of R2 in training / validation / test none of them made me overfitting. You think that you should use more data? thank you very much.

请先登录，再回答此问题。

 采纳的回答

Greg Heath
Greg Heath 2013-2-26

0 个投票

1. Use original trn/val/tst data and 'divideblock' to create and evaluate the open loop net.
2. If val and tst performance is unsatisfactory, try to design better nets via changing delays, numHidden, random initial weights and/or data-split ratios. Oherwise ...
3. Close the loop and evaluate using the ORIGINAL data. Unbiased val and tst performances are much more important than the biased trn performance (even if the latter bias is mitigated by a degree-of-freedom-adjustment).
4. Predict closed loop performance using NEW test input data. If corresponding NEW target data is available, evaluate the performance.
I think data are insufficient to achieve the desired accuracy. Besides the variables are correlated but do not show much correlation (0.35 or so).
Are you referring to zero lag correlations obtained from corrcoef or dynamic correlations obtained from nncorr, xcorr or crosscorr? Nonzero lag correlations are usually significant above 0.14 or so.
In openloop get accuracy R2a = 0.82 with MSE = 0.0016 ...
Relatively irrelevant. What are R2val and R2tst?
and performing the analysis of R2 in training / validation / test none of them made me overfitting.
Overfitting is usually detected by high R2trn but low R2val, R2tst and possibly R2trna.
You think that you should use more data?
If Nw << Ntrneq, you probably have enough training data.
If std(MSEtst) << MSEtst you probably have enough test data and similarly for val.
Look up the expression for std(MSE) assuming MSE is Chi-square distributed in Wikipedia (I only remember it for binomially distributed classifications).
Hope this helps.
Thank you for formally accepting my answer.
Greg

4 个评论
显示 2更早的评论 隐藏 2更早的评论

FRANCISCO
FRANCISCO 2013-2-27
Thank you very much for your reply. I achieved incredible details in openloop, around 96% using highly correlated variables and doing some processing to achieve such precision. high R2 values both train, validation and test. In no case is made overfitting. I need a little more polishing the details when I close the loop, I think using a rather high value of hidden nodes significantly degrades accuracy so try to make the closeloop with a low value of hidden nodes, to fit precisely to the line checking. Aside will realize few iterations since the error increases iteration after iteration. Thank you very much.
FRANCISCO
FRANCISCO 2013-2-27
One question: To assess the accuracy in closeloop do as follows:
if true
% code
t3=t2(1,1:31); %Top 30 t2 values (targets of check)
errors = gsubtract(t3,yPredicted); %yPredicted are predicted by closeloop outputs, namely 30
t3=tonndata(t3,true,false);
MSEclosed = perform(netc,t3,yPredicted)
R2=1-MSEclosed/MSE00
end
Is this a correct way to evaluate the accuracy in closeloop?
Many thanks
Greg Heath
Greg Heath 2013-2-27
I don't know. How are your t1, t2, t3 and MSE00 defined?
What are I, N, O, Ntrn, Nval, Ntst, ID, FD, and H?
Let
mxD = max(ID,FD) % size of delay buffer
then
1. The most unbiased approach is to
a. Only use the prior information stored in netc
b. Only use test set data that was used in no way for design
2. If Ntst is not sufficiently large to satisfy std(MSEtst) << MSEtst it may be useful to add a small bias by using the preceeding mxID vectors from the design data (val or trn if Nval = 0).
FRANCISCO
FRANCISCO 2013-2-27
Well, in this case, I've reduced the sample to 250 data because before about 2200 and used the network sobreajustaba me, then I decided to try a smaller sample size. To obtain this sample:
N = 250
Neq = 250
Nw = 121
O = 1
ID = 1:2
FD = 1:2
H = 12
openloop and get results:
MSE = 6.2847 e-05
MSE00 = 0.000121
R2 = 0.974646
R2a = 0.951088
R2trn = 0.9823031
R2val = 0.89007373
R2tst = 0.92343341
The code I used is as follows: if true % code MSE = perform(net,targets,outputs);
MSEa=Neq*MSE/(Neq-Nw);
R2=1-MSE/MSE00; R2a=1-MSEa/MSE00a; MSEtrn=tr.perf(end);
MSEval=tr.vperf(end);
MSEtst=tr.tperf(end);
R2trn=1-MSEtrn/MSE00;
R2val=1-MSEval/MSE00;
R2tst=1-MSEtst/MSE00;
end
And I used
if true
% code
net.divideFcn='divideblock';
net.divideParam.trainRatio=0.70;
net.divideParam.valRatio=0.15;
net.divideParam.testRatio=0.15;
net.trainParam.goal=0.01*MSE00;
end
I would like to have uploaded a photo of when I do because I can closeloop very close to the actual data and performing 10 iterations. Many thanks Greg

请先登录，再进行评论。

更多回答（0 个）

类别

在 帮助中心 和 File Exchange 中查找有关 Modeling and Prediction with NARX and Time-Delay Networks 的更多信息

产品

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by