I'm trying to predict the next period of some nearly periodic data using the previous few periods (this is a case where we may be missing some data less than a period long due to sensor saturation), so I divided up the data set into sequential training, validation, and test blocks (MATLAB defaults to a random distribution of data) using
net.divideFcn = 'divideblock';
Using MATLAB's Neural Network Time Series Tool and the NARX problem (I have an input series x and target y, and y's history is known), I was wondering if MATLAB is actually predicting the 'test' data set or whether it uses that data as part of training too. So I decided to test it myself.
Here's an example using 10 periods of a sin function (want to predict the last 1.5 periods). In the second picture, I trained an NN using the same input data except I zeroed the last 1.5 periods):
My data set is not too different (it's not as smooth as a sin(), but it is very periodic), but here's my question:
- In the second picture, even when I replaced the input data set with zeroes, why does the NN prediction still follow the zeroes when it's not supposed to look at the red test data at all?
- Or, why isn't the red prediction in the two pictures identical, if the blue training set and green validation sets are identical?
I may be misunderstanding how this works, so apologies if this is a stupid question.
The full code (everything is default except the block division instead of random):
t = 0:0.01:20*pi;
r = sin(t);
r(5341:end) = 0;
X = tonndata(t',false,false);
T = tonndata(r',false,false);
trainFcn = 'trainlm';
inputDelays = 1:2;
feedbackDelays = 1:2;
hiddenLayerSize = 10;
net = narxnet(inputDelays,feedbackDelays,hiddenLayerSize,'open',trainFcn);
[x,xi,ai,t] = preparets(net,X,{},T);
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;
net.divideFcn = 'divideblock';
[net,tr] = train(net,x,t,xi,ai);
y = net(x,xi,ai);
e = gsubtract(t,y);
performance = perform(net,t,y);
figure, plotresponse(t,y)
