Hi everyone,
I am training a recurrent neural network, and I have to give the data one by one to the network. This is neccessary due to my research. What I wonder is, giving the input one by one decreases the performance of the training. To test this, I created a network and tried to train the network in both ways. I want to show you the results.
%% First Part
rng(1);
numFeatures = 6;
numHiddenUnits = 500;
numResponses = 1;
emir = [ ...
sequenceInputLayer(numFeatures)
lstmLayer(numHiddenUnits,'OutputMode','sequence','InputWeightsInitializer','glorot')
fullyConnectedLayer(numResponses,'WeightsInitializer','glorot')
regressionLayer];
%
%
maxEpochs = 1;
miniBatchSize = 1;
options = trainingOptions('adam', ...
'MaxEpochs',1, ...
'MiniBatchSize',miniBatchSize, ...
'InitialLearnRate',0.01, ...
'GradientThreshold',1, ...
'Plots','training-progress', ...
'Verbose',0);
[emir,info] = trainNetwork(in_norm_arranged ,f_norm_arranged ,emir ,options);
%% Second Part
TrainingLoss = [];
TrainingRMSE = [];
emir = [ ...
sequenceInputLayer(numFeatures)
lstmLayer(numHiddenUnits,'OutputMode','sequence','InputWeightsInitializer','glorot')
fullyConnectedLayer(numResponses,'WeightsInitializer','glorot')
regressionLayer];
%
%
maxEpochs = 1;
miniBatchSize = 1;
options = trainingOptions('adam', ...
'MaxEpochs',1, ...
'MiniBatchSize',miniBatchSize, ...
'InitialLearnRate',0.01, ...
'GradientThreshold',1, ...
'Verbose',0);
[emir,info] = trainNetwork(in_norm_arranged{1} ,f_norm_arranged{1} ,emir ,options);
for i=2:10000
[emir,info] = trainNetwork(in_norm_arranged{i} ,f_norm_arranged{i} ,emir.Layers ,options);
TrainingLoss = [TrainingLoss info.TrainingLoss];
TrainingRMSE = [TrainingRMSE info.TrainingRMSE];
if i == 100;
100
end
end
I stopped the training in 100th iteration to show the difference between these two;
The first image is when I don't use for loop. But the minibatch size is 1, so the input is given one by one.
This figure is when I use for loop and give each input one by one.
I think in for loop, the same ''options'' are used in training, but in the other one some parameters of options are changing throughout training.
Is there a way to train a network giving inputs one by one in MATLAB?
Thanks!
Emirhan
