I have a database called processed_data, which contains cells structured like this:
0.980999999999767 0.945912306864893 1
1.46300000000338 0.926617136227153 1
0.511999999995169 0.868790509137634 2
1.00600000000122 0.978074194186882 1
0.995999999999185 0.884817478795566 2
1.12400000000343 0.740093883803231 2
1.35399999999936 0.418494628137842 2
0.653999999994994 0.399199457500103 2
1.00600000000122 0.438938088213894 2
0.999000000003434 0.566427539286267 2
The first column represents the seconds that have passed since the previous row, the second column is the value at that given time (normalized), and the third column contains the categorical values 1 and 2. The first two columns are the predictors, and the third column is the target.
Each cell represents one day and I have around 215 days worth of data, each with a varying number of observations. The goal is to create an LSTM algorithm that, based on the predictors, can predict whether the value will increase (2) or decrease (1) in the future. During training, I'm keeping each day separate by stopping the learning when the last batch for a given day is reached, then loading the data for the next day and resuming the training.
The problem is that when training resumes after loading the next day's data, it's as if the network is completely reset and starts learning from scratch. It repeatedly produces the same accuracy results (excluding the first iteration) with only slight changes in loss values. The accuracy in the remaining iterations remains unchanged, always producing the same values, as if the network is not learning at all. Here is an example output for day 1:
Iteration Epoch TimeElapsed LearnRate TrainingLoss TrainingAccuracy
_________ _____ ___________ _________ ____________ ________________
1 1 00:00:00 0.001 0.69781 40.625
50 1 00:00:00 0.001 0.65881 64.844
100 1 00:00:00 0.001 0.70176 50.781
117 1 00:00:00 0.001 0.63057 69.531
Training stopped: Max epochs completed
Iteration Epoch TimeElapsed LearnRate TrainingLoss TrainingAccuracy
_________ _____ ___________ _________ ____________ ________________
1 1 00:00:00 0.001 0.70017 41.406
50 1 00:00:00 0.001 0.65913 64.844
100 1 00:00:00 0.001 0.6985 50.781
117 1 00:00:00 0.001 0.62994 69.531
Training stopped: Max epochs completed
Iteration Epoch TimeElapsed LearnRate TrainingLoss TrainingAccuracy
_________ _____ ___________ _________ ____________ ________________
1 1 00:00:00 0.001 0.69753 42.188
50 1 00:00:00 0.001 0.6619 64.844
100 1 00:00:00 0.001 0.70356 50.781
117 1 00:00:00 0.001 0.6291 69.531
Training stopped: Max epochs completed
Here is my code snippet:
train_opts = trainingOptions( ...
InitialLearnRate = 0.001, ...
sequenceInputLayer(2,"Name","input")
lstmLayer(256,"Name","lstm","OutputMode","last")
dropoutLayer(0.5,"Name","dropout")
fullyConnectedLayer(2,"Name","output")
net = addLayers(net, temp_net);
train_data_length = round(length(processed_data) * 0.9);
train_data = processed_data(1: train_data_length);
for epoch = 1:num_of_epochs
for day = 1:train_data_length
train_X = processed_data{day}(:, 1:2);
train_X = dlarray(train_X, "BCT");
train_Y = categorical(processed_data{day}(:, 3));
net = trainnet(train_X, train_Y, net, "crossentropy", train_opts);
