LSTM High Training Loss not decreasing and remain constant

43 次查看（过去 30 天）
Elaine Yong
Elaine Yong 2023-1-8
回答： Varun Sai Alaparthi 2023-1-13
I need to classify my signals into 4 classes, I am feeding 600 .mat files for every class with 360x1 data points each, I seperated training-test as 70-30.
surface "surface"
"floating"
"corona" "void"
%%%define structure
layers = [ ...
sequenceInputLayer(360)
bilstmLayer(200,'OutputMode','last') %increased from 100
fullyConnectedLayer(4)
softmaxLayer
classificationLayer
]
options = trainingOptions('adam', ...
'MaxEpochs',10, ...
'MiniBatchSize', 150, ...
'InitialLearnRate', 0.001, ...
'SequenceLength', 1000, ...
'GradientThreshold', 1, ...
'ExecutionEnvironment',"auto",...
'plots','training-progress', ...
'shuffle','every-epoch',...
'Verbose',false);
The code I took reference is from this mathwork prioject: Classify ECG Signals Using Long Short-Term Memory Networks
Despite training many times, my accuracy would not exceed 75%, the class "corona" always shows least accuracy compare to other class, these are some of the training option I have tuned one by one:
  • LSTM/BiLSTMLayer : LSTM layer accuracy will remain flat at 25%, only using BiLSTM I can see an increase
  • NumHiddenUnits: increase from 100 -> 200 only slight increase in accuracy with extended training time
  • Mini Batch Size: change to 128/256 but shows an decrease
  • Initial Learn Rate: 0.01 -> 0.001, huge increase from 25% to 75%
  • Sequence Length: 1000->100, training takes longer time
  • Shuffle every epoch: training takes longer time
I am suspecting the high loss that I have is the cause. Are there any ways I can make the training loss drop?

请先登录，再回答此问题。

回答（1 个）

Varun Sai Alaparthi
Hello Elaine,
There can be many reasons for constant high training loss without much drop. More information on validation loss and accuracy would be helpful to trace down the issue. Some possible solutions that can work to reduce the loss and increase the training accuracy would be
  1. If it’s underfitting it can be the case that model is too simple for the data or the data is uncleaned and contains lot of noise. In either case the training loss would be high. Increasing the model’s parameters or go for even better models or more cleaned data might improve the accuracy.
  2. Further decrease of learning rate also sometimes helps further decreasing the loss effectively.
  3. Try tuning more hyperparameters, try to change optimizers, adding regularization on your loss, etc.
Sometimes this can be the upper limit for those architectures for the given data too.
I hope this information helps and please reach out for any further issues.
Sincerely
Varun

类别

在 Help Center 和 File Exchange 中查找有关 Pattern Recognition and Classification 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by