- Consider shuffling your dataset at the start of each epoch. This would ensure all the dataset are used for testing, validation and training.
- Additionally, simplifying your network's complexity can help reduce the risk of overfitting. I observe that you are using two ‘fullyConnectedLayer’ consecutively which without an activation function in between is same as one fullyConnectedLayer, so you can delete one.
Zero Mini-batch Accuracy and drastic increase in Mini-batch Loss during LSTM training Network
1 次查看(过去 30 天)
显示 更早的评论
I am training a weighted loss LSTM on an unbalanced data, as belew:
% classes with fewer number of samples have larger weight
classWeights = [.10,.60,.30]
% sum up to 1
layers = [ ...
sequenceInputLayer(numFeatures)
lstmLayer(numHiddenUnits,'OutputMode','last')
dropoutLayer(0.2)
fullyConnectedLayer(100)
fullyConnectedLayer(50)
fullyConnectedLayer(numClasses)
softmaxLayer
weightedClassificationLayerLSTM(classWeights,"classoutput")]
miniBatchSize = 36;
options = trainingOptions('adam', ...
'InitialLearnRate',0.0001, ...
'SquaredGradientDecayFactor',0.99, ...
'Epsilon',1.0000e-08, ...
'MaxEpochs',130, ...
'GradientThreshold',2, ...
'MiniBatchSize',miniBatchSize , ...
'ValidationFrequency', 10, ...
'Verbose',1, ...
'Plots','training-progress',...
'LearnRateDropFactor',0.9, ...
'LearnRateDropPeriod',10, ...
'LearnRateSchedule','piecewise', ...
'Shuffle', 'never')
During the training sometimes the Mini-batch Accuracy drops to zero and simultaneously the Mini-batch Loss increases, I appreciate any hint to resolve this issue.
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/515242/image.png)
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/515247/image.png)
0 个评论
回答(1 个)
Krishna
2024-4-24
Hello Poorya,
I understand that you're experiencing an issue where your network's mini-batch accuracy suddenly falls to zero. If this is happening with the training accuracy, it could be a sign that your model is overfitting the data. To confirm this, I suggest splitting your dataset into training, validation, and testing segments. If your training accuracy is good but the validation and testing accuracy are not good enough, then the data is overfitting. To prevent this,
Hope this helps.
0 个评论
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Sequence and Numeric Feature Data Workflows 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!