Hello Poorya,
I understand that you're experiencing an issue where your network's mini-batch accuracy suddenly falls to zero. If this is happening with the training accuracy, it could be a sign that your model is overfitting the data. To confirm this, I suggest splitting your dataset into training, validation, and testing segments. If your training accuracy is good but the validation and testing accuracy are not good enough, then the data is overfitting. To prevent this,
- Consider shuffling your dataset at the start of each epoch. This would ensure all the dataset are used for testing, validation and training.
- Additionally, simplifying your network's complexity can help reduce the risk of overfitting. I observe that you are using two ‘fullyConnectedLayer’ consecutively which without an activation function in between is same as one fullyConnectedLayer, so you can delete one.
Hope this helps.
