Training accuracy remains constant and loss keeps decreasing

Question

navneet brar 2021-6-16

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/857355-training-accuracy-remains-constant-and-loss-keeps-decreasing

评论： navneet brar 2021-7-24

Hi… I am using CNN for image classification. From the simulation results it is observed that the mini batch accuracy reaches 100% after few epochs and mini batch loss keeps on decreases as shown below:

| 1 | 1 |00:00:10 | 43.75% | 8.9676 | 1.0000e-04 |

| 1 | 9 | 00:00:34 | 43.75% | 2.3432 | 1.0000e-04 |

| 2 | 18 | 00:00:57 | 68.75% | 0.4471 | 1.0000e-04 |

| 3 | 27 | 00:01:19 | 56.25% | 0.8530 | 1.0000e-04 |

| 4 | 36 | 00:01:43 | 81.25% | 0.4184 | 1.0000e-04 |

| 5 | 45 | 00:02:08 | 93.75% | 0.3022 | 1.0000e-04 |

| 6 | 54 | 00:02:33 | 81.25% | 0.2594 | 1.0000e-04 |

| 7 | 63 | 00:02:59 | 87.50% | 0.5467 | 1.0000e-04 |

| 8 | 72 | 00:03:23 | 93.75% | 0.1394 | 1.0000e-04 |

| 9 | 81 | 00:03:50 | 100.00% | 0.0409 | 1.0000e-04 |

| 10 | 90 | 00:04:14 | 100.00% | 0.0920 | 1.0000e-04 |

| 11 | 99 | 00:04:38 | 100.00% | 0.0318 | 1.0000e-04 |

| 12 | 108 | 00:05:02 | 93.75% | 0.1280 | 1.0000e-04 |

| 13 | 117 | 00:05:26 | 93.75% | 0.1724 | 1.0000e-04 |

| 14 | 126 | 00:05:50 | 87.50% | 0.2529 | 1.0000e-04 |

| 15 | 135 | 00:06:14 | 100.00% | 0.0252 | 1.0000e-04 |

| 16 | 144 | 00:06:38 | 100.00% | 0.0362 | 1.0000e-04 |

| 17 | 153 | 00:07:04 | 100.00% | 0.0345 | 1.0000e-04 |

| 18 | 162 | 00:07:28 | 100.00% | 0.0400 | 1.0000e-04 |

| 19 | 171 | 00:07:53 | 100.00% | 0.0552 | 1.0000e-04 |

| 20 | 180 | 00:08:17 | 100.00% | 0.0704 | 1.0000e-04 |

Is it correct to get such results or I am doing something wrong??

What is the reason behind these values??

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Divyam Gupta 2021-6-16

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/857355-training-accuracy-remains-constant-and-loss-keeps-decreasing#answer_725930

Hi,

I notice that your loss is fluctuating a lot after the 6th epoch of training while the accuracy stagnates for a certain number of epochs. There could be multiple reasons for this, including a high learning rate, outlier data being used while training etc. You can try reducing the learning rate or progressively scaling down the learning rate using the 'LearnRateSchedule' parameter in the trainingOptions documentation.

Additionally, you could try stopping the training process early, i.e., if the accuracy doesn't change for a few epochs, the model stops the training process, thus preventing further fluctuations. Please go through the following link to achieve this: https://www.mathworks.com/help/deeplearning/ug/customize-output-during-deep-learning-training.html

Hope this helps.

2 个评论
显示无隐藏无

Philip Brown 2021-6-21

The minibatch loss is the loss computed for just a single minibatch (which might be a tiny fraction of your dataset), so it's not too surprising that the loss and accuracy fluctuate between minibatches. Each minibatch contains only a small number of images.

Given that you are getting 100 % accuracy on your training set after a small number of iterations, it could be valuable to check if your network is going to overfit when used on new data it hasn't seen before. Take a look at the section "Specify Validation Data" on this doc page for more info.

I agree with Divyam's suggestion that you might want to stop training earlier once the loss stops decreasing. Take a look at the ValidationPatience option in trainingOptions for details of this.

navneet brar 2021-7-24