How to plot test and validation accuracy every epoch using Computer vision system toolbox? And what about overfitting?

Question

Jay 2017-5-18

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/340893-how-to-plot-test-and-validation-accuracy-every-epoch-using-computer-vision-system-toolbox-and-what

评论： Sid Khan 2022-3-1

I am finding many blogs of CNN and its related classification strategy in Matlab but I couldn't find how I could actually plot the validation set accuracy for every epoch along with the training set accuracy.

https://www.mathworks.com/help/nnet/ref/trainingoptions.html#namevaluepairarguments

above documentation shows that I could plot training accuracy every epoch but not the validation set accuracy.

If that is not possible how to make sure that my network is not overfitting?

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Sebastian K 2017-5-26

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/340893-how-to-plot-test-and-validation-accuracy-every-epoch-using-computer-vision-system-toolbox-and-what#answer_268601

Hi Jay,

There is no option to display the validation set accuracy. However this should not be necessary. Functions in the Neural Network Toolbox employ a technique called Early Stopping, which takes the validation set error into consideration to prevent overfitting. The article I am linking to provides some nice discussions on overfitting, I would recommend you to read the other parts as well.

Cheers,

Sebastian

5 个评论
显示 3更早的评论隐藏 3更早的评论

Jay 2017-6-6

编辑：Jay 2017-6-6

Hi Sebastian,

I have already referred the link given. However, it seems that it is for conventional neural networks.

In CNN usually, a good network would not overfit when you increase the number of Epoch.

For example, let's say I train the network for 60 epoch in total but my training and validation error is converging around 40 epochs. Now it is always a good practice to run for little more number of epoch, to check whether any parameter is causing the overfitting if the network gets overtrained. It that is the case I would tune the parameters that might cause the overfitting. Last 20 Epoch will perform this operation.

Refer the image below. For above description.

ONE SOLUTION: I have thought about the solution of plotting these types of graph is, let the training complete and for total number of epoch. for every epoch save the check points. Once training gets done, load every checkpoint and measure the accuracy on the validation set for every particular checkpoint.

I am not sure if that is possible though! I haven't implemented it yet.

Keke Zhang 2018-5-15

Hi Jay: I want to know how did you draw the picture you gave? Thank you very much!

Anu Sebastian 2021-4-28

Hlo jay How to code this

请先登录，再进行评论。

Answer 2

Evan Koester 2018-3-27

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/340893-how-to-plot-test-and-validation-accuracy-every-epoch-using-computer-vision-system-toolbox-and-what#answer_312289

编辑：Evan Koester 2018-3-27

This problem has been addressed in the newly released MATLAB 2018a. If you have a groundtruth of your data, you can load it as a pixelLabelImageDatastore. This can be used in your ValidationData trainingOptions.

An example:

val4data = imageDatastore(location of image data); val4label = load('location of label groundtruth');

val4label = pixelLabelDatastore(val4label.gTruth); val4gt = pixelLabelImageDatastore(val4data,val4label);

opts = trainingOptions('sgdm', ... 'MaxEpochs', 5000, ... 'InitialLearnRate', .05, ... 'VerboseFrequency',validationFrequency,... 'ValidationData',val4gt,... 'ValidationFrequency',5,... 'Plots','training-progress',... 'CheckpointPath', tempdir,... 'MiniBatchSize', 48);

This will plot the validation data loss on the same plot as training loss when training your CNN. In my application I used pixel-wise labeling. Prior to MATLAB version 2018a, there was not a way to perform this without making a checkpoint, test validation data, continue training type of algorithm as mentioned above.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

Answer 3

Saira 2020-6-15

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/340893-how-to-plot-test-and-validation-accuracy-every-epoch-using-computer-vision-system-toolbox-and-what#answer_451431

Hi,

I have 5600 training images. I have extracted features using Principal Component Analysis (PCA). Then I am applying CNN on extracted features. My training accuracy is 30%. How to increase training accuracy?

Feature column vector size: 640*1

My training code:

% Convolutional neural network architecture

layers = [

imageInputLayer([1 640 1]);

reluLayer

fullyConnectedLayer(7);

softmaxLayer();

classificationLayer()];

options = trainingOptions('sgdm', 'Momentum',0.95, 'InitialLearnRate',0.0001, 'L2Regularization', 1e-4, 'MaxEpochs',5000, 'MiniBatchSize',8192, 'Verbose', true);

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Sid Khan 2022-3-1

increase dataset size

请先登录，再进行评论。

How to plot test and validation accuracy every epoch using Computer vision system toolbox? And what about overfitting?

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

采纳的回答

5 个评论
显示 3更早的评论隐藏 3更早的评论

更多回答（2 个）

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

另请参阅

类别

标签

产品

Community Treasure Hunt

How to plot test and validation accuracy every epoch using Computer vision system toolbox? And what about overfitting?

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

采纳的回答

5 个评论 显示 3更早的评论隐藏 3更早的评论

更多回答（2 个）

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

1 个评论 显示 -1更早的评论隐藏 -1更早的评论

另请参阅

类别

标签

产品

Community Treasure Hunt

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

5 个评论
显示 3更早的评论隐藏 3更早的评论

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

1 个评论
显示 -1更早的评论隐藏 -1更早的评论