Validation Accuracy on Neural network

Question

Andrik Rampun 2019-2-18

1
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/445599-validation-accuracy-on-neural-network

回答： Sevda Kemba 2022-6-6

Hello..I wonder if any of you who have used deep learning on matlab can help me to troubleshoot my problem. I don't understand why I got a sudden drop of my validation accuracy at the end of the graph? It's a simple network with one convolution layer to classify cases with low or high risk of having breast cancer. After the final iteration it displays a validation accuracy of above 80% but then suddenly it dropped to 73% without an iteration. I don't understand that.

Here's my code

%set training dataset folder
digitDatasetPath = fullfile('C:\Users\UOS\Documents\Desiree Data\Run 
2\dataBreast\training2');
%training set
imdsTrain = imageDatastore(digitDatasetPath, ...
'IncludeSubfolders',true,'LabelSource','foldernames');
%set validation dataset folder
validationPath = fullfile('C:\Users\UOS\Documents\Desiree Data\Run 
2\dataBreast\validation2');
%testing set
imdsValidation = imageDatastore(validationPath, ...
'IncludeSubfolders',true,'LabelSource','foldernames');
%create a clipped ReLu layer
layer = clippedReluLayer(10,'Name','clip1');
% define network architecture
layers = [
imageInputLayer([256 256 1]);
% conv_1
convolution2dLayer(3,32,'Stride',1)
batchNormalizationLayer
clippedReluLayer(10);
maxPooling2dLayer(2,'Stride',2)
%fc
fullyConnectedLayer(100)
dropoutLayer(0.7,'Name','drop1');
%fc
fullyConnectedLayer(25)
dropoutLayer(0.8,'Name','drop2'); 
% fc layer
fullyConnectedLayer(2)
softmaxLayer
classificationLayer];
% specify training option
options = trainingOptions('adam', ...
'InitialLearnRate',0.001, ...
'MaxEpochs',15, ...
'Shuffle','every-epoch', ...
'ValidationData',imdsValidation, ...
'ValidationFrequency',30, ...
'Verbose',false, ...
'Plots','training-progress');
% train network using training data
net = trainNetwork(imdsTrain,layers,options);
% classify validation images and compute accuracy
YPred = classify(net,imdsValidation);
YValidation = imdsValidation.Labels;
%calculate accuracy
accuracy = sum(YPred == YValidation)/numel(YValidation);

8 个评论
显示 6更早的评论隐藏 6更早的评论

Andrik Rampun 2019-2-20

编辑：Andrik Rampun 2019-2-20

在 MATLAB Online 中打开

Hi Jon, Don,

Here I train the network without batch normalization and I got no jump/drop at the end of the training but as you can see the results are are really low. I notice in matlab tutorial they also using batch normalization and when I run the code I didn't get a jump/drop at the end of the iteration.

% set training dataset folder
digitDatasetPath = fullfile('C:\Users\UOS\Documents\Desiree Data\Run 2\dataBreast\training2');
imdsTrain = imageDatastore(digitDatasetPath, ...
    'IncludeSubfolders',true,'LabelSource','foldernames');
% set validation dataset folder
validationPath = fullfile('C:\Users\UOS\Documents\Desiree Data\Run 2\dataBreast\validation2');
imdsValidation = imageDatastore(validationPath, ...
    'IncludeSubfolders',true,'LabelSource','foldernames');
% create a clipped ReLu layer
layer = clippedReluLayer(10,'Name','clip1');
% define network architecture
layers = [
    imageInputLayer([256 256 1], 'Normalization', 'none')
    
    % conv_1
    convolution2dLayer(3,16,'Stride',1)
    %batchNormalizationLayer
    clippedReluLayer(10);
    maxPooling2dLayer(2,'Stride',2)
    
    
    % fc layer
    fullyConnectedLayer(2)
    softmaxLayer
    classificationLayer];
% specify training option
options = trainingOptions('adam', ...
    'InitialLearnRate',0.001, ...
    'MaxEpochs',15, ...
    'Shuffle','every-epoch', ...
    'ValidationData',imdsValidation, ...
    'ValidationFrequency',30, ...
    'Verbose',false, ...
    'MiniBatchSize',32, ...
    'Plots','training-progress');
% train network using training data
net = trainNetwork(imdsTrain,layers,options);
% classify validation images and compute accuracy
YPred = classify(net,imdsValidation);
YValidation = imdsValidation.Labels;
accuracy = sum(YPred == YValidation)/numel(YValidation)

amir aslam 2019-4-16

I have done the same experiment but accuracy does not drop

code:

clc;

clear;

%set training dataset folder

digitDatasetPath = fullfile('C:\Program Files\MATLAB\R2018aaa\toolbox\nnet\nndemos\nndatasets\Experiments\Train');

%training set

imdsTrain = imageDatastore(digitDatasetPath, ...

'IncludeSubfolders',true,'LabelSource','foldernames');

%set validation dataset folder

validationPath = fullfile('C:\Program Files\MATLAB\R2018aaa\toolbox\nnet\nndemos\nndatasets\Experiments\Test');

%testing set

imdsValidation = imageDatastore(validationPath, ...

'IncludeSubfolders',true,'LabelSource','foldernames');

%create a clipped ReLu layer

%layer = clippedReluLayer(10,'Name','clip1');

% define network architecture

layers = [

imageInputLayer([48 48 3]);

% conv_1

convolution2dLayer(3,32,'Stride',1)

batchNormalizationLayer

reluLayer;

maxPooling2dLayer(2,'Stride',2)

%fc

fullyConnectedLayer(100)

dropoutLayer(0.7,'Name','drop1');

%fc

fullyConnectedLayer(25)

dropoutLayer(0.8,'Name','drop2');

% fc layer

fullyConnectedLayer(7)

softmaxLayer

classificationLayer];

% specify training option

options = trainingOptions('adam', ...

'InitialLearnRate',0.0001, ...

'MaxEpochs',15, ...

'Shuffle','every-epoch', ...

'ValidationData',imdsValidation, ...

'ValidationFrequency',30, ...

'Verbose',false, ...

'Plots','training-progress');

% train network using training data

net = trainNetwork(imdsTrain,layers,options);

% classify validation images and compute accuracy

YPred = classify(net,imdsValidation);

YValidation = imdsValidation.Labels;

%calculate accuracy

accuracy = sum(YPred == YValidation)/numel(YValidation);

Sridharan K 2021-3-10

i got 100% accuracy. thanks for this program.

Santhosh Surya Kiran 2021-7-1

What u got is wrong..

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Andrik Rampun 2019-2-19

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/445599-validation-accuracy-on-neural-network#answer_361779

在 MATLAB Online 中打开

Some updates. I got similar results (sudden drop) at the end of the graph. I find this really strange.

% set training dataset folder
digitDatasetPath = fullfile('C:\Users\UOS\Documents\Desiree Data\Run 2\dataBreast\training2');
imdsTrain = imageDatastore(digitDatasetPath, ...
    'IncludeSubfolders',true,'LabelSource','foldernames');
% set validation dataset folder
validationPath = fullfile('C:\Users\UOS\Documents\Desiree Data\Run 2\dataBreast\validation2');
imdsValidation = imageDatastore(validationPath, ...
    'IncludeSubfolders',true,'LabelSource','foldernames');
% create a clipped ReLu layer
layer = clippedReluLayer(10,'Name','clip1');
% define network architecture
layers = [
    imageInputLayer([256 256 1], 'Normalization', 'none')
    
    % conv_1
    convolution2dLayer(3,16,'Stride',1)
    batchNormalizationLayer
    clippedReluLayer(10);
    maxPooling2dLayer(2,'Stride',2)
    
    % conv_2
    convolution2dLayer(3,16,'Stride',1)
    batchNormalizationLayer
    clippedReluLayer(10);
    maxPooling2dLayer(2,'Stride',2)
   
    % conv_3
    convolution2dLayer(3,32,'Stride',1)
    batchNormalizationLayer
    clippedReluLayer(10);
    maxPooling2dLayer(2,'Stride',2)
    
    % conv_4
    convolution2dLayer(3,64,'Stride',1)
    batchNormalizationLayer
    clippedReluLayer(10);
    maxPooling2dLayer(2,'Stride',2)
    
    % conv_5
    convolution2dLayer(3,128,'Stride',1)
    batchNormalizationLayer
    clippedReluLayer(10);
    maxPooling2dLayer(2,'Stride',2)
    
    % conv_6
    convolution2dLayer(3,256,'Stride',1)
    batchNormalizationLayer
    clippedReluLayer(10);
    maxPooling2dLayer(2,'Stride',2)
    
    % fc5
    fullyConnectedLayer(500)
    dropoutLayer(0.5,'Name','drop1'); 
    
        
    % fc5
    fullyConnectedLayer(250)
    dropoutLayer(0.5,'Name','drop2');
    
    
    % fc5
    fullyConnectedLayer(50)
    dropoutLayer(0.5,'Name','drop3'); 
    
    % fc layer
    fullyConnectedLayer(2)
    softmaxLayer
    classificationLayer];
% soecify training option
options = trainingOptions('adam', ...
    'InitialLearnRate',0.001, ...
    'MaxEpochs',30, ...
    'Shuffle','every-epoch', ...
    'ValidationData',imdsValidation, ...
    'ValidationFrequency',30, ...
    'Verbose',false, ...
    'Plots','training-progress');
% train network using training data
net = trainNetwork(imdsTrain,layers,options);
% classify validation images and compute accuracy
YPred = classify(net,imdsValidation);
YValidation = imdsValidation.Labels;
accuracy = sum(YPred == YValidation)/numel(YValidation)
%YPred = predict(net,X);
analyzeNetwork(net)

18 个评论
显示 16更早的评论隐藏 16更早的评论

Andrik Rampun 2019-2-26

在 MATLAB Online 中打开

Hi Don,

Just to follow up some updates here...

I ran the following code

clear all
% set training dataset folder
digitDatasetPath = fullfile('/data/md1ybr/dataBreast/Run1/training');
imdsTrain = imageDatastore(digitDatasetPath, ...
    'IncludeSubfolders',true,'LabelSource','foldernames');
% set validation dataset folder
validationPath = fullfile('/data/md1ybr/dataBreast/Run1/validation');
imdsValidation = imageDatastore(validationPath, ...
    'IncludeSubfolders',true,'LabelSource','foldernames');
% create a clipped ReLu layer
layer = clippedReluLayer(5,'Name','clip1');
% define network architecture
layers = [
    imageInputLayer([256 256 1], 'Normalization', 'zerocenter')
    
    % conv_1
    convolution2dLayer(7,64,'Stride',1)
    batchNormalizationLayer
    clippedReluLayer(5);
    maxPooling2dLayer(2,'Stride',2)
    % fc2
    fullyConnectedLayer(3136)
    dropoutLayer(0.1,'Name','drop1');     
        
    % fc3
    fullyConnectedLayer(3136)
    dropoutLayer(0.1,'Name','drop2');
     
    % fc layer
    fullyConnectedLayer(2)
    softmaxLayer
    classificationLayer];
% soecify training option
options = trainingOptions('adam', ...
    'InitialLearnRate',0.001, ...
    'MaxEpochs',200, ...
    'Shuffle','every-epoch', ...
    'ValidationData',imdsValidation, ...
    'MiniBatchSize',2048, ...
    'ValidationFrequency',30, ...
    'Verbose',true);
% train network using training data
net = trainNetwork(imdsTrain,layers,options);
% classify validation images and compute accuracy
YPred = classify(net,imdsValidation);
YValidation = imdsValidation.Labels;
accuracy = sum(YPred == YValidation)/numel(YValidation);
disp('Training Acc:');
disp(accuracy);
% set validation dataset folder
testingPath = fullfile('/data/md1ybr/dataBreast/Run1/testing');
imdsTesting = imageDatastore(testingPath, ...
    'IncludeSubfolders',true,'LabelSource','foldernames');
ZValidation = imdsTesting.Labels;
ZPred = predict(net,imdsTesting);
pre=[];
for i=1:size(ZPred,1)
   if ZPred(i,1)>ZPred(i,2)
       pre=[pre;1];
   else
       pre=[pre;2];
   end
end
testAccuracy = sum(pre == grp2idx(ZValidation))/numel(ZValidation);
disp('Testing Acc:');
disp(testAccuracy);

I got the following error (I don't think the size of the image is that large to be honest.)

Error using trainNetwork (line 150)
Maximum variable size allowed on the device is exceeded.
Error in brestDensityClassification5v4a (line 53)
net = trainNetwork(imdsTrain,layers,options);
Caused by:
    Error using gpuArray
    Maximum variable size allowed on the device is exceeded.

Andrik Rampun 2019-2-26

Hi Don,

When you said shirink batchsize do you mean my MinibatchSize?

Another question is I can even reduce my hidden units in my FC layer?

Thanks

Don Mathis 2019-2-26

Yes, MiniBatchSize. And I meant the outputSize of your fullyConnectedLayers could be reduced to something smaller than 3136:

fullyConnectedLayer(3136)

请先登录，再进行评论。

Answer 2

Saira 2020-6-15

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/445599-validation-accuracy-on-neural-network#answer_451422

Hi,

I have 5600 training images. I have extracted features using Principal Component Analysis (PCA). Then I am applying CNN on extracted features. My training accuracy is 30%. How to increase training accuracy?

Feature column vector size: 640*1

My training code:

% Convolutional neural network architecture

layers = [

imageInputLayer([1 640 1]);

reluLayer

fullyConnectedLayer(7);

softmaxLayer();

classificationLayer()];

options = trainingOptions('sgdm', 'Momentum',0.95, 'InitialLearnRate',0.0001, 'L2Regularization', 1e-4, 'MaxEpochs',5000, 'MiniBatchSize',8192, 'Verbose', true);

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

Answer 3

Sevda Kemba 2022-6-6

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/445599-validation-accuracy-on-neural-network#answer_979815

@Andrik Rampun Hello. In Matlab, we load the data set with code and limit it in deep learning. But when we train, validation accuracy stays between 40-50%. What can we do to increase it to 90%? We would be very happy if you could help.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

Answer 4

Sevda Kemba 2022-6-6

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/445599-validation-accuracy-on-neural-network#answer_979825

@Saira Hello. In Matlab, we load the data set with code and limit it in deep learning. But when we train, validation accuracy stays between 40-50%. What can we do to increase it to 90%? We would be very happy if you could help.