Invalid training data. Responses must be nonempty when using networkTrain on CNN

2020 9 3
1 个回答
回答已采纳
更新时间：2021 7 9
2 次查看（30 天）

1 个投票

Hello,
I'm having issues with the transform of an ImageDatastore when is used as input to the networkTrain method. If I use the original ImageDatastore as input to the networkTrain, everything works as a charm. By using the read method on the transformed datastore and visualizing the images with imshow() I can see that the transform function is doing what is supposed to do (i.e., adding noise to the images in the ImageDatastore). However, when passing the TransformedDatastore to the networkTrain, the following message is thrown:
Error using trainNetwork (line 170)
Invalid training data. Responses must be nonempty.

请先登录，再回答此问题。

 采纳的回答

Mohammad Sami
Mohammad Sami 2020-9-4

1 个投票

If you have image processing toolbox, you may perhaps use denoisingImageDatastore
https://www.mathworks.com/help/releases/R2020a/images/ref/denoisingimagedatastore.html

9 个评论
显示 7更早的评论 隐藏 7更早的评论

Carlos Ramirez
Carlos Ramirez 2020-9-4
编辑：Carlos Ramirez 2020-9-4
Thank you Sami. Unfortunately, is more than just noise. I'm trying to change luminance conditions, non-linear streching of the color histograms, etc., so it will be nice to be able to take advantage of the Transformed Datastore, which by the way is doing what I am asking it to do, it's just that networkTraing doesn't like it as input. In fact, if I pass to the networkTrain, the 'transformedDataStore.UnderlyingDatastore' , the networkTrain method smiles and start doing it's job. But, when I pass the 'transformedDataStore' itself to the networkTrain it complains that responses must not be empty, which does not make sense.
Mohammad Sami
Mohammad Sami 2020-9-4
Can you check the labels property on the transformed data store
Carlos Ramirez
Carlos Ramirez 2020-9-4
编辑：Carlos Ramirez 2020-9-5
Sami, the transformed data store does not have a Labels property. I believe it relies on the labels stored in the UnderlyingDatastore. I borrowed a simple example from the Matlab help to reproduce the problem. Here it is:
digitDatasetPath = fullfile(matlabroot,'toolbox','nnet', ...
'nndemos','nndatasets','DigitDataset');
imds = imageDatastore(digitDatasetPath, ...
'IncludeSubfolders',true, ...
'LabelSource','foldernames');
numTrainingFiles = 750;
[imdsTrain,imdsTest] = splitEachLabel(imds,numTrainingFiles,'randomize');
layers = [ ...
imageInputLayer([28 28 1])
convolution2dLayer(5,20)
reluLayer
maxPooling2dLayer(2,'Stride',2)
fullyConnectedLayer(10)
softmaxLayer
classificationLayer];
options = trainingOptions('sgdm', ...
'MaxEpochs',20,...
'InitialLearnRate',1e-4, ...
'Verbose',false, ...
'Plots','training-progress');
idmsNew = transform(imds,@transformFcn);
net = trainNetwork(idmsNew,layers,options);
function [dataOut]= transformFcn(dataIn)
Gray=dataIn;
%Gray=Gray+20; // do whatever transformation you want to do here
dataOut=Gray;
end
which results in:
Error using trainNetwork (line 170)
Invalid training data. Responses must be nonempty.
Error in test (line 28)
net = trainNetwork(idmsNew,layers,options);
Mohammad Sami
Mohammad Sami 2020-9-5
编辑：Mohammad Sami 2020-9-5
Seems that the read function must output a cell array with input + 1 number of columns.
The last column should contain the responses (Labels).
First you need set includeinfo to true, when calling the transform.
idmsNew = transform(imds,@transformFcn,'IncludeInfo',true);
You will then need to amend your transform function as follows.
function [dataOut,info] = transformFcn(data,info)
if(length(info) > 1)
numRows = length(info);
dataOut = cell(numRows,2);
else
% if readsize = 1
numRows = 1;
data = {data};
end
for idx = 1:numRows
% Randomized 90 degree rotation
imgOut = rot90(data{idx,1},randi(4)-1);
% Return the label from info struct as the
% second column in dataOut.
dataOut(idx,:) = {imgOut,info.Label(idx)};
end
end
See the example at this link below
https://www.mathworks.com/help/deeplearning/ug/datastores-for-deep-learning.html
M J
M J 2020-10-22
编辑：M J 2020-10-22
Hi there, my transforming function actually takes a subset of the entire training image datastore. Can I apply it the same way as you did so that my function creates custom batches during the training process (every time it gets called)? Thanks!
Mohammad Sami
Mohammad Sami 2020-10-23
Yes. The structure of the code can potentially be the same.
drummer
drummer 2020-10-23
Great, it worked to me too.
Thanks.
I'm performing affine geometric transformation to volumetric images and MATLAB was also returning such error.
But only one thing I would like to ask to @Carlos is why you transformed all the imds?
Wasn't the purpose of your transformation to augment training data? This way, only imdsTrain would be applied the transformation...
I know there are several purposes to transform data. My question is just out of curiosity...
cheers
hello,
I am also facing the same problem, but the transform function I used is for Image contrast enhancement for brightness preservation based on dynamic streatching.Can you suggest any methods to correct it
Thank You

请先登录，再进行评论。

更多回答（0 个）

类别

在 帮助中心 和 File Exchange 中查找有关 Deep Learning Toolbox 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by