Error in using trainNetwork with training data and labels

29 次查看（过去 30 天）
Ben Hur
Ben Hur 2017-11-20
评论： Hamza 2023-5-17
I am trying to classify image datasets using deep learning.
after getting feature vector of each single image I ve got a matrix 18000x24000 which indicates to No. of images x features.
I used:
trainNetwork (X, Y, Layers, Options)
Where X is the train data and Y is the Labels which is 18000x1. But there is an error says Invalid training data. X and Y must have the same number of observations.
I think I should change the train matrix to 4-D but I don't know how, and if it is correct?
  3 个评论
显示 1更早的评论
Ali Yar Khan
Ali Yar Khan 2019-12-6
编辑：Ali Yar Khan 2019-12-6
i have row of feature in data and one distinct value in label for each row of data ... my network is also giving me the same error ... here is my code.
%loading train and test data
trainingData = load('train.mat');
trainData = trainingData.data;
trainLabels = categorical(trainingData.labels);
testingData = load('test.mat');
testData = testingData.data;
testLabels = testingData.labels;
% Define Network Architecture
% Define the convolutional neural network architecture.
layers = [
imageInputLayer([1 144 1]) % 22X1X1 refers to number of features per sample
convolution2dLayer(3,16,'Padding','same')
reluLayer
fullyConnectedLayer(384) % 384 refers to number of neurons in next FC hidden layer
fullyConnectedLayer(384) % 384 refers to number of neurons in next FC hidden layer
fullyConnectedLayer(2) % 2 refers to number of neurons in next output layer (number of output classes)
softmaxLayer
classificationLayer];
options = trainingOptions('sgdm',...
'MaxEpochs',500, ...
'Verbose',false,...
'Plots','training-progress');
%train the network
net = trainNetwork(trainData,trainLabels,layers,options);
%test the network
predictedLabels = classify(net,trainD)'
accuracy = sum(predictedLabels == testLabels)/numel(testLabels)
It gives me this error
Invalid training data. X and Y must have the same number of observations.
Error in convnet1d (line 27)
net = trainNetwork(trainData,trainLabels.',layers,options);
Caused by:
Error using nnet.internal.cnn.util.NetworkDataValidator/assertXAndYHaveSameNumberOfObservations (line 142)
Invalid training data. X and Y must have the same number of observations.
Any help regarding this . I have this work space ... that may help you about the dimensions of dataScreenshot (13).png

请先登录，再进行评论。

请先登录，再回答此问题。

回答（4 个）

Bhartendu
Bhartendu 2018-4-8
Try the following:
If your data ( 18000 data points) is in form of images with dimensions say, 120*200 (equals to 24000), then reshape it as follows:
X_train = reshape(X, [120,200,1,size(X,1)]);
This should create 4-D Matrix X_train of size (120,200,1,18000), then train the network using:
net = trainNetwork(X_train,Y,Layers,Options)
  1 个评论
Radians
Radians 2020-2-5
I have done exactly that but to no avail.
I Capture.PNG
where 'X_1' is my X and 'correspondingvalues' is my Y. basically, x is a 16*9500 items collection of 16*16 images. Y is their labels.
Please help.

请先登录，再进行评论。

US
US 2020-7-26
编辑：US 2020-7-26
I am also getting the same error (my layers structure is a prediction CNN):
>> whos X Y
Name Size Bytes Class Attributes
X 64x64x52x1800 3067084800 double
Y 1800x1 14400 double
>> MyPrediction = trainNetwork(X, Y, layers, opts);
Error using trainNetwork (line 165)
Number of observations in X and Y disagree.
It gives me the same error when I transpose Y, or when I reshape X to [64*64,52,1,1800].
And I have:
MATLAB Version: 9.6.0.1072779 (R2019a)
I am starting to think that this MATLAB function trainNetwork or a sub-function that it calls is somehow not behaving properly or I am missing something.
Any help will be appreciated.
michael scheinfeild
hi i have similar issue and cant solve it i also look at the example Sequence Classification Using Deep Learning
i try to classify between different signals based frequency future
%%generete the signal
fs=500;
f=20;
t=[0:1/fs:5-1/fs];
xsig=10*sin(2*pi*f*t);
sampleLen=length(xsig);
NFFT = 2.^nextpow2(sampleLen);
% signal fft
hwin=hamming(length(xsig))';
Y = fft(xsig.*hwin,NFFT);
magnitudeY = abs(Y);
xsampleFFT=20*log10(magnitudeY(1:NFFT/2));
figure,plot(xsampleFFT);title('signal fft')
% noise fft
xnoise= randn(size(xsig));
Yn = fft(xnoise.*hwin,NFFT);
magnitudeYn = abs(Yn);
xnoiseFFT=20*log10(magnitudeYn(1:NFFT/2));
figure,plot(xnoiseFFT);title('noise')
%%make the data
sampleLen = 100;
NFFT = 128;
hwin=hamming(sampleLen)';
nsamples=length(xsig)/sampleLen;
xsampleFFT={};%zeros(nsamples,NFFT/2);
hwin=hamming(sampleLen)';
kj=1;
for(k=1:sampleLen:length(xsig)-sampleLen+1)
cursig=xsig(k:k+sampleLen-1);
Y = fft(cursig.*hwin,NFFT);
magnitudeY = abs(Y); % Magnitude of the FFT
xTrain{kj}=20*log10(magnitudeY(1:NFFT/2));
%figure,plot(xTrain{kj})
yTrain(kj)=categorical(1);
kj=kj+1;
end
disp('data types')
[size(xTrain) size(yTrain)]
[size(xTrain{1})]
class(xTrain)
class(yTrain)
%append
for(k=1:sampleLen:length(xnoise)-sampleLen+1)
cursig=xnoise(k:k+sampleLen-1);
Y = fft(cursig.*hwin,NFFT);
magnitudeY = abs(Y); % Magnitude of the FFT
xTrain{kj}=20*log10(magnitudeY(1:NFFT/2));
%figure,plot(xTrain{kj})
yTrain(kj)=categorical(0);
kj=kj+1;
end
disp('data types')
[size(xTrain) size(yTrain)]
[size(xTrain{1})]
class(xTrain)
class(yTrain)
figure,plot(yTrain)
figure,subplot(2,1,1),plot(xTrain{10})
subplot(2,1,2);plot(xTrain{30})
function [net] = train_lstm(XTrainLoc,YTrainLoc)
%%lstm
inputSize = 1;
numHiddenUnits = 100;
numClasses = 2;
layers = [ ...
sequenceInputLayer(inputSize)
lstmLayer(numHiddenUnits,'OutputMode','sequence')
fullyConnectedLayer(numClasses)
softmaxLayer
classificationLayer];
maxEpochs = 1;
miniBatchSize = 100;
options = trainingOptions('adam', ...
'ExecutionEnvironment','cpu', ...
'GradientThreshold',1, ...
'MaxEpochs',maxEpochs, ...
'MiniBatchSize',miniBatchSize, ...
'SequenceLength','longest', ...
'Shuffle','never', ...
'Verbose',0, ...
'Plots','training-progress');
% options = trainingOptions('adam', ...
% 'ExecutionEnvironment','cpu', ...
% 'GradientThreshold',1, ...
% 'MaxEpochs',maxEpochs, ...
% 'MiniBatchSize',miniBatchSize, ...
% 'SequenceLength','longest', ...
% 'Shuffle','never', ...
% 'Verbose',0, ...
% 'Plots','training-progress',...
% 'ValidationData',{XValidation,YValidation},...
% 'ValidationPatience',Inf);
%%train
net = trainNetwork(XTrainLoc,YTrainLoc,layers,options);
====== i recive error
[net] = train_lstm(xTrain,yTrain)
*Error using trainNetwork (line 154)
Invalid training data. If all recurrent layers have output
mode 'sequence', then the responses must be a cell array of
categorical sequences, or a categorical sequence.*
_Error in train_lstm (line 42) net = trainNetwork(XTrainLoc,YTrainLoc,layers,options);
Caused by: Error using nnet.internal.cnn.util.NetworkDataValidator/assertOutputModeCorrespondsToDataForClassification (line 380) Invalid training data. If all recurrent layers have output mode 'sequence', then the responses must be a cell array of categorical sequences, or a categorical sequence._
so what is the issue i tried also change y to cell array of category , transpose the internal x, change network in. i think in this fft i have actually one sample each time with nfft feature. this is the same as the Japanese sample but they have 12 features
SARAH LONER
SARAH LONER 2019-11-29
sir i also have the same issue cant able to train network for an image .
my work was image segmentation based on unet segment cant able to clear the code
it showing error at
net = trainNetwork(imds,layers,options);
where in imds i have taken image
kindly help to solve this error
  2 个评论
SARAH LONER
SARAH LONER 2019-11-30
inp=b % my input image is dicom image
DatasetPath=fullfile('C:\Users\Desktop\to');
imds=imageDatastore(DatasetPath, 'IncludeSubfolders', true,...
'LabelSource','foldernames','fileextension',{'.dcm'});
labelDir = fullfile(DatasetPath,'testImages');
I = readimage(imds,1);
I = histeq(I);
imshow(I)
classes = [
"MALIGNANT","BENIGN"
];
labelIDs=[255 0]
inputlayer = imageInputLayer([512 512 1],'Name','inp')
numFilters = 64;
numLayers = 16;
layers = [ ...
imageInputLayer([512 512 1])
convolution2dLayer(5,20)
batchNormalizationLayer
reluLayer
maxPooling2dLayer(2,'Stride',2)
convolution2dLayer(5,20)
batchNormalizationLayer
reluLayer
maxPooling2dLayer(2,'Stride',2)
transposedConv2dLayer(4,numFilters,'Stride',2,'Cropping',1);
convolution2dLayer(5,20)
batchNormalizationLayer
reluLayer
transposedConv2dLayer(4,numFilters,'Stride',2,'Cropping',1);
convolution2dLayer(5,20)
batchNormalizationLayer
reluLayer
convolution2dLayer(5,20)
fullyConnectedLayer(4)
softmaxLayer
pixelClassificationLayer
]
pxds = pixelLabelDatastore(labelDir,classes,labelIDs);
options = trainingOptions('sgdm', ...
'InitialLearnRate',0.01, ...
'MaxEpochs',1, ...
'Shuffle','every-epoch', ...
'ValidationFrequency',30, ...
'Verbose',false);
ds = pixelImageDatastore(imds,pxds);
net=trainNetwork(ds,layers,options);
I = read(imds);
C = read(pxds)
C = semanticseg(I, net);
% Overlay pixel label data on the image and display.
B = labeloverlay(I, C);
figure(12)
imshow(B)
i got error at
ds = pixelImageDatastore(imds,pxds);
once i cleared that error means again i got error at
net=trainNetwork(ds,layers,options);

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Image Data Workflows 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by