How can I train multi-input deep network without DataStore

Question

Y. K. 2020-4-30

1
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/522086-how-can-i-train-multi-input-deep-network-without-datastore

评论：鑫陈 2022-5-25

I want to build two inputs, one output network.

But the first input is an image and the second input is a vector.

When I try to train the network with cell array including two sub arrays (one for images, one for vector), I got an error.

"Invalid training data for multiple-input network. For multiple-input training, use a single datastore."

I created 4D image array, a vector array for each input and labels array for training.

How can I combine these data to a DataStore.

Matlab Datastore couldn't get the data from defined variable from workspace.

2 个评论
显示无隐藏无

Y. K. 2020-4-30

options = trainingOptions('sgdm', ...

'MaxEpochs',50,...

'MiniBatchSize',64,...

'ValidationFrequency',30, ...

'InitialLearnRate',1e-3, ...

'Verbose',true,...

'Plots','training-progress');

net=trainNetwork({X_train_V, X_trainIMG}, Y_train,lgraph,options);

%X_Train_V : 100x1x5000, X_TrainIMG: 62x62x3x5000, Y_train: 5000x1

Srivardhan Gadila 2021-10-5

From R2020b onwards we can directly use arrayDatastore function instead of saving the data to disk and loading it, as mentioned in the answer. For versions less than R2020b the answer would be the workaround.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Srivardhan Gadila 2020-5-3

5
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/522086-how-can-i-train-multi-input-deep-network-without-datastore#answer_429907

在 MATLAB Online 中打开

To define and train a deep learning network with multiple inputs, specify the network architecture using a layerGraph object and train using the trainNetwork function by specifying the multiple inputs using a combinedDatastore or transformedDatastore object.

For networks with multiple inputs, the datastore must be a combined or transformed datastore that returns a cell array with (numInputs+1) columns containing the predictors and the responses, where numInputs is the number of network inputs and numResponses is the number of responses. For i less than or equal to numInputs, the ith element of the cell array corresponds to the input layers.InputNames(i), where layers is the layer graph defining the network architecture. The last column of the cell array corresponds to the responses.

data = read(ds)
data =
  4×3 cell array
    {28×28 double}    {128×128 double}    {[7]}
    {28×28 double}    {128×128 double}    {[7]}
    {28×28 double}    {128×128 double}    {[9]}
    {28×28 double}    {128×128 double}    {[9]}

Hence in order to train Multiple input networks you have to use a datastore. Refer to Input Arguments & ds of trainNetwork and Multiple-Input and Multiple-Output Networks, Input Datastore for Training, Validation, and Inference for more information.

You can make use of the following code as workaround if your training data is saved in a .mat file:

%% Generate random training data for muliple input network
imgDat = randn([28 28 3 200]);
vectorData = randn([10 1 200]);
trainLabels = categorical(randi([0 1], 200,1));
%% Convert trianing data into cell arrays
imgCells = mat2cell(imgDat,28,28,3,ones(200,1));
vectorCells = mat2cell(vectorData,10,1,ones(200,1));
imgCells = reshape(imgCells,[200 1 1]);
vectorCells = reshape(vectorCells,[200 1 1]);
labelCells = arrayfun(@(x)x,trainLabels,'UniformOutput',false);
combinedCells = [imgCells vectorCells labelCells];
%% Save the converted data so that it can be loaded by filedatastore
save('traingData.mat','combinedCells');
filedatastore = fileDatastore('traingdata.mat','ReadFcn',@load);
trainingDatastore = transform(filedatastore,@rearrangeData);
%% Define muliple input network
layers1 = [
    imageInputLayer([28 28 3],'Name','input')  
    convolution2dLayer(3,16,'Padding','same','Name','conv_1')
    batchNormalizationLayer('Name','BN_1')
    reluLayer('Name','relu_1')
    fullyConnectedLayer(10,'Name','fc11')
    additionLayer(2,'Name','add')
    fullyConnectedLayer(2,'Name','fc12')
    softmaxLayer('Name','softmax')
    classificationLayer('Name','classOutput')];
lgraph = layerGraph(layers1);
layers2 = [imageInputLayer([10 1],'Name','vinput')
    fullyConnectedLayer(10,'Name','fc21')];
lgraph = addLayers(lgraph,layers2);
lgraph = connectLayers(lgraph,'fc21','add/in2');
plot(lgraph)
%% Define trainingOptions and also set 'Shuffle' to 'never' for this workaround to work
options = trainingOptions('adam', ...
    'InitialLearnRate',0.005, ...
    'LearnRateSchedule','piecewise',...
    'MaxEpochs',300, ...
    'MiniBatchSize',1024, ...
    'Verbose',1, ...
    'Plots','training-progress',...
    'Shuffle','never');
net = trainNetwork(trainingDatastore,lgraph,options);
%% function to be used to transform the filedatastore 
%to ensure the read(datastore) returns M-by-3 cell array ie., (numInputs+1) columns
function out = rearrangeData(ds)
out = ds.combinedCells;
end

7 个评论
显示 5更早的评论隐藏 5更早的评论

zhushaolong 2022-3-11

I have the same data,

I want to build two inputs, one output network.

the first input is an image(224 224 3 768) and the second input is a vector(46 768).

But my purpose is regression analysis.

How can I combine these data to a DataStore.

san su 2022-3-17

If the multi-input data can be oganized as [input1, input2, ..., inputN], it would be an easier way.

请先登录，再进行评论。

Answer 2

David Willingham 2022-3-11

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/522086-how-can-i-train-multi-input-deep-network-without-datastore#answer_915329

Please see this example, released in R2022a that shows how train a multi-input network. It still uses datastores, but shows how they can be combined easily.

3 个评论
显示 1更早的评论隐藏 1更早的评论

zhushaolong 2022-3-13

在 MATLAB Online 中打开

dsX1Train = arrayDatastore(X1Train,IterationDimension=4);
dsX2Train = arrayDatastore(X2Train);
dsTTrain = arrayDatastore(TTrain);
dsTrain = combine(dsX1Train,dsX2Train,dsTTrain);
%% 
lgraph = layerGraph();
tempLayers = [
    imageInputLayer([224 224 3],"Name","imageinput_1")
    convolution2dLayer([3 3],8,"Name","conv_1","Padding","same")
    batchNormalizationLayer("Name","batchnorm_1")
    reluLayer("Name","relu_1")
    averagePooling2dLayer([2 2],"Name","avgpool2d_1","Stride",[2 2])
    convolution2dLayer([3 3],16,"Name","conv_2","Padding","same")
    batchNormalizationLayer("Name","batchnorm_2")
    reluLayer("Name","relu_2")
    averagePooling2dLayer([2 2],"Name","avgpool2d_2","Stride",[2 2])
    convolution2dLayer([3 3],32,"Name","conv_3","Padding","same")
    batchNormalizationLayer("Name","batchnorm_3")
    reluLayer("Name","relu_3")
    convolution2dLayer([3 3],32,"Name","conv_4","Padding","same")
    batchNormalizationLayer("Name","batchnorm_4")
    reluLayer("Name","relu_4")
    dropoutLayer(0.2,"Name","dropout")
    fullyConnectedLayer(1,"Name","fc_1")];
lgraph = addLayers(lgraph,tempLayers);
tempLayers = [
    imageInputLayer([1 46 1],"Name","imageinput_2")
    fullyConnectedLayer(1,"Name","fc_2")];
lgraph = addLayers(lgraph,tempLayers);
tempLayers = [
    concatenationLayer(2,2,"Name","concat")
    fullyConnectedLayer(1,"Name","fc_3")
    regressionLayer("Name","regressionoutput")];
lgraph = addLayers(lgraph,tempLayers);
clear tempLayers;
lgraph = connectLayers(lgraph,"fc_2","concat/in1");
lgraph = connectLayers(lgraph,"fc_1","concat/in2");
%% 
options = trainingOptions("sgdm", ...
    MaxEpochs=15, ...
    InitialLearnRate=0.001, ...
    Plots="training-progress", ...
    Verbose=0);
net = trainNetwork(dsTrain,lgraph,options);

我引用了这个例子，but,

Warning: Training stops at iteration 3 because the training loss is NaN. Predictions using the output network may contain NaN values.

san su 2022-3-17

I have 6 inputs, 1 output. The network does not work. I do not know why. "error: 无效的输入层。网络最多只能包含一个序列输入层。"

鑫陈 2022-5-25

For unsupervised multi-input neural networks, without labels,I do not know how to define the labels in the third column and how to generate training data with combine, such as the image regression problem of two image inputs？

请先登录，再进行评论。