InputSize definition for sequenceInputLayer for multidimensional time sequences

12 次查看（过去 30 天）
Jacopo Biasetti
Jacopo Biasetti 2024-7-17
回答： Jacopo Biasetti 2024-7-18
Hi,
I am trying to input data to a LSTM network using a sequenceInputLayer. The input data is a cell array with dimensions 6766x1, in which, each cell is a 3D tensor of dimensions 1024 x numOfTimeSteps x 2. Each rows x depth slice is a time step, and the numOfTimeSteps is variable between each cell. It is not clear how the property InputSize should be set in this case, as I am dealing with a vector sequence but one that is multidimensional and of variable length between cells.
Any thoughts?
Thanks,
Jacopo

请先登录，再回答此问题。

采纳的回答

Paras Gupta
Paras Gupta 2024-7-18
Hi Jacopo,
I understand that you are trying to input data to an LSTM network using a "sequenceInputLayer". Your input data is a cell array with dimensions 6766x1, where each cell is a 3D tensor of dimensions 1024 x numOfTimeSteps x 2.
For the "sequenceInputLayer", the "InputSize" should be set to the size of each time step. In your case, each time step is a 2D slice of size [1024, 2], which will be the input size for the "sequenceInputLayer". Additionally, while training and inference, you need to consider the 'STCB' (Sequence, Time, Channel, Batch) data format for input data.
The following example code illustrates the use of "sequenceInputLayer" for your use-case:
% Define parameters
numSequences = 6766;
maxNumTimeSteps = 50;
inputSize = [1024, 2];
numHiddenUnits = 100;
numResponses = inputSize(2);
% Generate dummy data
data = cell(numSequences, 1);
taregt = cell(numSequences, 1);
for i = 1:numSequences
numTimeSteps = randi([1, maxNumTimeSteps]); % Random number of time steps
data{i} = randn(inputSize(1), numTimeSteps, inputSize(2)); % Random 3D tensor
target{i} = rand(numResponses, numTimeSteps) % random target data
end
% Define the network
layers = [
sequenceInputLayer(inputSize)
lstmLayer(128)
fullyConnectedLayer(numResponses)];
% Define training options
options = trainingOptions("adam", ...
InputDataFormats="STCB",...
TargetDataFormats="CTB",...
MaxEpochs=1, ...
Plots="training-progress");
% Train the network
net = trainnet(data, target, layers, 'mse', options);
% Pass input and make prediction
pred = predict(net, ones(1024, 40, 2), InputDataFormats="STCB", OutputDataFormats="CTB");
Please refer to the following documentation links for more information on the functions and data formats use in the code above:
Hope this helps you move forward with your work.

更多回答（1 个）

Jacopo Biasetti
Jacopo Biasetti 2024-7-18
Hi Paras,
Many thanks for your reply.
I am following the example https://se.mathworks.com/help/deeplearning/ug/classify-videos-using-deep-learning-with-custom-training-loop.html with just the input data being modified to be 3D tensors and not 2D tensors as explained in my first message. I modified the input data format as you suggested, but despite that, when I run the code, I get the following error:
Error using dlnetwork/forward (line 580)
Layer 'sequence': Invalid input data. Invalid size of channel dimension. Layer expects
input with channel dimension size 2 but received input with size 392.
Error in classify_video_sequences_full_network_v1>modelGradients (line 472)
[dlYPred,state] = forward(dlnet,dlX);
Error in deep.internal.dlfeval (line 17)
[varargout{1:nargout}] = fun(x{:});
Error in deep.internal.dlfevalWithNestingCheck (line 19)
[varargout{1:nargout}] = deep.internal.dlfeval(fun,varargin{:});
Error in dlfeval (line 31)
[varargout{1:nargout}] = deep.internal.dlfevalWithNestingCheck(fun,varargin{:});
Error in classify_video_sequences_full_network_v1 (line 207)
[gradients,state,loss] = dlfeval(@modelGradients,dlnet,dlX,dlY);
The whole training code is exactly the same as in the tutorial I linked above, and reported below (only the relevant portions):
layersLSTM = [
sequenceInputLayer([1024 2],'Name','sequence')
bilstmLayer(2000,'OutputMode','last','Name','bilstm')
dropoutLayer(0.5,'Name','drop')
fullyConnectedLayer(numClasses,'Name','fc')
softmaxLayer('Name','softmax')
];
dlnet = dlnetwork(layersLSTM); %#ok<*NASGU>
numEpochs = 15;
miniBatchSize = 128;
initialLearnRate = 1e-4;
decay = 0.001;
gradDecay = 0.9;
sqGradDecay = 0.999;
plots = "training-progress";
mbq = minibatchqueue(dsTrain,...
'MiniBatchSize',miniBatchSize,...
'MiniBatchFcn', @preprocessLabeledSequences,...
'MiniBatchFormat',{'SCTB',''});
if plots == "training-progress"
figure
lineLossTrain = animatedline('Color',[0.85 0.325 0.098]);
ylim([0 inf])
xlabel("Iteration")
ylabel("Loss")
grid on
end
averageGrad = [];
averageSqGrad = [];
iteration = 0;
start = tic;
% Loop over epochs.
for epoch = 1:numEpochs
% Shuffle data.
shuffle(mbq);
% Loop over mini-batches.
while hasdata(mbq)
iteration = iteration + 1;
% Read mini-batch of data.
[dlX, dlY] = next(mbq);
% Evaluate the model gradients, state, and loss using dlfeval and the
% modelGradients function.
[gradients,state,loss] = dlfeval(@modelGradients,dlnet,dlX,dlY);
% Determine learning rate for time-based decay learning rate schedule.
learnRate = initialLearnRate/(1 + decay*iteration);
% Update the network parameters using the Adam optimizer.
[dlnet,averageGrad,averageSqGrad] = adamupdate(dlnet,gradients,averageGrad,averageSqGrad, ...
iteration,learnRate,gradDecay,sqGradDecay);
% Display the training progress.
if plots == "training-progress"
D = duration(0,0,toc(start),'Format','hh:mm:ss');
addpoints(lineLossTrain,iteration,double(gather(extractdata(loss))))
title("Epoch: " + epoch + " of " + numEpochs + ", Elapsed: " + string(D))
drawnow
end
end
end
function [gradients,state,loss] = modelGradients(dlnet,dlX,Y)
[dlYPred,state] = forward(dlnet,dlX);
loss = crossentropy(dlYPred,Y);
gradients = dlgradient(loss,dlnet.Learnables);
end
Hope you can provide an answer as to why this error pops us.
Many thanks,
Jacopo

类别

在 Help Center 和 File Exchange 中查找有关 Sequence and Numeric Feature Data Workflows 的更多信息

标签

产品


版本

R2024a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by