I would like to training a network using both CNN-LSTM Network ? Is this possible in Matlab

Question

Manoj Devaraju 2022-5-21

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1724420-i-would-like-to-training-a-network-using-both-cnn-lstm-network-is-this-possible-in-matlab

评论： Ben 2023-6-23

I have a image data and I use imageInputLayer as a input for the 2D Conv layer then I would like to use LSTM network. Is this possible to use in Matlab. Like the architecture below picture (found in some research paper image on Google). I have tried using this but unfortunatly not sucessfull. Can you please give some ideas how can we implement this.

layers = [ ...
    %CNN
    imageInputLayer([129 35 1])
    sequenceInputLayer(inputSize,'Name','input')
    
    convolution2dLayer(3,32,'Padding','same')
    batchNormalizationLayer
    reluLayer
    maxPooling2dLayer(2,'Stride',2)
    convolution2dLayer(3,32,'Padding','same')
    batchNormalizationLayer
    reluLayer
    maxPooling2dLayer(2,'Stride',2)
    convolution2dLayer(3,64,'Padding','same')
    batchNormalizationLayer
    reluLayer
    maxPooling2dLayer(2,'Stride',2)
    
    flattenLayer('Name','flatten')    
    %LSTM
    lstmLayer(numHiddenUnits,'OutputMode','last','Name','lstm')    
    fullyConnectedLayer(numClasses, 'Name','fc')
    softmaxLayer('Name','softmax')
    classificationLayer('Name','classification')];

2 个评论
显示无隐藏无

Ullah Nadeem 2023-6-23

The reply is too late though but it may help for next search.

My problem has been resolved by putting sequenceFoldingLayer right after imageInputLayer and sequenceUnfoldingLayer before flatten layer.

I think, it is because LSTM layers need sequential information to keep long-range dependencies.

Cheers~

Ben 2023-6-23

@Ullah Nadeem - thanks for replying, you're right that you need sequenceFoldingLayer and sequenceUnfoldingLayer when using trainNetwork for CNN-LSTM networks. We have this example that shows training an LSTM on CNN embeddings of video frames, the final network combines the CNN and LSTM for prediction using the sequence folding layers. We also have this example demonstrating training a CNN-LSTM on audio data.

Note that you need a sequenceInputLayer to input sequences of images into the CNN-LSTM network.

Also note that you do not need sequenceFoldingLayer or sequenceUnfoldingLayer when using convolution2dLayer in dlnetwork with sequences of images - by default the convolution2dLayer in dlnetwork will "distribute" over the sequence dimension on sequences of images. To train the dlnetwork you will need to use a custom training loop.

请先登录，再进行评论。

请先登录，再回答此问题。

I would like to training a network using both CNN-LSTM Network ? Is this possible in Matlab

2 个评论
显示无隐藏无

回答（0 个）

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

I would like to training a network using both CNN-LSTM Network ? Is this possible in Matlab

2 个评论 显示 无隐藏 无

回答（0 个）

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

2 个评论
显示无隐藏无