How to prepare the training data for neural net with concatenationLayer, which accepts the combination of sequence inputs and normal inputs?

13 次查看（过去 30 天）
Liangwu Yan
Liangwu Yan 2023-1-11
回答： Ben 2023-3-16
I looked into the example code for Vector-Sequence and Feature Classification Network in MATLAB example: Example Deep Learning Networks Architectures. The URL is here https://www.mathworks.com/help/deeplearning/ug/example-deep-learning-networks.html
In the visualization of the neural net structure, there are two input sources and a concatenationLayer in the middle, as below
On the top left is the sequence input, suppose I have two channels in the sequence input with the same fixed length (100). And on the top right for the input layer, suppose I would like to feed a vector of size 5. And I have 1000 training points. The output is a scaler. In other words, each output is associated with a sequence input (2X100) and a vector of size 5.
Given the trainNetwork(XTrain,YTrain,layers,options), with layers as the neural net layer, in what order should I arrange the data in XTrain to make sure the sequence input and the vector input are correctly identified and used for training? I have done some feedfoward neural net training and I know that the number of column means the number of training data, given XTrain,YTrain are 2D matrices.
But things will get complex when the sequence input has more than one channels, since from other posts, I notice that the sequence training data are arranged in cell array of 2D matrices, e.g., see MATLAB Example: Sequence Classification Using Deep Learning. The URL is here: https://www.mathworks.com/help/deeplearning/ug/classify-sequence-data-using-lstm-networks.html.
So when I have both sequence input and the normal vector input, what form will my XTrain be?
I clicked the Example tab and could not find a relevant usage of training this kind of neural net. So I wrote a post here.
Answers and relevant links are greatly appreciate it, :)!
  1 个评论
Liangwu Yan
Liangwu Yan 2023-1-11
I think I found a relevant MATLAB example (Train Network on Image and Feature Data) which could help me. The URL is here: https://www.mathworks.com/help/deeplearning/ug/train-network-on-image-and-feature-data.html
In the example, the training data are converted into datastore Type via arrayDatastore and then combined into dsTrain, as seen in the picture below
Seems like the sequence of the combined data is the same as the input required by the neural net, as seen below
dsTrain = combine(dsX1Train,dsX2Train,dsTTrain);
dsX1Train(ImageInput), dsX2Train(rotation angle), dsTTrain(output).
Am I correct?
However, an answer from an experienced user or Mathworker would help a lot, :D.

请先登录，再进行评论。

请先登录，再回答此问题。

回答（1 个）

Ben
Ben 2023-3-16
You are right that to use trainNetwork with a network that has multiple inputs you will need to use a datastore. There is documentation for this here https://uk.mathworks.com/help/deeplearning/ug/multiple-input-and-multiple-output-networks.html which also links to the example you have found for image and feature data. Sequence data can be used as an additional input so long as the datastore reads it out in our expected format - a cell array where each entry is one sequence with the sequence dimension last.
You can also use dlnetwork and a custom training loop, in which case you don't necessarily need datastores as you can directly call forward(net, x1, x2) in your training loop, where net is a dlnetwork, and x1,x2 are the two input data. However this may require more effort if you haven't used custom training loops before.

类别

在 Help Center 和 File Exchange 中查找有关 Image Data Workflows 的更多信息

标签

产品


版本

R2022b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by