Multiple softmax vectors in output layer of neural network using softmaxLayer

7 次查看（过去 30 天）
Isuru Rathnayaka
Isuru Rathnayaka 2021-12-12
评论： Christian Holz 2023-6-14
I'm using deep learning toolbox in MATLAB 2021a. And the neural network that I'm trying to build has multiple softmax vectors in output layer. (e.g. 10 softmax vectors of length 8). That is, the calculation is similar to how in-built softmax() function applies to each column of a matrix.
e.g.
>> a = randn(2,2)
a =
-1.1803 0.2963
1.6926 -0.1352
>> softmax(a)
ans =
0.0535 0.6062
0.9465 0.3938
However, I couldn't find a way to do this with softmaxLayer.
My code looks like this.
layersDNN = [
featureInputLayer(numInputs, 'Name', 'in')
fullyConnectedLayer(numInputs*2, 'Name', 'fc1')
batchNormalizationLayer('Name', 'bn1')
reluLayer('Name', 'relu1')
fullyConnectedLayer(numInputs*8, 'Name', 'fc2')
softmaxLayer('Name', 'sm1')
];
I'm trying to get the softmaxLayer to divide numInputs*8 nodes in last layer to numInputs vectors of length 8 and apply softmax function separately.
Alternatively I'm trying to remove softmaxLayer and apply softmax to reshaped output of network. Something like this.
lgraphDNN = layerGraph(layersDNN);
dlnetDNN = dlnetwork(lgraphDNN);
out1 = forward(dlnetDNN, X);
out2 = reshape(out1, [numInputs, 8]);
pred = softmax(out2);
% calculate loss, gradients etc.
I'm not sure if this is a good solution. I'd like to know if there's a way to do this using softmaxLayer, since the requirement doesn't feel like an extreme case.
  2 个评论
Isuru Rathnayaka
Isuru Rathnayaka 2021-12-12
编辑：Isuru Rathnayaka 2021-12-12
Hi Abolfazl, I'd like to know if this is doable using softmaxLayer. My idea seems like a bit of a hack and the requirement doesn't feel strange enough to resort to a hack. I rephrased the end of the question to make my question clear.
As for the alternative, it doesn't give any errors. However, I couldn't verify if it works yet because of another issue in my code. I'm not very familiar with Matlab and deep learning toolbox.

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Prachi Kulkarni
Prachi Kulkarni 2022-1-10
编辑：Prachi Kulkarni 2022-1-12
Hi,
From the R2021b release onwards, you can create numInputs number of fully connected layers, each with output size 8. Every fully connected layer can then be connected to its own softmax layer.
The outputs from the softmax layers can be concatenated using a concatenation layer and then passed on to the output layer.
For more information, see the documentation for Concatenation layer.
  2 个评论
Isuru Rathnayaka
Isuru Rathnayaka 2022-1-12
Hi Prachi,
Thanks very much for the detailed answer. I didn't know about the concatenation layer. I'll try this method.
Christian Holz
Christian Holz 2023-6-14
Hello all,
a question regarding the proposed solution: were you able to implement it? Can you show the details?
Thank you

请先登录，再进行评论。

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Image Data Workflows 的更多信息

标签

产品


版本

R2021a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by