Update a parameter which is not learnable in Custom Layers Deep Learning

4 次查看（过去 30 天）
Mathieu Chêne
Mathieu Chêne 2022-1-14
评论： Mathieu Chêne 2022-1-31
Hello,
I am working on a deep learning project in which I use a custom layer. In this layer, I have a parameter, α, that depend of the weight. I want to update when the weight is updated thus, for me, α is not a learnable parameter. Here is my predict function :
function [Z] = predict(layer, X)
% Z = predict(layer, X1, ..., Xn) forwards the input data X1,
% ..., Xn through the layer and outputs the result Z.
B=layer.Bias;
W = layer.Weights;
numel=size(X,2);
% Initialize output
Z = zeros(layer.OutputSize,numel,"single");
%alpha coef calculation
e=zeros(1,layer.InputSize);
numel=size(X,2);
for j= 1:size(layer.Graphe.neighbors(layer.TargetNode))
for i=1:numel
e(:,j)=mean(e(:,j)+W(:,layer.TargetIndex)*X(layer.TargetIndex,i)+W(:,j)*X(j,i),1);
end
end
e=leakyRELU(e,0.2);
A=softmax(e');
A=A';
layer.Alpha=A;
% Weighted addition
Z=(A.*W)*X+B;
end
Alpha is declared as a parameter here
properties
InputSize
OutputSize
TargetNode
Graphe
TargetIndex
Alpha
end
However when my training ends net.layer(3,1).Alpha gives me the initial value of α and it is the same thing in the backward function.
How can I do to update α ?
Thank you in advance for your futur help.
Mathieu

请先登录，再回答此问题。

采纳的回答

Katja Mogalle
Katja Mogalle 2022-1-21
The technical and not too helpful answer is that custom layers are not handle classes and the predict function doesn't return the modified layer object so the framework doesn't/can't get the updated layer.
The question is, what do you plan to do with Alpha? From the sounds of it, you want to look at it after training? Or during training? Is it used anywhere in the training process?
One possible solution would be to declare a second output on the custom layer (look for NumOutputs and OutputNames properties in this custom layer doc page) which returns the alpha value. If you use dlnetwork and custom training loops, you can easily get the second (Alpha) output at any time during or after training without actually using it in the training process.
I hope this helps you move forward with your project.
  2 个评论
Mathieu Chêne
Mathieu Chêne 2022-1-24
Hello,
Thank you for your answer. Yes the idea here is to see the final value of this coefficient. I will try to declare a second Output because I don't use custom training loops yet.
Mathieu Chêne
Mathieu Chêne 2022-1-31
Hello,
As you adviced me to do ,I declared alpha as an output. I used a regression Layer as output and now I can see its value.
Thank you for your help
Mathieu

请先登录，再进行评论。

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Build Deep Neural Networks 的更多信息

标签

产品


版本

R2021b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by