How to Realize 'Gradient Reversal Layer' ?

7 次查看（过去 30 天）
Percy Hu
Percy Hu 2021-6-16
评论： Percy Hu 2021-6-25
How can i complete a 'Gradient Reversal Layer' in matlab like in pytorch or tensorflow?
It is normally used in transfer learning network when a GAN-like loss is adopted.
Could i realize it by define a custom layer?
It is very grateful if you can offer an example of some detailed advice. Thank you for your help.
  2 个评论
Percy Hu
Percy Hu 2021-6-16
编辑：Percy Hu 2021-6-16
The gradient reversal layer has no parameters associated with it. During the forward propagation, the GRL acts as an identity transformation. During the backpropagation however, the GRL takes the gradient from the subsequent level and changes its sign, i.e., multiplies it by -1, before passing it to the preceding layer. Implementing such a layer using existing object-oriented packages for deep learning is simple, requiring only to dene procedures for the forward propagation (identity transformation), and backpropagation (multiplying by -1). The layer requires no parameter update.

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Philip Brown
Philip Brown 2021-6-21
It looks like you should be able to do this by writing your own custom layer. See the "Intermediate Layer Template" for some code to get started.
There's a custom layer used in a visualization example which does something a little bit similar (to modify the behavior of a ReLU gradient), here.
I think the custom layer code you need looks something like this:
classdef GradientReversalLayer < nnet.layer.Layer
methods
function Z = predict(layer, X)
Z = X; % Identity
end
function dLdX = backward(layer, X, dLdZ)
dLdX = -dLdZ; % Reverse gradient
end
end
end
If you want to define the constant you multiple the gradient by, you could make it a property of the custom layer and include that in your backward function.
  1 个评论
Percy Hu
Percy Hu 2021-6-25
thank you very much for your help. It is really kind of you for your detailed explanation. and i hope your answer will help more people.

请先登录，再进行评论。

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Image Data Workflows 的更多信息

标签

产品


版本

R2021a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by