How to implement in Matlab Deep Learning PyTorch detach or TensorFlow stop_gradient?

Question

John Smith 2021-8-19

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1436644-how-to-implement-in-matlab-deep-learning-pytorch-detach-or-tensorflow-stop_gradient

回答： Damien T 2022-11-7

Hello!

Pytorch has a facility to detach a tensor so that it will never require a gradient, i.e. (from here):

In order to enable automatic differentiation, PyTorch keeps track of all operations involving tensors for which the gradient may need to be computed (i.e., require_grad is True). The operations are recorded as a directed graph. The detach() method constructs a new view on a tensor which is declared not to need gradients, i.e., it is to be excluded from further tracking of operations, and therefore the subgraph involving this view is not recorded.

TensorFlow has a comparable facility stop_gradient.

These are useful when one needs copies of expressions that are treated as constants and whose gradient should not be calculated during learning.

How does one implement such a thing in Matlab's Deep Learning Toolbox? (Possibly in a custom training loop)

Thx,

D

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Artem Lensky 2022-7-14

Are there any updates on implementing detach in Matlab?

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Damien T 2022-11-7

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1436644-how-to-implement-in-matlab-deep-learning-pytorch-detach-or-tensorflow-stop_gradient#answer_1092928

在 MATLAB Online 中打开

You just need to call the following. It will turn a traced dlarray into a standard Matlab variable, hence the autodiff engine will treat the new ("detached") variable as a constant.

myDetachedVar = extractdata(myVar);