Is there a way to extract partial derivatives of specific layers in deep learning toolbox?

Question

SC 2020-11-10

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/642535-is-there-a-way-to-extract-partial-derivatives-of-specific-layers-in-deep-learning-toolbox

评论： SC 2020-11-21

Hi,

I asked this question last year, in which I would like to know if it is possible to extract partial derivatives

involved in back propagation, for the parameters

of layer

so that I can use

for other purpose. At that time, the latest MATLAB version is 2019b, and I was told in the above post that it is only possible when the final output y is a scalar, while my desired y can be a vector, matrix or even a tensor (e.g. reconstruction tasks).

Now, is it possible to extract the partial derivatives of layer

in 2020b? Thanks.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Amanpreetsingh Arora 2020-11-12

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/642535-is-there-a-way-to-extract-partial-derivatives-of-specific-layers-in-deep-learning-toolbox#answer_542413

As mentioned in the answer to the question referred by you, the only way to find partial derivatives of a tensor is by looping over elements and calling "dlgradient" as "dlgradient" only supports scalar input for auto differentiation. However, I understand your concern that this will waste time recomputing overlapping traces. In order to improve time performance, you can call "dlgradient" with 'RetainData' parameter set to 'true' which will retain the intermediate computation of gradient and any subsequent calls to the function will reuse these intermediate computations. Refer to the following documentation for more information on this.

https://in.mathworks.com/help/deeplearning/ref/dlarray.dlgradient.html#mw_10cd0d63-9a6a-461e-a2f7-4a3596f62a92

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

SC 2020-11-21

Thanks. I just submitted a new feature request. It is a useful feature for network surgery and it is available in Tensorflow as well. I hope I can use a more convenient partial derivative extraction function in Matlab in the future. Since the values are already stored in the model during the backward propagation, it shouldn't be difficult to allow the user to extract those values as there is not additional computation required.

请先登录，再进行评论。

Is there a way to extract partial derivatives of specific layers in deep learning toolbox?

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

回答（1 个）

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

Is there a way to extract partial derivatives of specific layers in deep learning toolbox?

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

回答（1 个）

1 个评论 显示 -1更早的评论隐藏 -1更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

1 个评论
显示 -1更早的评论隐藏 -1更早的评论