How to compute the derivative of the neural network?

Question

Filipe 2012-10-19

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/51319-how-to-compute-the-derivative-of-the-neural-network

评论： soo-choon kang 2021-8-14

Hi,

Once you have trained a neural network, is it possible to obtain a derivative of it? I have a neural network "net" in a structure. I would like to know if there is a routine that will provide the derivatives of net (derivative of its outputs with respect to its inputs).

It is probably not difficult, for a feedforward model, there is just matrix multiplications and sigmoid functions, but it would be nice to have a routine that will do that directly on "net".

Thanks!

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Greg Heath 2012-10-20

2
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/51319-how-to-compute-the-derivative-of-the-neural-network#answer_62612

Differentiate to obtain dyi/dxn

y = b2 + LW*h

h = tanh(b1+IW*x)

or, with tensor notation(i.e., summation over repeated indices),

yi = b2i + Lwij*hj

hj = tanh(b1j + IWjk*xk)

Now just use the chain rule.

Hope this helps.

Thank you for formally accepting my answer

Greg

2 个评论
显示无隐藏无

Tittu Mathew 2018-12-7

ANN_FOD.m

Hi Greg and Filipe,

I am approaching you based on your query years ago on getting the partial derivative of trained ANN outputs w.r.t each of the input parameters by using the MATLAB toolbox, years ago.

I am trying to represent a simple function P = f(X,Y,Z) where P is a scalar output and the input to ANN is a vector with 3 elements, namely X, Y and Z. Now, using the MATLAB toolbox, I was able to train, test and validate a shallow feedforward ANN. It has 3 layers, one for the input, one for the hidden and the last one being the output. So, the ANN configuration in my case will be of the form 3-x-1, where 'x' is the number of neurons in the hidden layer. Apart for the tanh() activation function used in the middle layer, linear activation functions were used in both input and output layers. MapMinMax() was used for normalizing both the inputs and the output of the ANN.

However, after successfully training the ANN, when it comes to calculating the first order derivative of the ANN output with each of the inputs, I am getting orders of difference in the results when compared with derivatives obtained using analytical equation. I tried to understand and implement the codes provided in the works of "Approximation of functions and their derivatives:A neural network implementation with applications" but in vain. The code which I developed to do the same is attached with this message (ANN_FOD.m).

I would greatly appreciate if you could help me with the implementation of the code for ANN derivatives. Thank you for your time and kind regards,

Tittu V Mathew

soo-choon kang 2021-8-14

script needs to take care the normalization procedure.

请先登录，再进行评论。

Answer 2

Filipe 2012-10-20

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/51319-how-to-compute-the-derivative-of-the-neural-network#answer_62627

Thanks Greg!

I was able to make this derivation, but it is not that easy to do on the "net" structure. You need to dig in the "net" structure to take into account the pre- and post-processing made automatically by Matlab. I was able to do it, but it was not that easy.

I think these NN derivatives are very useful in a lot of applications.

Thanks for your help!

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Greg Heath 2012-10-24

You can try to make life easier by doing the pre and postprocessign yourself before and after training.

请先登录，再进行评论。

Answer 3

trevor 2013-11-7

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/51319-how-to-compute-the-derivative-of-the-neural-network#answer_114439

Hi Filipe,

Could you possibly share your code for computing the partial derivative of the ANN, or provide some info on the steps you used? That would be immensely useful!

Thanks, Trevor