How to bypass automatic differentiation?

1 次查看（过去 30 天）
sunny Yang
sunny Yang 2020-2-21
评论： Xuefu Dong 2023-7-5
I'm trying to implement a simple speech recognition by Deep Learning Toolbox.
the network receives a piece of spectrogram,and it gives the possibility of each phone（ɔi au əu，。。。 etc) at each timetick.
I have some speech sentences recording & corresponding phonetic sign，but I must use CTC(Connectionist Temporal Classification) algorithm to match the phoentic sequence onto the possibility sequence outputed by CNN.
But the problem is:
CTC algorithm contains several nesting looped process,making it extremely unfriendly to automatic differentiation.A typical CTC call takes less then 0.1sec,but it takes 20 seconds to finish under dlfeval().
CTC algorithm gives a relation index,like(#1~#5 output of NN-> first sign in Supervise output，#6~#7 -> second sign...etc ),
On the other words,Loss = sum( [Neural_Network_Outp_vector - Supervise(CTC_relation_index) ].^2,'all'),and (CTC relation index) is a constant indexing vector here .
So,How to bypass the automatic differentiation here,to let the CTC call gives the relation index quickly?
Now I use this method:call neural network forward() and CTC outside dlfeval() to get the relation index ,and send both the input and this index into dlfeval(),forward() again to get the gradient.
Can I do forward() once?

请先登录，再回答此问题。

回答（1 个）

sunny Yang
sunny Yang 2020-4-16
编辑：sunny Yang 2020-4-16
use extractdata() to bypass grad:
x=dlarray(1);
[y,Y,dy_dx,dY_dx] = dlfeval(@goat,x)
% y=Y=4
dy_dx=4=d(x^2+2*x+1)/dx|x=1
dY_dx=2=d(x^2+1)/dx|x=1
function [y,Y,dy_dx,dY_dx] = goat(x)
X=x.extractdata();
y=x^2+2*x+1;
Y = x^2+2*X+1;
dy_dx = dlgradient(y,x);
dY_dx = dlgradient(Y,x);
end
better

类别

在 Help Center 和 File Exchange 中查找有关 Sequence and Numeric Feature Data Workflows 的更多信息

标签

产品


版本

R2019b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by