Teacher forcing for a LSTM network

5 次查看（过去 30 天）

Philip Hua 2022-5-22

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1724770-teacher-forcing-for-a-lstm-network

评论： Philip Hua 2022-6-7

Is there a way to implement teacher forcing for a LSTM network in MATLAB? Hopefully there is an option buried somewhere.

回答（2 个）

David Willingham 2022-5-24

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1724770-teacher-forcing-for-a-lstm-network#answer_970775

Hi Philip,

This example shows how teacher forcing can be implemented with LSTM's in MATLAB.Sequence-to-Sequence Translation Using Attention.

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Philip Hua 2022-5-24

hi David,

Thank you for your help but this is a attention model. I just need TF for a standard LSTM model.

请先登录，再进行评论。

David Willingham 2022-5-24

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1724770-teacher-forcing-for-a-lstm-network#answer_970830

编辑：David Willingham 2022-5-24

Please see the attached example, trainLSTM_seq2seq. Is this what you were looking for? I.e. an example for a standard LSTM model?

5 个评论
显示 3更早的评论隐藏 3更早的评论

David Willingham 2022-6-7

If your problem is a time-series problem then it possibly could. Have you tried adapting this example to meet your use case? If so, is it getting the result you expect?

Philip Hua 2022-6-7

hi David,

Thank you for your email. I just implemented the basic code without teacher forcing yesterday and during training it gives 30-40% accuracy. The author

https://www.mlmi.eng.cam.ac.uk/files/feynman_liang_8224771_assignsubmission_file_liangfeynmanthesis.pdf

seems to suggest that the model should perform much better. There are two things missing from my model currently:

1) i have not embedded the tokens (page 27) and

2) No teacher forcing

The problem with 1) is that these tokens are categorical tokens so after embedding (using the embed functions), I don't know how to retrieve the original tokens from the embedded data. I also presume that the data has to be discretized and there is no mention in the thesis of this so I am not sure what is going on TBH. Perhaps you can shed some light on this

2) i was going to try to use what you send to implement teacher forcing but I thought the open loop is a much neater way to implement the solution in this case. I am also dubious about TF - i suspect that all it's going to do is to overfit the data.

请先登录，再进行评论。

请先登录，再回答此问题。