Teacher forcing for a LSTM network

5 次查看（过去 30 天）
Philip Hua
Philip Hua 2022-5-22
评论： Philip Hua 2022-6-7
Is there a way to implement teacher forcing for a LSTM network in MATLAB? Hopefully there is an option buried somewhere.

请先登录，再回答此问题。

回答（2 个）

David Willingham
David Willingham 2022-5-24
Hi Philip,
This example shows how teacher forcing can be implemented with LSTM's in MATLAB.Sequence-to-Sequence Translation Using Attention.
  1 个评论
Philip Hua
Philip Hua 2022-5-24
hi David,
Thank you for your help but this is a attention model. I just need TF for a standard LSTM model.

请先登录，再进行评论。

David Willingham
David Willingham 2022-5-24
编辑：David Willingham 2022-5-24
Please see the attached example, trainLSTM_seq2seq. Is this what you were looking for? I.e. an example for a standard LSTM model?
  5 个评论
显示 3更早的评论
David Willingham
David Willingham 2022-6-7
If your problem is a time-series problem then it possibly could. Have you tried adapting this example to meet your use case? If so, is it getting the result you expect?
Philip Hua
Philip Hua 2022-6-7
hi David,
Thank you for your email. I just implemented the basic code without teacher forcing yesterday and during training it gives 30-40% accuracy. The author
https://www.mlmi.eng.cam.ac.uk/files/feynman_liang_8224771_assignsubmission_file_liangfeynmanthesis.pdf
seems to suggest that the model should perform much better. There are two things missing from my model currently:
1) i have not embedded the tokens (page 27) and
2) No teacher forcing
The problem with 1) is that these tokens are categorical tokens so after embedding (using the embed functions), I don't know how to retrieve the original tokens from the embedded data. I also presume that the data has to be discretized and there is no mention in the thesis of this so I am not sure what is going on TBH. Perhaps you can shed some light on this
2) i was going to try to use what you send to implement teacher forcing but I thought the open loop is a much neater way to implement the solution in this case. I am also dubious about TF - i suspect that all it's going to do is to overfit the data.

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Recognition, Object Detection, and Semantic Segmentation 的更多信息

标签

产品


版本

R2021a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by