LSTM padding and masking

Question

Ao Du 2020-12-10

2
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/689229-lstm-padding-and-masking

回答： Haijun Ruan 2021-7-21

I am solving a sequence-to-sequence classification problem based on LSTM using Matlab 2020b. The sequences have varaible length so padding within each minibatch is needed. However, I am not sure if Matlab automatically do the masking when calculating the crossentroy loss as well as the training/validation accuracy. From the training plot, the reported accuracy (around 70%) is much lower than those manually calculated by using checkpoints (where I get around 90% accuracy). I suspect although Matlab 2020b supports sequence padding and validation data in LSTM, it still did not offer the option of masking to reduce the influence caused by padding. Any insights?

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Aditya Patil 2020-12-22

2
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/689229-lstm-padding-and-masking#answer_581495

Currently, masking is not supported in MATLAB. I have brought the request to the notice of concerned people.

As a workaround, you can sort the inputs so that the amount of padding required is minimized. You may also set the minibatch size to 1, so that no padding is required.