LSTM Python hyperparameters v MATLAB

1 次查看（过去 30 天）
Philip Hua
Philip Hua 2022-5-30
回答： Philip Hua 2022-6-3
I am reading a LSTM research paper and it states:
The following experiments investigate deep RNN models parameterized by the following hyperparameters: 1. num_layers – the number of memory cell layers 2. rnn_size – the number of hidden units per memory cell (i.e. hidden state dimension) 3. wordvec – dimension of vector embeddings 4. seq_length – number of frames before truncating BPTT gradient
I can see 2 and 3 as the number of hidden units and the input size but I cannot find where one would set 1 and 4.

请先登录，再回答此问题。

回答（2 个）

David Willingham
David Willingham 2022-6-1
Hi Philip,
For 1, by default layers are not a "settable" parameter. You need to setup an experiment that tests networks of different sizes and see which one might give the best results. This example Try Multiple Pretrained Networks for Transfer Learning shows how you can use the Experiment Manager App in MATLAB to do this.
For 4, while I don't have an example to share on this. You could use Experiment Manager to setup an experiment that changes the sequence length of the input data used to feed the LSTM training.
  2 个评论
Philip Hua
Philip Hua 2022-6-1
hi David,
Thank you for your help and suggestion. The author already tried different settings (in Python RNN) and came up with the the "optimal" set of hyper-parameters. Is there any plan for Mathworks to include these rather basic options?
David Willingham
David Willingham 2022-6-1
For 1, in MATLAB this isn't a settable parameter, however you can set them manually:
[lstmLayer(64); lstmLayer(64)]
For 4, there is an option to set sequencelength for the mini batch in the trainingoptions:
https://www.mathworks.com/help/deeplearning/ref/trainingoptions.html

请先登录，再进行评论。

Philip Hua
Philip Hua 2022-6-3
Thank you David. Could you however, clarify the suggested network configuration above? The number of memory cells i think is not the same as the number of lstm layers right? Perhaps you could kindly send an unrolled network diagram and label the suggested configuration ?

类别

在 Help Center 和 File Exchange 中查找有关 Sequence and Numeric Feature Data Workflows 的更多信息

标签

产品


版本

R2021b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by