Training a dense layer along with an lstm layer

Question

Yildirim Kocoglu 2021-1-7

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/710868-training-a-dense-layer-along-with-an-lstm-layer

评论： Yildirim Kocoglu 2021-2-9

I'm trying to set my initial hidden states in the lstm network using a dense layer but, I have been having problems with using multiple inputs in my network. The flow of the network looks something like below. The dense layer between static features (input) and init_hidden_states should be trainable. So far, I tried using a DAG network but, failed due to an error (If there is sequential input layer, there can't be any other input layer present). In my case the sequential input layer is used for time series input and I'm using an image input layer to input the static features. Any advice or example is appreciated. Thank you.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Asvin Kumar 2021-2-9

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/710868-training-a-dense-layer-along-with-an-lstm-layer#answer_618932

I suppose what you're asking for is to have an LSTM layer in which the initial values of hidden state is trainable. If that's the case, I'm afraid this is not currently possible. The initial value of hidden state of an LSTM is a property of the layer, the 'HiddenState' property, and it is a state parameter which is not learnable.

Refer the following doc page and the linked example for more details: HiddenState - LSTM Layer and LSTM Layer and Properties.

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Yildirim Kocoglu 2021-2-9

Thank you for letting me know Asvin. Yes, that’s what I meant: for initial values of the hidden state to be trainable. I was also looking to find out how the dimensions of the initial hidden state is decided: if I have num_hidden_units=50 in lstm, I end up having initial hidden state with dimensions (50,1). I was wondering where this “1” is from? When does it change to “2” for instance (if it does)? Thank you again.

请先登录，再进行评论。