How RL + lstm work

Question

Jin 2025-9-8

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2179846-how-rl-lstm-work

回答： Shantanu 2025-9-12

I'm using MATLAB for reinforcement learning. I've activated the RNN of the agent and noticed that a layer of LSTM has been added to the network. Now I want to know whether this LSTM uses the parameters of the previous network output at the current time as the time series, or uses the observations at different times or the previous layer of the LSTM network as the time series. Also, are there any relevant literatures on RL + LSTM?

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Shantanu 2025-9-12

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2179846-how-rl-lstm-work#answer_1570255

Hi Jin,

As for the LSTM input, it uses the activations (not parameters) from the previous network layer at the current time as its main input. It handles the "time series" aspect by combining this with its internal hidden state (its memory) from the previous time step.

Therefore, it processes observations one by one, not all at once. As for the RL agents that can use LSTMs, the main ones are DQN, PPO, A2C, DDPG, SAC, and TD3.

Some resources and examples that may be helpful

https://www.mathworks.com/help/reinforcement-learning/ug/create-agents-for-reinforcement-learning.html?requestedDomain=

https://www.mathworks.com/help/reinforcement-learning/ug/train-dqn-to-control-house-heating.html