Agent is suddently doing random actions and training diverge

Question

Reinforcement Learning 2021-3-21

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/779497-agent-is-suddently-doing-random-actions-and-training-diverge

编辑： Emmanouil Tzorakoleftherakis 2021-3-22

Hello,

I am training an DQN agent to replace a controller. Everytime the agent is about to converge, it starts to make random move and diverge. Although greedy epsilon was set to (0.3, 0.5, 0.6 etc.). Any idea what the reason might be?

Thanks in advance!

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Emmanouil Tzorakoleftherakis 2021-3-22

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/779497-agent-is-suddently-doing-random-actions-and-training-diverge#answer_654812

编辑：Emmanouil Tzorakoleftherakis 2021-3-22

This is normal behavior - one common misconception is that once the reward starts going up, it will remain up. This is not true as the agent may start exploring a completely different part of the state space and that can lead to sudden dips in the reward as you can see.

Once you observe good behavior for a few episodes in a row/good average behavior over a number of episodes, that's a good indication that you can stop training. So I would stop training after episode 50/60 in your case and see if the result works, or I would let it train for more time and see if it recovers/surpasses the previous max (this is also common)