Agent is suddently doing random actions and training diverge
1 次查看(过去 30 天)
显示 更早的评论
Reinforcement Learning
2021-3-21
编辑: Emmanouil Tzorakoleftherakis
2021-3-22
Hello,
I am training an DQN agent to replace a controller. Everytime the agent is about to converge, it starts to make random move and diverge. Although greedy epsilon was set to (0.3, 0.5, 0.6 etc.). Any idea what the reason might be?
Thanks in advance!
0 个评论
采纳的回答
Emmanouil Tzorakoleftherakis
2021-3-22
编辑:Emmanouil Tzorakoleftherakis
2021-3-22
This is normal behavior - one common misconception is that once the reward starts going up, it will remain up. This is not true as the agent may start exploring a completely different part of the state space and that can lead to sudden dips in the reward as you can see.
Once you observe good behavior for a few episodes in a row/good average behavior over a number of episodes, that's a good indication that you can stop training. So I would stop training after episode 50/60 in your case and see if the result works, or I would let it train for more time and see if it recovers/surpasses the previous max (this is also common)
0 个评论
更多回答(0 个)
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Training and Simulation 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!