Determine the reward value to stop training in RL agent

3 次查看（过去 30 天）
H. M.
H. M. 2022-10-17
评论： Francisco Serra 2023-12-14
I saw in example of using RL agent, this sentence:
  • Stop training when the agent receives an average cumulative reward greater than -355 over 100 consecutive episodes. At this point, the agent can control the level of water in the tank.
https://www.mathworks.com/help/reinforcement-learning/ug/tune-pi-controller-using-td3.html#:~:text=Stop%20training%20when%20the%20agent%20receives%20an%20average%20cumulative%20reward%20greater%20than%20%2D355%20over%20100%20consecutive%20episodes.%20At%20this%20point%2C%20the%20agent%20can%20control%20the%20level%20of%20water%20in%20the%20tank.
how did he calculate the exact reward -355 over 100 episodes? Is there any tips could help know when to stop the training at specific point before get worst.
thank you advance

请先登录，再回答此问题。

采纳的回答

Emmanouil Tzorakoleftherakis
编辑：Emmanouil Tzorakoleftherakis 2023-1-25
For some problems you may be able to calculate what the maximum reward that can be collected in an episode is, so you can use this knowledge accordingly in the training settings. In general, there is no recipe that will tell you when it would be good to stop training. You would typically need to train for a large number of episodes to see how the training goes and that could help you identify what a good average reward is. You could also just train for a set number of episodes instead (similar to how you would train for a certain number of epochs in supervised learning).
Hope that helps

更多回答（1 个）

Sam Chak
Sam Chak 2022-10-17
Hi @Haitham M.
There is an option to set the StopTrainingValue.
  2 个评论
H. M.
H. M. 2022-10-17
Thank you @Sam Chak for answering.
what I mean is how did he know that if the average cumulative reward reach -355, then the agent can control the level. why -355 exactly?
Francisco Serra
Francisco Serra 2023-12-14
For example, imagine your are using a RL agent for a control problem. You can use a classic controller to have a reference and apply to it the same cost function you use in the RL Agent. Then you do some simulations with that controller, you see how it goes and then you have an idea of how your RL Agent should perform. However, if you don't have a working reference to guide yourself you have to do what @Emmanouil Tzorakoleftherakis said.

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Reinforcement Learning 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by