DDPG Agent isn't learning (reward 0 for every episode)

8 次查看（过去 30 天）
Reinforcement Learning
评论： yovel atia 2021-12-2
Hello,
I suspect that the SampleTime Ts has something to do with it, I set Ts = 1e-6, but the trainnig is still going very fast.
I tried the same thing that was used for the Water Tank Model:
https://de.mathworks.com/help/reinforcement-learning/ug/create-simulink-environment-and-train-agent.html?searchHighlight=create%20simulink%20rl&s_tid=srchtitle
Just like in the example my system is very well controlled with a PI-Controller, but the DDPG Agent isn't learning anything.
What exactly is wrong with this system?

请先登录，再回答此问题。

采纳的回答

Emmanouil Tzorakoleftherakis
The reason why you see 0 rewards is because thw IsDone flag (which is used to terminate episodes early) is immediately set to true at the beginning of each episode. Either set it to false, or set the appropriate logic for you system.
Note that there are other things that need to be fixed in your setup, the most notable one being the agent sample time (which is currently very small) and the episode duration/max steps (which is currently very large). Unless you adjust these to more reasonable values for RL, your training will take days
  5 个评论
显示 3更早的评论
Emmanouil Tzorakoleftherakis
Emmanouil Tzorakoleftherakis 2021-3-22
Tha's what I am saying, you may need to reconsider your inputs/outputs. I am not sure what you mean by "made it even worse". If you are not seeing good training results, there could be a lot of other reasons including training options and hyperparameters, networks architectures etc. I would give the duty cycle idea another try and spend more time on the hyperparameters
yovel atia
yovel atia 2021-12-2
Hi
I get the same error, only for me the graph converges to -5000 constant
I do not know how to fix it I would be very happy to help
thanks!

请先登录，再进行评论。

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Reinforcement Learning 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by