Oscillating reward in DDPG using Matlab Reinforcement learning toolbox with simulink environment

5 次查看（过去 30 天）
Arman Ali
Arman Ali 2022-9-1
回答： Emmanouil Tzorakoleftherakis 2023-1-25
I have 3 observations for each agent in a multi-agent environment.
Similirly, there is one action for each agent between [-1 1].
I am using DDPG agents to train my model. However, the graph for reward is oscillating after some episodes, at first it looks like the reward is converging to a value bit lower than optimal (i.e. episode reward 0 appx) but then it starts oscillating between high values and lower values for reward. The standard deviation for noise is 0.1, decay rate is 0.0001. What are the possible causes? how to improve reward and avoid oscillation? Thank you.

请先登录，再回答此问题。

回答（1 个）

Emmanouil Tzorakoleftherakis
I think I have mentioned this in another post as well, but you should not expect your episode reward to be monotonic. Once it "converges" around some value for a few episodes, that is a reasonable time to stop training and see if you are happy with the result. If you keep training, the optimization might move to a different direction that leads to worse behavior, similarly to what you are seeing here.

类别

在 Help Center 和 File Exchange 中查找有关 Reinforcement Learning 的更多信息

标签

产品


版本

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by