Noise parameters in Reinforcement learning DDPG

51 次查看（过去 30 天）

Surya teja Tunuguntla 2019-6-14

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/467153-noise-parameters-in-reinforcement-learning-ddpg

评论： Atikah Surriani 2023-5-8

What should be the values of Noise parameters (for agent) if my action range is between -0.5 to -5 in DDPG reinforcement learning I want to explore whole action range for each sample time? Also is there anyway to make the noise options (for agent) independent of sample time?

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Drew Davis 2019-6-19

3
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/467153-noise-parameters-in-reinforcement-learning-ddpg#answer_379990

编辑：Drew Davis 2019-6-19

在 MATLAB Online 中打开

Hi Surya

It is fairly common to have Variance*sqrt(SampleTime) somewhere between 1 and 10% of your action range for Ornstein Uhlenbeck (OU) action noise. So in your case, the variance can be set between 4.5*0.01/sqrt(SampleTime) and 4.5*0.10/sqrt(SampleTime). The other important factor is the VarianceDecayRate, which will dictate how fast the variance will decay. You can calculate how many samples it will take for your variance to be halved by this simple formula:

halflife = log(0.5)/log(1-VarianceDecayRate)

It is critically important for your agent to explore while learning so keeping the VarianceDecayRate small (or even zero) is a good idea. The other noise parameters can usually be left as default.

You can check out this pendulum example which does a pretty good job of exploring during training.

The sample time of the noise options will be inherited by the agent, so it is not necessary to configure. By default, the noise model will be queried at the same rate as the agent.

Hope this helps

Drew