Reinforcement Learning Noise Model Mean Attraction Constant

11 次查看（过去 30 天）

Tech Logg Ding 2020-12-4

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/677798-reinforcement-learning-noise-model-mean-attraction-constant

评论： Sourabh 2024-1-9

What does the mean attraction constant do? How can I tune it properly to promote exploration and learning? I can't seem to get the logic behind it.

With a sample time of 2, when I set it to 1 I get very noisy outputs. In the following graphs, rpm and valve%opening are the agents outputs and they are already scaled by a scaling layer.

When I set it to 0.05, then it seems like the noise model is not doing much explorations.

I also noticed that by applying the abs(1 - MeanAttractionConstant.*SampleTime) formula,

When sample time is 2 and the MAC is 1, the formula gives 1.

When sample time is 2 and the MAC is 0.05, the formula gives 0.9.

How does this relate to how fast the noise converge to the mean?

Thank you very much.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Emmanouil Tzorakoleftherakis 2020-12-4

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/677798-reinforcement-learning-noise-model-mean-attraction-constant#answer_565503

Assuming you are using DDPG, there is some information on the noise model here. I wouldn't worry too much about the mean attraction constant. The value of variance, variancedecayrate and variancemin play a much bigger role on 1) how much noise is added to the agent output and 2) for how long. If you want less noise to be added, reduce the variance value. If you want to explore for longer time, reduce the decay rate and set variancemin to a larger value.