Why my RL Agent action still passing the upper and lower limit ?

Question

ardi ferdyhana 2021-6-7

1
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/849525-why-my-rl-agent-action-still-passing-the-upper-and-lower-limit

编辑： Azmi Yagli 2023-9-5

I am using Policy Gradient Agent, I want that my action only in range 0 - 100 and i already set up my UpperLimit to 100, and LowerLimit to 0. But as you can see -scope display 3-, my action still can passing the limit. How can i fix that ?

2 个评论
显示无隐藏无

Emmanouil Tzorakoleftherakis 2021-6-9

which one is the action here? How does your actor network look like?

denny 2021-12-7

I have solve my similar problem.

actInfo =rlNumericSpec([ 1],'UpperLimit',0.0771,'LowerLimit',-0.0405)

it means the minimum value is -0.0405, the maximum value is -0.0405+0.0771*2.

but your output is -1000 to 1000, I also donot know it.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

denny 2021-11-18

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/849525-why-my-rl-agent-action-still-passing-the-upper-and-lower-limit#answer_834784

I have the same problems. How to solve it?

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

Answer 2

Azmi Yagli 2023-9-5

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/849525-why-my-rl-agent-action-still-passing-the-upper-and-lower-limit#answer_1302071

编辑：Azmi Yagli 2023-9-5

If you look at rlNumericSpec, you can see this on LoweLimit or UpperLimit section.

DDPG, TD3 and SAC agents use this property to enforce lower limits on the action. When using other agents, if you need to enforce constraints on the action, you must do so within the environment.

So if you use other algorithms you can use saturation, but it didn't work for me.

You can try discretize actions of your agent so it can have boundaries.

Or you can give negative reward, if your agent exceeds limits for action.