Specifying Actions using rlFiniteSetSpec or rlNumericSpec (Reinforcement Learning)

Question

Huzaifah Shamim 2020-6-17

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/550113-specifying-actions-using-rlfinitesetspec-or-rlnumericspec-reinforcement-learning

回答： jie lu 2022-4-26

I understand that I can use CreateGridWorld for a Grid Maze environment for reinforcement learning. However, when creating my own, I have to specify the action settings using either rlFiniteSetSpec or rlNumericSpec. How am I to specify them if my actions are 4 (U, D, L, R). Any suggestions and help would be much appreciated!

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Rushi Vyas 2020-6-18

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/550113-specifying-actions-using-rlfinitesetspec-or-rlnumericspec-reinforcement-learning#answer_453069

在 MATLAB Online 中打开

Hi Huzaifah,

It is my understanding that have 4 actions (U, D, L, R) and want to specify them in your RL agent.

Generally, rlNumericSpec is used when you have a continuous action space and rlFiniteSetSpec is used when you have a discrete action space.

In your case, the action space is discrete. So, you should go with rlFiniteSetSpec.

As rlFiniteSetSpec takes only numeric input, you have to encode your actions to numbers. For example, (U,D,L,R) -> (1,2,3,4)

>> actInfo = rlFiniteSetSpec([1,2,3,4])  
actInfo = 
  rlFiniteSetSpec with properties:
       Elements: [4×1 double]
           Name: [0×0 string]
    Description: [0×0 string]
      Dimension: [1 1]
       DataType: "double"

For more information you can refer to : https://in.mathworks.com/help/reinforcement-learning/ref/rl.util.rlfinitesetspec.html

3 个评论
显示 1更早的评论隐藏 1更早的评论

Dongqi Liu 2020-10-28

在 MATLAB Online 中打开

Hi Huzaifah,

If you want to use rlFiniteSetSpec to specify 4 actions,you could try like this.

actInfo = rlFiniteSetSpec({[1 5],[1 6],[2 5],[2 6]})

For example,this code can decide your first action in 1 or 2,and your second action in 5 or 6.

If there are 4 actions and 5 possbilities each action,you might create a 1x625 cell element to specify actinfo.

It is not a good way,but I hope thiscould help you.

Dr. Ali Peker 2021-3-15

If we have a custom step function which will use these actions. How we can access individual actions inside actInfo for rlNumericSpec and rlFiniteSetSpec?

请先登录，再进行评论。