Can someone please show me how to implement the stepfunction in Reinforcement Learning environment using "ActionVectors" in a continous action space (rlNumericSpec)?

Question

mohamed gamal 2021-12-23

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1616415-can-someone-please-show-me-how-to-implement-the-stepfunction-in-reinforcement-learning-environment-u

回答： Shubham 2024-2-13

I am currently trying to implement Reinforcment Learning over an environment with 6 continous actions. I tried to implement the stepfunction set in the example but I got a dimensionality error in the input variable "Action". I realized I need to implement a customized stepfunction with another input variable called "ActionVectors"(You can check the implementation of this variable with rlFiniteSpec action in the finance code). So, can someone provide me with an example that uses "ActionVectors" when actions are defined as rlNumericSpec ?

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Shubham 2024-2-13

Hi Mohamed,

To resolve the dimensionality error in your MATLAB Reinforcement Learning environment when using rlNumericSpec for continuous actions, follow these steps:

Define the Action Space Correctly: Use rlNumericSpec to define the continuous action space accurately, specifying the number of actions and their respective lower and upper bounds.
Customize the Step Function: Write a custom step function that accepts an action input argument with the correct dimensionality. This step function should take into account the action vector provided by the RL agent, which will have the same dimensions as specified by your rlNumericSpec.
Implement Action Processing: Inside the step function, process the input action vector to determine the next state of the environment, calculate the reward, and decide if the episode is done.
Ensure Consistency: Make sure that the action vector passed to the step function by the RL agent during training or simulation matches the dimensions specified by your rlNumericSpec. Typically, this vector will be a column vector where each element represents a continuous action.
Create the Environment Interface: Use rlFunctionEnv or a similar function to create the environment interface, providing the custom step function, the observation space specification, and the action space specification.
Train the RL Agent: Train your RL agent using the environment you've created. The agent will generate action vectors that conform to the action space specification, and these vectors will be passed to your custom step function during training.