Problem with RL agent block

18 次查看（过去 30 天）
Ethan Serapio
Ethan Serapio 2024-1-18
评论： Ethan Serapio 2024-1-27
I have this simulink model which mainly utilizes functions blocks to process signal. The observation input for the RL agent is a scalar. obsinfo uses rlNumericSpec and ainfo uses rlFiniteSetSpec given its discrete property. I intend to validate the environment generated via the function env = rlSimulinkenv(mdl,agentblk,obsinfo,ainfo). To validate, I uses the function validateEnvironment(env). This generates an error.
The error states:
Error using rl.env.SimulinkEnvWithAgent/validateEnvironment. An error occurred while running the simulation for the model 'simulink2' with the following RL agent blocks: simulink2/RL agent
Model 'simulink2' has underspecified signal dimensions and Simulink has assigned these dimensions by default. To prevent ambiguities, consider speficying these dimensions in the block dialog box. Simulink has assigned these signal dimensions:
  • Block 'simulink2/RL Agent/last_observation' 'Output Port 1' has been set to [1]`
Before, I encounter 3 bullets with the same error. The two were pertaining to the same error regarding the specification of signal dimensions for last_action and external_action. This error were solved by enabling the RL agent parameter for this and assigning the appropriate signal values.
Now for the last_observation, I do not see how to access this parameter to assign the appropriate dimension in the RL agent block dialog box. What would be the way to let the model know that the dimension for the last_observation is indeed [1]? Is there a way to access this parameter in the block dialog box and where?

请先登录，再回答此问题。

采纳的回答

Emmanouil Tzorakoleftherakis
编辑：Emmanouil Tzorakoleftherakis 2024-1-24
You can use a delay block for the last observation and set the initial value of the delay in the block dialog. That should resolve the dimension issue for this signal.
  1 个评论
Ethan Serapio
Ethan Serapio 2024-1-27
I have solved the problem by adding a delay block in the model. Also, I have found in several forums that this is not an error but rather a warning although it appears as an error in the matlab window. The prompt may appear as an error given that it has a red font but in the default settings of the configuration in simulink, it is set as a warning. By this, the prompt can be disregarded by setting it as 'none' instead of 'warning'. The train function stops even for warnings thus the solution.
This option may not be practical for other models if the assumption of matlab with the dimension is incorrect, otherwise it is valid. This is also the solution for removing the matlab error regarding last_action and for the other one which I forgot which goes along with the error I've posted.
I have my RL model up and running. Thank you for your response.

请先登录，再进行评论。

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Reinforcement Learning 的更多信息

标签

产品


版本

R2023a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by