Create policy evaluation function for RL agent

2 次查看（过去 30 天）
Juan Nunez
Juan Nunez 2019-9-23
评论： Sviatoslav Klos 2020-5-25
Hello,
I'm using the Reinforcement Learning Toolbox to train an agent for a particular task. Once the training is complete, I want to generate a policy evaluation function so I can use it in a simulink file. I can create the policy function using "generatePolicyFuncion" command. The matlab script that is generated is the following:
function action1 = evaluatePolicy(observation1)
%#codegen
% Reinforcement Learning Toolbox
% Generated on: 23-Sep-2019 09:27:39
action1 = localEvaluate(observation1);
end
%% Local Functions
function action1 = localEvaluate(observation1)
persistent policy
if isempty(policy)
policy = coder.loadDeepLearningNetwork('agentData.mat','policy');
end
action1 = predict(policy,observation1);
end
When I run the simulink file, the following errors pop up:
Simulink does not have enough information to determine output sizes for this block. If you think the errors below are inaccurate, try specifying types for the block inputs and/or sizes for the block outputs.
Component:MATLAB Function | Category:Coder error
Dot indexing is not supported for variables of this type.
Function 'MATLAB Function' (#171.286.341), line 13, column 11:
"coder.loadDeepLearningNetwork('agentData.mat','policy')"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Persistent variable 'policy' must be assigned before it is used. The only exception is a check using 'isempty(policy)' that can be performed prior to assignment.
Function 'MATLAB Function' (#171.365.371), line 15, column 19:
"policy"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Function call failed.
Function 'MATLAB Function' (#171.140.167), line 7, column 11:
"localEvaluate(observation1)"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Errors occurred during parsing of MATLAB function 'test_RL/MATLAB Function'
Component:MATLAB Function | Category:Coder error
Simulink cannot determine sizes and/or types of the outputs for block 'test_RL/MATLAB Function' due to errors in the block body, or limitations of the underlying analysis. The errors might be inaccurate. Fix the indicated errors, or explicitly specify sizes and/or types for all block outputs.
Component:MATLAB Function | Category:Coder error
Dot indexing is not supported for variables of this type.
Function 'MATLAB Function' (#171.286.341), line 13, column 11:
"coder.loadDeepLearningNetwork('agentData.mat','policy')"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Persistent variable 'policy' must be assigned before it is used. The only exception is a check using 'isempty(policy)' that can be performed prior to assignment.
Function 'MATLAB Function' (#171.365.371), line 15, column 19:
"policy"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Function call failed.
Function 'MATLAB Function' (#171.140.167), line 7, column 11:
"localEvaluate(observation1)"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Errors occurred during parsing of MATLAB function 'test_RL/MATLAB Function'
How can I solve this issue?
Thanks

请先登录，再回答此问题。

采纳的回答

Emmanouil Tzorakoleftherakis
编辑：Emmanouil Tzorakoleftherakis 2019-9-25
Can you try defining the size of inputs and outputs in the MATLAB Function block? This seems to be coming up a lot in the error message
  3 个评论
显示 1更早的评论
J
J 2019-10-17
Hi there,
Were you able to resolve this issuse? I am facing the same problem as you and haven't been able to figure it out...
SomeMatlabUser
SomeMatlabUser 2020-2-20
编辑：SomeMatlabUser 2020-2-20
Any Update? I am having the same problem. Also there is no documentation by Mathworks itself how to setup an Simulinkmodel executing the policy for code generation.

请先登录，再进行评论。

更多回答（1 个）

Sviatoslav Klos
Sviatoslav Klos 2020-3-3
Hello.
I experienced the same issue. If you do not need code generation (only normal mode simulation is requested) I might suggest to substitute your two functions with the following code and insert it as a MATLAB User-defined fcn. Please, respond if it suits you.
function action1 = evaluatePolicy(observation1)%, policy) %#codegen
coder.extrinsic('evalin', 'predict');
action1 = single(0);
policy = evalin('base', 'policy') ;
action1 = predict(policy,observation1);
end
  2 个评论
Juan Nunez
Juan Nunez 2020-5-15
Hello Sviatoslav,
Thnk your for your response.
I'm sorry for my late reply. I tried your suggestion, but the following error keeps popping up:
"Variable 'action1' has conflicting type definitions. To resolve, verify that all preceding definitions of 'action1' are consistent.
Function 'MATLAB Function' (#420.371.409), line 14, column 1:
"action1 = predict(policy,observation1)"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Errors occurred during parsing of MATLAB function 'Copy_of_RF_FOC6/MATLAB Function'
Component:MATLAB Function | Category:Coder error
Errors occurred during parsing of MATLAB function 'Copy_of_RF_FOC6/MATLAB Function'"
Sviatoslav Klos
Sviatoslav Klos 2020-5-25
I think this is your project specific error. You might check the action of the RL agent. I might suggest to test this function with some RL example from MATLAB and the try to go into your application with obtained conclusions.

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Block Libraries 的更多信息

标签

产品


版本

R2019a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by