Create policy evaluation function for RL agent
4 次查看(过去 30 天)
显示 更早的评论
Hello,
I'm using the Reinforcement Learning Toolbox to train an agent for a particular task. Once the training is complete, I want to generate a policy evaluation function so I can use it in a simulink file. I can create the policy function using "generatePolicyFuncion" command. The matlab script that is generated is the following:
function action1 = evaluatePolicy(observation1)
%#codegen
% Reinforcement Learning Toolbox
% Generated on: 23-Sep-2019 09:27:39
action1 = localEvaluate(observation1);
end
%% Local Functions
function action1 = localEvaluate(observation1)
persistent policy
if isempty(policy)
policy = coder.loadDeepLearningNetwork('agentData.mat','policy');
end
action1 = predict(policy,observation1);
end
When I run the simulink file, the following errors pop up:
Simulink does not have enough information to determine output sizes for this block. If you think the errors below are inaccurate, try specifying types for the block inputs and/or sizes for the block outputs.
Component:MATLAB Function | Category:Coder error
Dot indexing is not supported for variables of this type.
Function 'MATLAB Function' (#171.286.341), line 13, column 11:
"coder.loadDeepLearningNetwork('agentData.mat','policy')"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Persistent variable 'policy' must be assigned before it is used. The only exception is a check using 'isempty(policy)' that can be performed prior to assignment.
Function 'MATLAB Function' (#171.365.371), line 15, column 19:
"policy"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Function call failed.
Function 'MATLAB Function' (#171.140.167), line 7, column 11:
"localEvaluate(observation1)"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Errors occurred during parsing of MATLAB function 'test_RL/MATLAB Function'
Component:MATLAB Function | Category:Coder error
Simulink cannot determine sizes and/or types of the outputs for block 'test_RL/MATLAB Function' due to errors in the block body, or limitations of the underlying analysis. The errors might be inaccurate. Fix the indicated errors, or explicitly specify sizes and/or types for all block outputs.
Component:MATLAB Function | Category:Coder error
Dot indexing is not supported for variables of this type.
Function 'MATLAB Function' (#171.286.341), line 13, column 11:
"coder.loadDeepLearningNetwork('agentData.mat','policy')"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Persistent variable 'policy' must be assigned before it is used. The only exception is a check using 'isempty(policy)' that can be performed prior to assignment.
Function 'MATLAB Function' (#171.365.371), line 15, column 19:
"policy"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Function call failed.
Function 'MATLAB Function' (#171.140.167), line 7, column 11:
"localEvaluate(observation1)"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Errors occurred during parsing of MATLAB function 'test_RL/MATLAB Function'
How can I solve this issue?
Thanks
0 个评论
采纳的回答
Emmanouil Tzorakoleftherakis
2019-9-25
编辑:Emmanouil Tzorakoleftherakis
2019-9-25
Can you try defining the size of inputs and outputs in the MATLAB Function block? This seems to be coming up a lot in the error message
3 个评论
J
2019-10-17
Hi there,
Were you able to resolve this issuse? I am facing the same problem as you and haven't been able to figure it out...
SomeMatlabUser
2020-2-20
编辑:SomeMatlabUser
2020-2-20
Any Update? I am having the same problem. Also there is no documentation by Mathworks itself how to setup an Simulinkmodel executing the policy for code generation.
更多回答(1 个)
Sviatoslav Klos
2020-3-3
Hello.
I experienced the same issue. If you do not need code generation (only normal mode simulation is requested) I might suggest to substitute your two functions with the following code and insert it as a MATLAB User-defined fcn. Please, respond if it suits you.
function action1 = evaluatePolicy(observation1)%, policy) %#codegen
coder.extrinsic('evalin', 'predict');
action1 = single(0);
policy = evalin('base', 'policy') ;
action1 = predict(policy,observation1);
end
2 个评论
Sviatoslav Klos
2020-5-25
I think this is your project specific error. You might check the action of the RL agent. I might suggest to test this function with some RL example from MATLAB and the try to go into your application with obtained conclusions.
另请参阅
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!