Error in "Soft-Actor critic" agent with "rlNumericSpec" Observations and "rlFiniteSpec" Actions.

Question

minhyuk jeung 2024-10-21，10:31

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2161060-error-in-soft-actor-critic-agent-with-rlnumericspec-observations-and-rlfinitespec-actions

回答： Gayathri 2024-10-29，3:13

Hello, I am trying to train SAC agent with "rlNumericSpec" Observations and "rlFiniteSpec" Actions.

This is my code:

ObservationInfo = rlNumericSpec([2 1],...
    'LowerLimit',[-inf -inf]',...
    'UpperLimit',[inf inf]');
ObservationInfo.Name = "observations";
ObservationInfo.Description = 'aa, bb';
numObservations = ObservationInfo.Dimension(1);
ActionInfo = rlFiniteSetSpec([1 2 3 4 5 6 7 8 9]);
ActionInfo.Name = 'Action';
ActionInfo.Description = 'test';
numActions = ActionInfo.Dimension(1);
net = [  
    featureInputLayer(ObservationInfo.Dimension(1))
    fullyConnectedLayer(16)
    reluLayer
    fullyConnectedLayer(16)
    reluLayer
    fullyConnectedLayer(9) 
    ];
net = dlnetwork(net);
critic = rlVectorQValueFunction(net,ObservationInfo,ActionInfo);
net1 = [  
    featureInputLayer(ObservationInfo.Dimension(1))
    fullyConnectedLayer(16)
    reluLayer
    fullyConnectedLayer(16)
    reluLayer
    fullyConnectedLayer(9) 
    ];
net1 = dlnetwork(net1);
actor = rlDiscreteCategoricalActor(net1,ObservationInfo,ActionInfo);
agentOptions = rlSACAgentOptions("SampleTime",Ts, ...
    "DiscountFactor", 0.001, ...
    "ExperienceBufferLength",1e6, ...
    "MiniBatchSize",64, ...
    "TargetSmoothFactor",0.005 ...
    );
agent = rlSACAgent(actor,critic,agentOptions);

However, when I trying to train the model, the error occurs:

Error using rlSACAgent (line 96)
First argument must be an rlDiscreteCategoricalActor object, an rlContinuousGaussianActor object or an observation specification created
using 'rlNumericSpec' or 'rlFiniteSetSpec' objects.
Error in mdl_RF (line 60)
agent = rlSACAgent(actor,critic,agentOptions);

Please help me if you know the solutions.

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Hitesh 2024-10-22，7:10

Hi @minhyuk jeung

I am not able to reproduce this error using the above snippet of code, Could you share the complete code where you are creating the "env" and passing it during training ?

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Gayathri 2024-10-29，3:13

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2161060-error-in-soft-actor-critic-agent-with-rlnumericspec-observations-and-rlfinitespec-actions#answer_1538500

Hi @minhyuk jeung,

I faced a similar issue in MATLAB R2024a version. But the same code runs fine in MATLAB R2024b version. The solution would be to upgrade to MATLAB R2024b version.

Hope you find this information helpful.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

Error in "Soft-Actor critic" agent with "rlNumericSpec" Observations and "rlFiniteSpec" Actions.

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

回答（1 个）

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

另请参阅

类别

标签

Community Treasure Hunt

Error in "Soft-Actor critic" agent with "rlNumericSpec" Observations and "rlFiniteSpec" Actions.

1 个评论 显示 -1更早的评论隐藏 -1更早的评论

回答（1 个）

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

另请参阅

类别

标签

Community Treasure Hunt

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

0 个评论
显示 -2更早的评论隐藏 -2更早的评论