why the continuous gaussian actor function is not working?

1 次查看（过去 30 天）
Ehab
Ehab 2022-6-27
回答： Ayush 2023-9-29
why the continuous gaussian actor function is not working and giving me this bug "Unrecognized field name "obsInfo""?
  3 个评论
显示 1更早的评论
Ehab
Ehab 2022-6-27
sure,
env = RocketLander;
actionInfo = getActionInfo(env);
obsInfo = getobsInfo(env);
numObs = obsInfo.Dimension(1);
numAct = numel(actionInfo.Elements);
Ts = 0.1;
rng(0)
criticLayerSizes = [400 300];
actorLayerSizes = [400 300];
criticNetwork = [
featureInputLayer(numObs,'Normalization','none','Name','observation')
fullyConnectedLayer(criticLayerSizes(1),'Name','CriticFC1', ...
'Weights',sqrt(2/numObs)*(rand(criticLayerSizes(1),numObs)-0.5), ...
'Bias',1e-3*ones(criticLayerSizes(1),1))
reluLayer('Name','CriticRelu1')
fullyConnectedLayer(criticLayerSizes(2),'Name','CriticFC2', ...
'Weights',sqrt(2/criticLayerSizes(1))*(rand(criticLayerSizes(2),criticLayerSizes(1))-0.5), ...
'Bias',1e-3*ones(criticLayerSizes(2),1))
reluLayer('Name','CriticRelu2')
fullyConnectedLayer(1,'Name','CriticOutput', ...
'Weights',sqrt(2/criticLayerSizes(2))*(rand(1,criticLayerSizes(2))-0.5), ...
'Bias',1e-3)];
criticNetwork = dlnetwork(criticNetwork);
criticOpts = rlOptimizerOptions('LearnRate',1e-4);
critic = rlValueFunction(criticNetwork,observationInfo);
actorNetwork = [featureInputLayer(numObs,'Normalization','none','Name','observation')
fullyConnectedLayer(actorLayerSizes(1),'Name','ActorFC1', ...
'Weights',sqrt(2/numObs)*(rand(actorLayerSizes(1),numObs)-0.5), ...
'Bias',1e-3*ones(actorLayerSizes(1),1))
reluLayer('Name','ActorRelu1')
fullyConnectedLayer(actorLayerSizes(2),'Name','ActorFC2', ...
'Weights',sqrt(2/actorLayerSizes(1))*(rand(actorLayerSizes(2),actorLayerSizes(1))-0.5), ...
'Bias',1e-3*ones(actorLayerSizes(2),1))
reluLayer('Name', 'ActorRelu2')
fullyConnectedLayer(numAct,'Name','Action', ...
'Weights',sqrt(2/actorLayerSizes(2))*(rand(numAct,actorLayerSizes(2))-0.5), ...
'Bias',1e-3*ones(numAct,1))
softmaxLayer('Name','actionProb')];
actorNetwork = dlnetwork(actorNetwork);
actorOpts = rlOptimizerOptions('LearnRate',1e-4);
actor = rlContinuousGaussianActor(actorNetwork,obsInfo,actionInfo);
agentOpts = rlPPOAgentOptions(...
'ExperienceHorizon',600,...
'ClipFactor',0.02,...
'EntropyLossWeight',0.01,...
'ActorOptimizerOptions',actorOpts,...
'CriticOptimizerOptions',criticOpts,...
'NumEpoch',3,...
'AdvantageEstimateMethod','gae',...
'GAEFactor',0.95,...
'SampleTime',Ts,...
'DiscountFactor',0.997);
agent = rlPPOAgent(actor,critic,agentOpts);
trainOpts = rlTrainingOptions(...
'MaxEpisodes',20000,...
'MaxStepsPerEpisode',600,...
'Plots','training-progress',...
'StopTrainingCriteria','AverageReward',...
'StopTrainingValue',430,...
'ScoreAveragingWindowLength',100,...
'SaveAgentCriteria',"EpisodeReward",...
'SaveAgentValue',700);
doTraining = true;
if doTraining
trainingStats = train(agent,env,trainOpts);
else
load('rocketLanderAgent.mat');
end

请先登录，再进行评论。

请先登录，再回答此问题。

回答（1 个）

Ayush
Ayush 2023-9-29
Hey Ehab,
I understand that while using the continuous gaussian actor function “rlContinuousGaussianActor”, you are encountering an error “Unrecognized field name "obsInfo"”.
The error occurs because the "rlContinuousGaussianActor" function expects the third input to be the “actionInfo”, not “obsInfo”. In your code, you have mistakenly passed “obsInfo” as the third input instead of “actionInfo”.
To resolve this issue, you need to update the creation of the “rlContinuousGaussianActor” as follows:
actor = rlContinuousGaussianActor(actorNetwork, actionInfo, obsInfo);
For more information on "rlContinuousGaussianActor" function, refer to MathWorks documentation link below:
https://in.mathworks.com/help/reinforcement-learning/ref/rl.function.rlcontinuousgaussianactor.html
Hope above provided information helps!
Regards,
Ayush Goyal

类别

在 Help Center 和 File Exchange 中查找有关 Applications 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by