Reinforcement Learning Error with two scalar inputs

Question

ali farid 2024-1-31

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2076621-reinforcement-learning-error-with-two-scalar-inputs

评论： Emmanouil Tzorakoleftherakis 2024-2-12

I have a strange error from a critic network that has 3 inputs, image, and two scalars. But I see the following error:

Error using rl.internal.validate.mapFunctionObservationInput
Unable to automatically specify deep neural network observation input layer names because some specifications have similar dimension. Specify "ObservationInputNames" name-value pair when
creating function object.
Error in rlContinuousGaussianActor (line 95)
modelInputMap = rl.internal.validate.mapFunctionObservationInput(model,observationInfo,nameValueArgs.ObservationInputNames);

Do you have any idea? It is really long time I am working on this error. My network is as follows:

type=3;
w1=2;
w2=1;
%  obsMat = [1 1];
obsMat = [4 3; 5 3; 6 3; 7 3; 8 3; 9 3; 5 11;  6 11; 7 11; 8 11; 6 12; 7 12; 10 12; ];
sA0 = [2 5];
sB0 = [11 5];
sC0 = [3 2];
sD0 = [6 5];
sE0 = [12 5];
sF0 = [6 12];
sG0 = [11 5];
sH0 = [3 11];
sI0 = [6 3];
sJ0 = [1 11];
s0 = [sA0; sB0; sC0];
% s0 = [sA0; sB0; sC0; sD0];
% s0 = [sA0; sB0; sC0];
Ts = 0.1;
Tf = 100;
maxsteps = ceil(Tf/Ts);
mdl = "rlAreaCoverage32024";
open_system(mdl)
% Define observation specifications.
scalarObs1Info = rlNumericSpec([1 1]);
scalarObs1Info.Name ="scalarObservation1";
scalarObs2Info = rlNumericSpec([1 1]);
scalarObs2Info.Name ="scalarObservation2";
obsSize = [12 12 4];
oinfo = rlNumericSpec(obsSize);
oinfo.Name ="image";
% oinfo.Name = "observations";
allObsInfo = [ oinfo, scalarObs1Info, scalarObs2Info];%, scalarObs3Info];
 
actionSpace = {1,2,3,4};
ActionInfo = rlNumericSpec([1, 2], 'Lowerlimit', -1, 'Upperlimit', 1); ainfo = ActionInfo;
ainfo.Name = "actions";
actInfo.UpperLimit=1;
actInfo.Lowerlimit=-1;
blks = mdl + ["/Agent A (Red)","/Agent B (Green)","/Agent C (Blue)"];
env = rlSimulinkEnv(mdl,blks,{allObsInfo,allObsInfo,allObsInfo},{ainfo,ainfo,ainfo});
env.ResetFcn = @(in) resetMap(in, obsMat);
rng(0)
for idx = 1:type
lgraph = layerGraph();
    tempLayers = [
    featureInputLayer(w1,"Name","scalarInput1")
    reluLayer("Name","relu_3")
    fullyConnectedLayer(1,"Name","fc_4")];
lgraph = addLayers(lgraph,tempLayers);
tempLayers = [
    featureInputLayer(w3,"Name","scalarInput3")
    reluLayer("Name","relu_1")
    fullyConnectedLayer(1,"Name","fc_2")];
lgraph = addLayers(lgraph,tempLayers);
tempLayers = [ 
        imageInputLayer(obsSize,Normalization="none")
        convolution2dLayer(8,16, ...
            Stride=1,Padding=1,WeightsInitializer="he")
        reluLayer
        convolution2dLayer(4,8, ...
            Stride=1,Padding="same",WeightsInitializer="he")
        reluLayer
        fullyConnectedLayer(256,WeightsInitializer="he")
        reluLayer
        fullyConnectedLayer(128,WeightsInitializer="he")
%         Hidden units (default = 128) are the number of units in the hidden layer of the neural network. Its size depends on the complexity of the problem, and should be set larger when there is a complex relationship between agent actions and observed variables.
        reluLayer
        fullyConnectedLayer(1,"Name","fc_1")];
lgraph = addLayers(lgraph,tempLayers);
tempLayers = [
    concatenationLayer(1,3,"Name","concat")
    softmaxLayer("Name","softmax")];
lgraph = addLayers(lgraph,tempLayers);
% clean up helper variable
clear tempLayers;
 lgraph = connectLayers(lgraph,"fc_2","concat/in3");
% lgraph = connectLayers(lgraph,"fc_3","concat/in2");
lgraph = connectLayers(lgraph,"fc_1","concat/in4");
lgraph = connectLayers(lgraph,"fc_4","concat/in1");
plot(lgraph);
actorNetwork=lgraph;
% dlnetwork(actorNetwork);
actorOptions = rlOptimizerOptions('LearnRate',0.1,'GradientThreshold',inf);
 actor(idx) = rlContinuousGaussianActor(actorNetwork,allObsInfo,ainfo);
    

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Emmanouil Tzorakoleftherakis 2024-2-12

The code you sent cannot be executed as a standalone. Also, I am getting different errors that what you sent. Please modify the code to reproduce the error you mentioned above.

请先登录，再进行评论。

请先登录，再回答此问题。

Reinforcement Learning Error with two scalar inputs

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

回答（0 个）

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

Reinforcement Learning Error with two scalar inputs

1 个评论 显示 -1更早的评论隐藏 -1更早的评论

回答（0 个）

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

1 个评论
显示 -1更早的评论隐藏 -1更早的评论