TD3 error message for using ltsm layer in Neural Network

4 次查看（过去 30 天）
Francesco Mogetti
Francesco Mogetti 2022-1-16
回答： Pratyush Roy 2022-1-19
Hi! I am trying to design a reinforcement learning model for landing mission on the moon in a defined region. I played with different Agents algorithm such as PPO, DDPG and TD3 to evaluate how they work differently.
With PPO I don't have problems related to error in the code or in the network architecture so at the moment I am working with it. The problem is when I try to use DDPG and TD3 with recurrent neural network, including an lstm layer in the architecture, I obtain the following error message:
Error using dlnetwork/predict (line 664)
Layer 'lstm': Invalid input data. Input data must contain a dimension labeled 'T' and must not contain any non-singleton dimensions labeled 'U'.
Error in rl.representation.model.rlDLNetworkModel/cacheNetworkSize (line 588)
[DummyOutput{:}] = predict(this.InternalNetwork,DummyInput{:},'Acceleration','none');
Error in rl.representation.model.rlDLNetworkModel (line 90)
this = cacheNetworkSize(this);
Error in rl.util.createInternalModelFactory (line 16)
Model = rl.representation.model.rlDLNetworkModel(Model, UseDevice, ObservationNames, ActionNames);
Error in rlDeterministicActorRepresentation (line 86)
Model = rl.util.createInternalModelFactory(Model, Options, ObservationNames, ActionNames, InputSize, OutputSize);
Error in agentCreator (line 236)
actor = rlDeterministicActorRepresentation(actnet,obsInfo,actInfo,"Observation","obs","Action","fcact",opts)
Error in main (line 26)
[agent] = agentCreator(numObs,obsInfo,obsInfocr,numAct,actInfo,'TD3_recurrent');
The Neural Network I am trying to implement is the following:
actnet = [featureInputLayer(numObs,"Name","obs");
fullyConnectedLayer(50,"Name","fc1");
fullyConnectedLayer(30,"Name","fc2");
reluLayer('Name','relu1');
lstmLayer(8,'OutputMode','sequence','Name','lstm')
fullyConnectedLayer(4,"Name","fcact")];
I looked for documentation but I cannot find any help.
Can someone clarify this for me?
Thaks!
  2 个评论
yanqi liu
yanqi liu 2022-1-17
yes，sir，may be upload your data mat file，then we can debug it
Francesco Mogetti
Francesco Mogetti 2022-1-17
编辑：Francesco Mogetti 2022-1-17
The code is nothing more than this:
numObs = 6;
numAct = 4;
obsInfo = rlNumericSpec([numObs 1]);
actInfo = rlNumericSpec([numAct 1]);
actInfo.UpperLimit = 1;
actInfo.LowerLimit = -1;
mdl = "EnvTD3";
env = rlSimulinkEnv(mdl,"EnvTD3/RL Agent",obsInfo,actInfo);
% env.ResetFcn = @ResetLanderCond2;
[agent] = agentCreator(numObs,obsInfo,numAct,actInfo,'TD3');
Then in the function "agentCreator" the neural network is defined both for actor and critic. The problem is related to the actor which is defined first:
actnet = [featureInputLayer(numObs,"Name","obs");
fullyConnectedLayer(50,"Name","fc1");
fullyConnectedLayer(30,"Name","fc2");
reluLayer('Name','relu1');
lstmLayer(8,'OutputMode','sequence','Name','lstm')
fullyConnectedLayer(4,"Name","fcact")];
opts = rlRepresentationOptions("Learnrate",5e-4,"GradientThreshold",10);
actor = rlDeterministicActorRepresentation(actnet,obsInfo,actInfo,"Observation","obs","Action","fcact",opts)
So the error in the command window:
Error using dlnetwork/predict (line 664)
Layer 'lstm': Invalid input data. Input data must contain a dimension labeled 'T' and must not contain any non-singleton dimensions labeled 'U'.
Error in rl.representation.model.rlDLNetworkModel/cacheNetworkSize (line 588)
[DummyOutput{:}] = predict(this.InternalNetwork,DummyInput{:},'Acceleration','none');
Error in rl.representation.model.rlDLNetworkModel (line 90)
this = cacheNetworkSize(this);
Error in rl.util.createInternalModelFactory (line 16)
Model = rl.representation.model.rlDLNetworkModel(Model, UseDevice, ObservationNames, ActionNames);
Error in rlDeterministicActorRepresentation (line 86)
Model = rl.util.createInternalModelFactory(Model, Options, ObservationNames, ActionNames, InputSize, OutputSize);
Error in agentCreator (line 236)
actor = rlDeterministicActorRepresentation(actnet,obsInfo,actInfo,"Observation","obs","Action","fcact",opts)
Error in main (line 26)
[agent] = agentCreator(numObs,obsInfo,obsInfocr,numAct,actInfo,'TD3_recurrent');

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Pratyush Roy
Pratyush Roy 2022-1-19
Hi Francesco,
The input format for the data which is passed to an lstmLayer can be either CBT (channel, batch, time), SCBT(spatial, channel, batch, time) or SSCBT(spatial,spatial, channel, batch, time) or SSSCBT(spatial, spatial, spatial, channel, batch, time) (please refer to this documentation).
So we should make sure that the data obtained from the previous layers is changed to the aforementioned formats before feeding that to the lstmLayer(please refer to the documentation here to more information on data formats).
Another workaround would be to change the architecture of the actor network by introducing a sequenceInputLayer followed by the lstmLayer, which might be followed by more layers.
Currently we are aware of this issue which shows the error message and our developers might address this in one of our future releases.
Hope this helps!

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Deep Learning Toolbox 的更多信息

标签

产品


版本

R2021b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by