Problems encountered in matlab reinforcement learning toolbox

Question

ying Zhao 2021-3-24

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/782278-problems-encountered-in-matlab-reinforcement-learning-toolbox

评论：成奥 2024-5-21

Hi,I encountered some problems when building a model with the reinforcement learning toolbox, and his error was as follows:

警告: 执行为类 rl.env.SimulinkEnvWithAgent 定义的事件 EpisodeFinished 的侦听程序回调时出现错误:

错误使用 rl.policy.AbstractPolicy/getAction (第 258 行)

Invalid observation type or size.

出错 rl.agent.rlDDPGAgent/evaluateQ0Impl (第 141 行)

action = getAction(this, observation);

出错 rl.agent.AbstractAgent/evaluateQ0 (第 275 行)

q0 = evaluateQ0Impl(this,observation);

出错 rl.train.TrainingManager/update (第 134 行)

q0 = evaluateQ0(this.Agents(idx),epinfo(idx).InitialObservation);

出错 rl.train.TrainingManager>@(info)update(this,info) (第 437 行)

trainer.FinishedEpisodeFcn = @(info) update(this,info);

出错 rl.train.Trainer/notifyEpisodeFinishedAndCheckStopTrain (第 56 行)

stopTraining = this.FinishedEpisodeFcn(info);

出错 rl.train.SeriesTrainer>iUpdateEpisodeFinished (第 31 行)

notifyEpisodeFinishedAndCheckStopTrain(this,ed.Data);

出错 rl.train.SeriesTrainer>@(src,ed)iUpdateEpisodeFinished(this,ed) (第 17 行)

@(src,ed) iUpdateEpisodeFinished(this,ed));

出错 rl.env.AbstractEnv/notifyEpisodeFinished (第 324 行)

notify(this,'EpisodeFinished',ed);

出错 rl.env.SimulinkEnvWithAgent/executeSimsWrapper/nestedSimFinishedBC (第 222 行)

notifyEpisodeFinished(this,...

出错 rl.env.SimulinkEnvWithAgent>@(src,ed)nestedSimFinishedBC(ed) (第 232 行)

simlist(1) = event.listener(this.SimMgr,'SimulationFinished' ,@(src,ed) nestedSimFinishedBC(ed));

出错 Simulink.SimulationManager/handleSimulationOutputAvailable

出错 Simulink.SimulationManager>@(varargin)obj.handleSimulationOutputAvailable(varargin{:})

出错 MultiSim.internal.SimulationRunnerSerial/executeImplSingle

出错 MultiSim.internal.SimulationRunnerSerial/executeImpl

出错 Simulink.SimulationManager/executeSims

出错 Simulink.SimulationManagerEngine/executeSims

出错 rl.env.SimulinkEnvWithAgent/executeSimsWrapper (第 244 行)

executeSims(this.SimEngine,simfh,in);

出错 rl.env.SimulinkEnvWithAgent/simWrapper (第 267 行)

simouts = executeSimsWrapper(this,in,simfh,simouts,opts);

出错 rl.env.SimulinkEnvWithAgent/simWithPolicyImpl (第 424 行)

simouts = simWrapper(env,policy,simData,in,opts);

出错 rl.env.AbstractEnv/simWithPolicy (第 82 行)

[experiences,varargout{1:(nargout-1)}] = simWithPolicyImpl(this,policy,opts,varargin{:});

出错 rl.task.SeriesTrainTask/runImpl (第 33 行)

[varargout{1},varargout{2}] = simWithPolicy(this.Env,this.Agent,simOpts);

出错 rl.task.Task/run (第 21 行)

[varargout{1:nargout}] = runImpl(this);

出错 rl.task.TaskSpec/internal_run (第 166 行)

[varargout{1:nargout}] = run(task);

出错 rl.task.TaskSpec/runDirect (第 170 行)

[this.Outputs{1:getNumOutputs(this)}] = internal_run(this);

出错 rl.task.TaskSpec/runScalarTask (第 194 行)

runDirect(this);

出错 rl.task.TaskSpec/run (第 69 行)

runScalarTask(task);

出错 rl.train.SeriesTrainer/run (第 24 行)

run(seriestaskspec);

出错 rl.train.TrainingManager/train (第 421 行)

run(trainer);

出错 rl.train.TrainingManager/run (第 211 行)

train(this);

出错 rl.agent.AbstractAgent/train (第 78 行)

TrainingStatistics = run(trainMgr);

出错 simulink_pmsm_ddpg (第 75 行)

trainingStats = train(agent,env,trainingOptions);

原因:

错误使用 rl.representation.rlAbstractRepresentation/validateInputData (第 525 行)

Input data dimensions must match the dimensions specified in the corresponding observation and action info specifications.

> 位置：rl.env/AbstractEnv/notifyEpisodeFinished (第 324 行)

位置: rl.env.SimulinkEnvWithAgent.executeSimsWrapper/nestedSimFinishedBC (第 222 行)

位置: rl.env.SimulinkEnvWithAgent>@(src,ed)nestedSimFinishedBC(ed) (第 232 行)

位置: Simulink/SimulationManager/handleSimulationOutputAvailable

位置: Simulink.SimulationManager>@(varargin)obj.handleSimulationOutputAvailable(varargin{:})

位置: MultiSim.internal/SimulationRunnerSerial/executeImplSingle

位置: MultiSim.internal/SimulationRunnerSerial/executeImpl

位置: Simulink/SimulationManager/executeSims

位置: Simulink/SimulationManagerEngine/executeSims

位置: rl.env/SimulinkEnvWithAgent/executeSimsWrapper (第 244 行)

位置: rl.env/SimulinkEnvWithAgent/simWrapper (第 267 行)

位置: rl.env/SimulinkEnvWithAgent/simWithPolicyImpl (第 424 行)

位置: rl.env/AbstractEnv/simWithPolicy (第 82 行)

位置: rl.task/SeriesTrainTask/runImpl (第 33 行)

位置: rl.task/Task/run (第 21 行)

位置: rl.task/TaskSpec/internal_run (第 166 行)

位置: rl.task/TaskSpec/runDirect (第 170 行)

位置: rl.task/TaskSpec/runScalarTask (第 194 行)

位置: rl.task/TaskSpec/run (第 69 行)

位置: rl.train/SeriesTrainer/run (第 24 行)

位置: rl.train/TrainingManager/train (第 421 行)

位置: rl.train/TrainingManager/run (第 211 行)

位置: rl.agent.AbstractAgent/train (第 78 行)

位置: simulink_pmsm_ddpg (第 75 行)

错误使用 rl.env.AbstractEnv/simWithPolicy (第 82 行)

An error occurred while simulating "rlpmsmSimscapeModel" with the agent "agent".