How to set multiple stopping or saving criteria for RL agent?

Question

Yihao Wan 2023-5-26

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1974039-how-to-set-multiple-stopping-or-saving-criteria-for-rl-agent

评论： Yihao Wan 2023-6-29

Hello, I wondered if it is possible to set multiple stopping or saving criteria for RL agent? E.g. Save the agent for average episode reward after 100 episodes.

Thanks in advance.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Emmanouil Tzorakoleftherakis 2023-5-26

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1974039-how-to-set-multiple-stopping-or-saving-criteria-for-rl-agent#answer_1245474

编辑：Emmanouil Tzorakoleftherakis 2023-5-26

This is currently not possible but keep an eye out on future releases - the development team has been working on this functionality

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

Answer 2

Lance 2023-6-23

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1974039-how-to-set-multiple-stopping-or-saving-criteria-for-rl-agent#answer_1261543

编辑：Lance 2023-6-29

在 MATLAB Online 中打开

From what I understand, the only other work around would be to write another training command. You would have to predfine this for every "checkpoint" ie. 10,20,30 episodes. The training-progress graph will continue to be actively updated. (Note I am using R2022a)

% Define all agents, observations, actions, environment, etc....
maxepisodes=500;
trainingOpts=rlMultiAgentTrainingOptions;
trainingOpts.SaveAgentCriteria="EpisodeCount";
trainingOpts.SaveAgentValue=maxepisodes
trainingStats=train([agent1,agent2],environment,trainingOpts); % Will train to max episodes and save agent
% Edit Trainingoptions to increase maxepisodes and save agent value
trainingStats(1,1).TrainingOptions.MaxEpisodes=1000;
trainingStats(1,1).TrainingOptions.SaveAgentValue=[1000,1000];
trainnigStats(1,1).TrainingOptions.StopTrainingValue=[1000,1000];
trainingStats(1,2).TrainingOptions.MaxEpisodes=1000;
trainingStats(1,2).TrainingOptions.SaveAgentValue=[1000,1000];
trainnigStats(1,2).TrainingOptions.StopTrainingValue=[1000,1000];
% Resume training -- Will train to 1000 episodes and save agent again
trainingStats2=train([agent1,agent2],environment,trainingStats) %Note you use trainingStats here not trainingOpts