How to save an rl agent after every 1000 episodes?

Question

Guru Bhargava Khandavalli 2021-3-7

2
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/765296-how-to-save-an-rl-agent-after-every-1000-episodes

编辑： Lance 2023-6-29

I am training a DDPG agent where the training runs over 1000 episodes. To see how it evolves, I would like to save the agents after every 1000 episodes. As i see the options available in rlTrainingOptions, it is only possible to save every agent after a critical value. This slows down the training process significantly because saving every agent consumes a lot of time. Is there an efficient way to save the agents only after every 1000 episodes?

Thank you.

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Heesu Kim 2021-3-12

编辑：Heesu Kim 2021-3-19

I agree with this. I don't understand why it does not have the most useful option. And I'm disappointed that this question still doesn't have any answer.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Madhav Thakker 2021-3-19

2
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/765296-how-to-save-an-rl-agent-after-every-1000-episodes#answer_652737

Hi Guru,

I understand currently in rlTrainingOptions, there is no option to save the agent after specific number of episodes. I have raised an enhancement request for the same and this might be considered in future releases.

Hope this helps.

2 个评论
显示无隐藏无

Heesu Kim 2021-3-19

Yay!

Dmitriy Ogureckiy 2023-1-20

I am the same opinion. Add this, please. Ball on your side.

请先登录，再进行评论。

Answer 2

Manuel Sebastian Rios Beltran 2022-6-2

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/765296-how-to-save-an-rl-agent-after-every-1000-episodes#answer_977110

@Madhav Thakker But they did not do it :( a year later

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Matteo D'Ambrosio 2023-5-28

2 years later! Agreed that this enhancement is much needed...

请先登录，再进行评论。

Answer 3

Lance 2023-6-23

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/765296-how-to-save-an-rl-agent-after-every-1000-episodes#answer_1261513

编辑：Lance 2023-6-29

在 MATLAB Online 中打开

From what I understand, the only other work around would be to write another training command. You would have to predfine this for every "checkpoint" ie. 10,20,30 episodes. The training-progress graph will continue to be actively updated. (Note I am using R2022a)

% Define all agents, observations, actions, environment, etc....
maxepisodes=500;
trainingOpts=rlMultiAgentTrainingOptions;
trainingOpts.SaveAgentCriteria="EpisodeCount";
trainingOpts.SaveAgentValue=maxepisodes
trainingStats=train([agent1,agent2],environment,trainingOpts); % Will train to max episodes and save agent
% Edit Trainingoptions to increase maxepisodes and save agent value
trainingStats(1,1).TrainingOptions.MaxEpisodes=1000;
trainingStats(1,1).TrainingOptions.SaveAgentValue=[1000,1000];
trainnigStats(1,1).TrainingOptions.StopTrainingValue=[1000,1000];
trainingStats(1,2).TrainingOptions.MaxEpisodes=1000;
trainingStats(1,2).TrainingOptions.SaveAgentValue=[1000,1000];
trainnigStats(1,2).TrainingOptions.StopTrainingValue=[1000,1000];
% Resume training -- Will train to 1000 episodes and save agent again
trainingStats2=train([agent1,agent2],environment,trainingStats) %Note you use trainingStats here not trainingOpts