Reinforcement Learning experience buffer length and parallelisation toolbox

Question

Tech Logg Ding 2020-12-2

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/673448-reinforcement-learning-experience-buffer-length-and-parallelisation-toolbox

编辑： Emmanouil Tzorakoleftherakis 2020-12-3

When parallelisation is used when training a DDPG agent with the following settings:

trainOpts.UseParallel = true;
trainOpts.ParallelizationOptions.Mode = 'async';
trainOpts.ParallelizationOptions.StepsUntilDataIsSent = -1;
trainOpts.ParallelizationOptions.DataToSendFromWorkers = 'Experiences';

Does the the parallel simulations have their own experience buffer? This could take up more memory hence I am hoping that only one experience buffer is stored to update the critic network.

From the documentations, it seems like there will only be one experience buffer as the experiences are sent back to the host.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Emmanouil Tzorakoleftherakis 2020-12-3

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/673448-reinforcement-learning-experience-buffer-length-and-parallelisation-toolbox#answer_564503

编辑：Emmanouil Tzorakoleftherakis 2020-12-3

Hello,

There is one big experience buffer on the host, the size of which you determine as usual in your agent options. Each worker has a much smaller buffer to collect experiences until you reach "StepsUntilDataIsSent".