Multi-Step (D)DQN using Parallelization

14 次查看（过去 30 天）
David Braun
David Braun 2022-9-5
回答： Ayush Modi 2023-10-20
I have noticed that with the current implementation of DQN it is not possible to use both multi-step returns (NumStepsToLookAhead>1) and parallelization. However I noticed that having multi-steps is essential for my application. Still, I would love to make use of all of my cpu cores.
Thus, I am wondering if it is possible to implement a custom DQN agent that allows for this. My goal is to arrive at an implementation where multiple workers generate experience samples. Learning is performed centrally and the updated policy is returned to the worker regularily.
Is this a reasonable idea? If yes, does anybody have an idea how I can implement this without generating too much code duplicates of the default dqn implementation?
Thank you very much.

请先登录，再回答此问题。

回答（1 个）

Ayush Modi
Ayush Modi 2023-10-20
Hi David,
As per my understanding, you would like to generate experience samples at worker nodes by training the model with local data and then send these model parameters to central server to train the central model.
You can achieve this by using the concept of Federated Learning.
Please refer to the following MathWorks documentation for more information on Federated Learning:
https://www.mathworks.com/help/deeplearning/ug/train-network-using-federated-learning.html
You can create a custom DQN/DDQN model as well.
I hope this resolves the issue you were facing.

类别

在 Help Center 和 File Exchange 中查找有关 Deep Learning Toolbox 的更多信息

标签

产品


版本

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by