Reinforcement learning and Paralle computation

ryunosuke tazawa

2022 4 13

0 个回答

4 次查看（30 天）

0 个投票

I am condcting reinforcement learning with Sac agent.

I tried to use GPU and Parallel computation, but in case of using paralle computaion, the training result changed.

At all, learning is much worse than without parallel processing.　

Do you know what caused it?

%% AGENT　setting
agentOptions = rlSACAgentOptions;
agentOptions.SampleTime = Ts;
agentOptions.DiscountFactor = 0.90;                       
agentOptions.TargetSmoothFactor = 1e-3;                   
agentOptions.ExperienceBufferLength = 500;                
agentOptions.MiniBatchSize = 256;                          
agentOptions.EntropyWeightOptions.TargetEntropy = -2;     
agentOptions.NumStepsToLookAhead  = 1;
agentOptions.ResetExperienceBufferBeforeTraining = false;
agent = rlSACAgent(actor,[critic1 critic2],agentOptions);
 
 
%% Learning setting
maxepisodes = 10000;                             
maxsteps = 1e6;                                  
trainingOptions = rlTrainingOptions(...
    'MaxEpisodes',maxepisodes,...
    'MaxStepsPerEpisode',maxsteps,...
    'StopOnError','on',...
    'Verbose',true,...
    'Plots','training-progress',...
    'StopTrainingCriteria','AverageReward',...
    'StopTrainingValue',Inf,...
    'ScoreAveragingWindowLength',10); 
trainingOptions.UseParallel = true;
trainingOptions.ParallelizationOptions.Mode = 'async';
trainingOptions.ParallelizationOptions.StepsUntilDataIsSent  = 32;
trainingOptions.ParallelizationOptions.DataToSendFromWorkers =  'Experiences';

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Takeshi Takahashi 2022-4-18

agentOptions.ExperienceBufferLength seems too short, which may indirectly affect the parallel training. Can you increase ExperienceBufferLength to 1e6 or more?

请先登录，再进行评论。

请先登录，再回答此问题。

Follow Question