How can I plot extra RL training data while using parallel computing?

3 次查看（过去 30 天）
Alvin Allen
Alvin Allen 2021-7-31
回答： Shadaab Siddiqie 2021-8-4
I'm working on a RL project using the Reinforcement Learning Designer, the PPO agent, and a custom environment. To help with diagnostics during the training process I set up some extra plots that track some statistics related to my agent's performance (separate from the RL designer's built in plot that shows reward, average reward, and q0). These plots are intended to capture statistics about the given episode regarding the final state of the environment as well as important moves made during the episode. This data is saved to environment properties inside the step() function during an episode, and then written to plots at the end of an episode in the reset() function.
For example,
classdef custEnv < rl.env.MATLABEnvironment
properties
numKeyMoves = 0;
numTotalMoves = 0;
numEpisodes = 0;
Figure1
end
methods
function [Observation, Reward, IsDone, LoggedSignals] = step(this, Action)
if 1 % if the move was a key move
this.numKeyMoves = this.numKeyMoves + 1;
end
this.numTotalMoves = this.numTotalMoves + 1;
end
function InitialObservation = reset(this)
ha = gca(this.Figure1);
% plot the percentage of key moves against episode num to track
% progress
plot(ha, this.numKeyMoves / this.numTotalMoves, this.episodeNum)
this.numKeyMoves = 0;
this.numTotalMoves = 0;
this.episodeNum = this.episodeNum + 1;
end
function plot(this)
this.Figure1 = figure('Visible', 'on', 'HandleVisibility', 'off');
end
end
end
This has worked well to date, but now I'd like to use parallel computing to speed up future training sessions. However it seems like the parallel workers aren't interacting with the step() and reset() functions in the same way, so the plots I've created are never getting populated with any data once parallel training is enabled. Is there a way to recreate the funcitonality above with parallel training, or any other stable way to similarly track environment properties as episodes progress?

请先登录，再回答此问题。

采纳的回答

Shadaab Siddiqie
Shadaab Siddiqie 2021-8-4
Form my understanding you are not getting correct plot when using parallel computing. I have heard that this issue is known, and the concerned parties may be investigating further.

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Environments 的更多信息

标签

产品


版本

R2021a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by