Debug mode for RL agent networks

1 次查看（过去 30 天）
Nicolas CRETIN
Nicolas CRETIN 2024-7-8
评论： Nicolas CRETIN 2024-7-19
Is there any way to enter debug mode, to see what is happening inside the RL agent nets while training is running?
Some of my layers output NaN and I would like to know which one. I would also like to monitor to outputs of each layer.
Thanks in advance!
Nicolas
  1 个评论
Nicolas CRETIN
Nicolas CRETIN 2024-7-18
It would maybe be a solution to add output layers to the network and display their result, but it's tedious.

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Shantanu Dixit
Shantanu Dixit 2024-7-19
编辑：Shantanu Dixit 2024-7-19
Hi Nicolas,
It is my understanding that you want to monitor the outputs of each layer and debug the RL agent nets while the training is running.
Similar to analyzing deep learning networks you can call the forward method for each layer of the agent's network to analyze the corresponding output during the training.
[Y1,...,YK] = forward(___,'Outputs',layerNames)
here 'layerNames' correspond to a string array, with 'layerNames(k)' representing kth layer of the agent's network, the corresponding outputs are stored in 'Yk'
Briefly you can follow the below steps:
  • Extract layer names from the actor network into a string array 'layerNames'
  • Convert Observation Buffer to dlarray for processing into the network
  • Forward Pass Through Each Layer using the forward method.
Refer the below code for monitoring the layer outputs corresponding to one observation after updating the actor
%% actorNetwork refers to the DNN of the actor
%% for one observation
dlX = dlarray(observationBuffer(:,:,1), 'CB'); %% format in which the network takes the input
[Y1, Y2, Y3, Y4, Y5, Y6] = forward(actorNetwork, dlX, 'Outputs', layerNames);
layerOutputs = {Y1, Y2, Y3, Y4, Y5, Y6};
for i = 1:numel(layerOutputs)
% disp(['Layer ', layerNames(i), ' output:']);
if any(isnan(extractdata(layerOutputs{i})), 'all')
disp(['Layer ', num2str(i), ' output contains NaNs']); %% check for NaNs
end
end
For a better understanding on forward pass and the custom training loop procedure, refer to the following MathWorks documentation

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Deep Learning Toolbox 的更多信息

标签

产品


版本

R2023a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by