Logging needed Information while training a Reinforcement learning agent.

Question

Apstergo 2024-3-1

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2089076-logging-needed-information-while-training-a-reinforcement-learning-agent

评论： Apstergo 2024-3-2

Hello everyone

Currently I am using RL to develop an algorithm to optimize 3D movementof a UAV using certaint models and calculations. some of the data are used for reward and some are not but they are important for my research. I found that i could save those in loggedsignals, but as it gets bigger, it slows the training after a while, although Its useful for one episode or one simulation. Is there a better way to log or save these unused yet important data?

Thank you

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Ari Biswas 2024-3-1

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2089076-logging-needed-information-while-training-a-reinforcement-learning-agent#answer_1419721

在 MATLAB Online 中打开

Unfortunately there is no straightforward way to do this currently but we may have a solution in the upcoming releases (stay tuned).

For now you can add the following code in your environment reset function. It will save the logged signals to disk and remove it from memory to improve performance.

currentDir = pwd;
env.ResetFcn = @(in) myResetFcn(in, currentDir);
 
function in = myResetFcn(in, currentDir)
    % your reset code...
    % create a unique name for each episode.
    % For parallel training use uuid to avoid incorrect episode indices.
    % s = matlab.lang.internal.uuid;
    s = string(datetime("now"),"yyyyMMddhhmmss");
    filename = fullfile(currentDir, "loggedData_" + s + ".mat");
    
    % process the logged data post episode simulation
    in = in.setPostSimFcn(@(x) myPostSim(x, filename));
end
 
function new_out = myPostSim(out, filename)  
    % save the logged data to disk     
    % "logsout" is the name specified in your model settings > Data Import/Export > Signal logging
    loggedData = out.logsout;  
    save(filename, "loggedData");
    % remove logged data from out
    new_out = out;
    new_out.logsout = [];
    new_out.tout = []
end