Is there a way to visualize simulation result (after RL agent training) at the command line?

5 次查看（过去 30 天）
Nicolas CRETIN
Nicolas CRETIN 2024-5-7
编辑： Nicolas CRETIN 2024-5-17
Hi!
Is there any way to visualize the result of a simulation at the command line, given a simulation result (as on the following picture for example)?
I mean, I can run the following lines to get the simulation result, but how could I then display it? How should I for example open the Reinforcement Learning Designer app at the command line and then 'plot' the simResults?
% Perform simulations
simResults = sim(env,agentPPO,simOptions);
Is there also a way of registering the "experience1" variable in the workspace?

请先登录，再回答此问题。

采纳的回答

Neha
Neha 2024-5-14
编辑：Neha 2024-5-14
Hi Nicolas,
I understand you're looking to export simulation results from the Reinforcement Learning Designer app to the workspace for visualization. Here's how you can do it:
  1. After obtaining the results in the app, click on "Save Session" within the "Reinforcement Learning" tab. This action saves a .mat file in your workspace.
  2. Access the results through the variable: "RLDesignerSession.Data.SimResults". This variable is a struct array containing all the experiments.
  3. For instance, to access the reward timeseries data for the first simulation of the first experiment, use: "RLDesignerSession.Data.SimResults(1).Data(1).Reward".
  4. To visualize the results, sum up the rewards for each simulation to obtain the cumulative reward. This data can then be used to plot a bar graph similar to the one in the Reinforcement Learning Designer App.
You can refer to the following documentation link for more information on the fields of the "RLDesignerSession.Data.SimResults" struct:
https://www.mathworks.com/help/reinforcement-learning/ref/rl.env.abstractenv.sim.html#:~:text=see%20rlSimulationOptions.-,Output%20Arguments,-collapse%20all
  1 个评论
Nicolas CRETIN
Nicolas CRETIN 2024-5-17
编辑：Nicolas CRETIN 2024-5-17
Thank you so much Neha!
Here is the code I finally used if someone is interrested (and the graphical result, in the case where the agent has been very poorly trained):
% -------- perform training --------
trainingStats = train(agentPPO, env, trainOpts);
% -------- perform the simulation to double check the training --------
% Create simulation options
simOptions = rlSimulationOptions();
simOptions.MaxSteps = 10;
simOptions.NumSimulations = 8;
% Perform simulations
simResults = sim(env,agentPPO,simOptions);
% plot reward for these simulations
sizeSimResults = size(simResults);
cumulativeReward = zeros(sizeSimResults);
for i = 1:sizeSimResults(1)
cumulativeReward(i) = sum(simResults(1).Reward.Data);
end
simIndex = 1:sizeSimResults(1);
meanReward = mean(cumulativeReward) * ones(sizeSimResults);
figure('Name', 'Simulation results');
bar(simIndex, cumulativeReward);
hold on
plot(simIndex, meanReward,'r--')
hold off
xlabel('Simulation index');
ylabel('Cumulative reward');
title('Cumulative rewards for each simulation');
legend('Cumulative Reward', 'Mean Reward');
% let's admit that simulation ii has a bad reward
% we want to display the output speed for this episode, to get to know what
% happened
ii = 1;
figure("Name", "Investigate bad simulation reward")
plot(simResults(ii).SimulationInfo.logsout{4}.Values.Time, ...
simResults(ii).SimulationInfo.logsout{4}.Values.Data)
hold on
plot(simResults(ii).SimulationInfo.logsout{5}.Values.Time, ...
simResults(ii).SimulationInfo.logsout{5}.Values.Data)
hold off
xlabel('time (seconds)');
ylabel('output speed (tr/min)');
title('Investigate bad simulation reward');
legend('reference', 'output speed');

请先登录，再进行评论。

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Training and Simulation 的更多信息

标签

产品


版本

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by