How to see actions when using the train() function in RL tool box.

Question

rtn 2022-5-24

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1726345-how-to-see-actions-when-using-the-train-function-in-rl-tool-box

回答： Emmanouil Tzorakoleftherakis 2022-6-2

Hi!

I use the train() function for my matlab env. I would like to also see the action input into the step function at each of the training steps. Is that possible? So far I only get the agent.

clear all
clc
close all
%load agent1.mat
ObservationInfo = rlNumericSpec([4  1501]);
ObservationInfo.Name = 'CartPole States';
ObservationInfo.Description = 'pendulum_force, cart position, cart velocity';
%ActionInfo = rlNumericSpec([1 1]);
ActionInfo = rlNumericSpec([2 1],'LowerLimit',[-1*pi/2;-1*pi],'UpperLimit',[pi/2;pi]);
ActionInfo.Name = 'CartPole Action';
env = rlFunctionEnv(ObservationInfo,ActionInfo,'myHapticStepFunction','myHapticResetFunction');
obsInfo = getObservationInfo(env);
actInfo = getActionInfo(env);
rng('shuffle')
%%
clc
load TD3_Haptic_MIMO
agent = TD3_Haptic_MIMO;
%Extract the deep neural networks from both the agent actor and critic.
actorNet = getModel(getActor(agent));
criticNet = getModel(getCritic(agent));
%Display the layers of the critic network, and verify that each hidden fully connected layer has 128 neurons
criticNet.Layers
% figure()
% plot(layerGraph(actorNet))
% figure()I wou
% plot(layerGraph(criticNet))
temp = getAction(agent,{rand(obsInfo.Dimension)})
%%
maxepisodes = 140;
maxsteps = 1;
trainOpts = rlTrainingOptions(...
    'MaxEpisodes',maxepisodes,...
    'MaxStepsPerEpisode',maxsteps,...
    'ScoreAveragingWindowLength',5,...
    'Verbose',false,...
    'Plots','training-progress',...
    'StopTrainingCriteria','AverageReward',...
    'StopTrainingValue',1.5, 'SaveAgentCriteria',"EpisodeReward",'SaveAgentValue',1.4',...
    'SaveAgentDirectory', pwd + "\Haptic\run1");
% LOOK INTO SAVING AGENT VALUE
% https://www.mathworks.com/help/reinforcement-learning/ug/train-reinforcement-learning-policy-using-custom-training.html
% https://www.mathworks.com/help/reinforcement-learning/ug/train-reinforcement-learning-agents.html
%https://www.mathworks.com/help/reinforcement-learning/ref/rltrainingoptions.html
trainingStats = train(agent,env,trainOpts);

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Emmanouil Tzorakoleftherakis 2022-6-2

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1726345-how-to-see-actions-when-using-the-train-function-in-rl-tool-box#answer_976940

Hello,

To log action data throughout an episode, you would need to do so from inside the step function of your environment. You can save actions into a mat/csv file as needed from there.

Hope this helps