I want to print out multiple actions in reinforcement learning
3 次查看(过去 30 天)
显示 更早的评论
I want to print out multiple actions in reinforcement learning
this is agent code
function agent = createAgent(observationInfo,actionInfo,Ts)
L = 16; % number of neurons
statePath = [
featureInputLayer(observationInfo.Dimension(1),'Normalization','none','Name','observation')
%fullyConnectedLayer(L,'Name','fc1')
%reluLayer('Name','relu1')
fullyConnectedLayer(L,'Name','fc2')
additionLayer(2,'Name','add')
reluLayer('Name','relu2')
%fullyConnectedLayer(L,'Name','fc3')
%reluLayer('Name','relu3')
fullyConnectedLayer(1,'Name','fc4')];
actionPath = [
featureInputLayer(actionInfo.Dimension(1),'Normalization','none','Name','action')
fullyConnectedLayer(L, 'Name', 'fc5')];
criticNetwork = layerGraph(statePath);
criticNetwork = addLayers(criticNetwork, actionPath);
criticNetwork = connectLayers(criticNetwork,'fc5','add/in2');
criticNetwork = dlnetwork(criticNetwork);
criticOptions = rlOptimizerOptions('LearnRate',1e-3,'GradientThreshold',1);
critic = rlQValueFunction(criticNetwork,observationInfo,actionInfo,...
'ObservationInputNames','observation','ActionInputNames','action');
actorNetwork = [
featureInputLayer(observationInfo.Dimension(1),'Normalization','none','Name','observation')
fullyConnectedLayer(L,'Name','fc1')
reluLayer('Name','relu1')
%fullyConnectedLayer(L,'Name','fc2')
%reluLayer('Name','relu2')
%fullyConnectedLayer(L,'Name','fc3')
%reluLayer('Name','relu3')
fullyConnectedLayer(1,'Name','fc4')
tanhLayer('Name','tanh1')
scalingLayer('Name','ActorScaling1','Scale',2.5,'Bias',-0.5)];
actorNetwork = dlnetwork(actorNetwork);
actorOptions = rlOptimizerOptions('LearnRate',1e-3,'GradientThreshold', 1);
actor = rlContinuousDeterministicActor(actorNetwork,observationInfo,actionInfo);
agentOptions = rlDDPGAgentOptions(...
'SampleTime',Ts,...
'CriticOptimizerOptions',criticOptions,...
'ActorOptimizerOptions',actorOptions,...
'ExperienceBufferLength',1e6);
agentOptions.NoiseOptions.Variance = 0.6;
agentOptions.NoiseOptions.VarianceDecayRate = 1e-5;
agent = rlDDPGAgent(actor,critic,agentOptions);
This is run code
%%
clear;
clc;
%% Create Environment
% load environment parameters
parameters
mdl = 'my_CAV_slx';
% open_system(mdl);
obsInfo = rlNumericSpec([10 1]);
actInfo = rlNumericSpec([1 1], 'LowerLimit', accMin, 'UpperLimit', accMax);
blks = mdl + "/RL Agent1";
env = rlSimulinkEnv(mdl, blks, obsInfo, actInfo);
% set reset function
env.ResetFcn = @(in)ResetParamFcn(in);
%% Create Agents
% fix seed
rng(0)
% longitudinal control
agent1 = createAgent(obsInfo, actInfo, Ts);
%% Train Agents
trainingOpts = rlTrainingOptions('MaxEpisodes',maxEpisodes,'MaxStepsPerEpisode',maxSteps,'Verbose',false,'Plots','training-progress','StopTrainingCriteria','AverageReward');
doTraining = true;
if doTraining
% Train the agent.
trainingStats = train(agent1, env, trainingOpts);
save('my_CAV.mat', 'agent1')
else
% Load pretrained agents for the example.
load('my_CAV.mat')
end
%% Simulate Agents
simOptions = rlSimulationOptions('MaxSteps',maxSteps);
experience = sim(env, agent1, simOptions);
2 个评论
Emmanouil Tzorakoleftherakis
2023-7-10
I am not sure what you mean by print out. Do you want your agent to output multiple actions? Or do you want to know how to do inference with the agent you created above and see what the output action is?
回答(1 个)
Emmanouil Tzorakoleftherakis
2023-7-11
Hi,
If you want to create an agent that outputs multiple actions, you need to make sure the actor network is set up accordingly. I would recommend using the 'default agent' feature as shown here. In summary, this feature creates a default network architecture for your actor and critic based on the observation and action spaces you provide. If your action space indicates that you have multiple actions, the actor and critic that will be created will reflect that. You can use this shortcut as a way to get an initial design which you can then tweak as needed.
The link I shared above also shows you how to perform inference on the actor/agent you already created using 'getAction'.
Hope that helps
0 个评论
另请参阅
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!