reinforcement learning toolbox - q table

11 次查看（过去 30 天）
Xinpeng Wang
Xinpeng Wang 2019-7-10
回答： Tuong Nguyen 2022-10-7
I'm a newbie to RL and the RL toolbox. I played with Q-learning agent with a model in simulink. My question is after training, How can I access to the trained Q table? The qTable used to generate the agent is all ZERO. I cannot figure out where the trained Q values and the policies are stored. Thank you!

请先登录，再回答此问题。

采纳的回答

Emmanouil Tzorakoleftherakis
Hi Xinpeng,
To see the trained table, you have to do is extract it using ‘getCritic’. Try:
critic = getCritic(agent);
The variable ‘critic’ has a field which contains the Qtable after training.

更多回答（5 个）

carlos pedreira
carlos pedreira 2020-1-13
OK, but, after that, HOW CAN I SEE the table....
Shikhar Sharma
Shikhar Sharma 2020-1-24
It should appear under the Workspace tab.
Umut Can Akdag
Umut Can Akdag 2020-5-18
For those who are still looking for the q table I think this is the solution.
critic = getCritic(agent);
qtable = getLearnableParameters(critic);
RUBEN HERNANDEZ
RUBEN HERNANDEZ 2022-4-19
Hi everyone
I want to simulate Q-learning agent for control inverted pendulum in simulink (with Q-table) (just for ilustrative example)
I've picked the rlsimplependulumModel.slx predefined in matlab.
This is my code
mdl = 'rlSimplePendulumModel';
open_system(mdl)
obsInfo = rlNumericSpec([3 1]); % vector of 3 observations: sin(theta), cos(theta), d(theta)/dt
actInfo = rlFiniteSetSpec([-2 0 2]); % 3 possible values for torque: -2 Nm, 0 Nm and 2 Nm
obsInfo.Name = 'observations';
actInfo.Name = 'torque';
agentBlk = [mdl '/RL Agent'];
env = rlSimulinkEnv(mdl,agentBlk,obsInfo,actInfo);
env.ResetFcn = @(in)setVariable(in,'theta0',pi,'Workspace',mdl);
Ts = 0.05; % simulation time
Tf = 20; % sample time
% Fix the random generator seed for reproducibility
rng(0)
%% To create a Q-learning agent:
%1 Create a critic using an rlQValueRepresentation object.
qTable = rlTable(obsInfo, actInfo);
qRepresentation = rlQValueRepresentation(qTable, obsInfo, actInfo);
qRepresentation.Options.LearnRate = 0.99;
%% 2 Specify agent options using an rlQAgentOptions object.
agentOpts = rlQAgentOptions;
agentOpts.DiscountFactor = 0.99;
agentOpts.EpsilonGreedyExploration.Epsilon = 0.9;
agentOpts.EpsilonGreedyExploration.EpsilonDecay = 0.01;
%% 3 Create the agent using an rlQAgent object.
qAgent = rlQAgent(qRepresentation,agentOpts);
%% Training Algorithm
% rlQAgentOptions.
trainOpts = rlTrainingOptions;
trainOpts.MaxStepsPerEpisode = ceil(Tf/Ts);
trainOpts.MaxEpisodes = 2000;
trainOpts.StopTrainingCriteria = "AverageReward";
trainOpts.StopTrainingValue = -740;
trainOpts.ScoreAveragingWindowLength = 5;
trainingStats = train(qAgent,env,trainOpts);
AND THIS IS THE ERROR MESSAGE
Error using rlTable/validateInput (line 131)
Input must be a scalar rlFiniteSetSpec.
Error in rlTable (line 51)
validateInput(obj, ObservationInfo)
Error in qlearningpendulum (line 30)
qTable = rlTable(obsInfo, actInfo);
any suggestions?
Tuong Nguyen
Tuong Nguyen 2022-10-7
I think to use tabular Q learning, your observation has to be discrete and finite. That means your obsInfo has to be rlFiniteSetSpec(allStates), where in "allStates" you list out all the possible observations. See https://www.mathworks.com/help/reinforcement-learning/ref/rltable.html for the rlTable and https://www.mathworks.com/help/reinforcement-learning/ref/rl.util.rlfinitesetspec.html for the rlFiniteSetSpec.

类别

在 Help Center 和 File Exchange 中查找有关 Reinforcement Learning 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by