reinforcement learning toolbox - q table

2019 7 10
6 个回答
回答已采纳
更新时间：2022 10 7
7 次查看（30 天）

0 个投票

I'm a newbie to RL and the RL toolbox. I played with Q-learning agent with a model in simulink. My question is after training, How can I access to the trained Q table? The qTable used to generate the agent is all ZERO. I cannot figure out where the trained Q values and the policies are stored. Thank you!

请先登录，再回答此问题。

 采纳的回答

1 个投票

Hi Xinpeng,
To see the trained table, you have to do is extract it using ‘getCritic’. Try:
critic = getCritic(agent);
The variable ‘critic’ has a field which contains the Qtable after training.

更多回答（5 个）

carlos pedreira
carlos pedreira 2020-1-13

0 个投票

OK, but, after that, HOW CAN I SEE the table....
Umut Can Akdag
Umut Can Akdag 2020-5-18

0 个投票

For those who are still looking for the q table I think this is the solution.
critic = getCritic(agent);
qtable = getLearnableParameters(critic);
RUBEN HERNANDEZ
RUBEN HERNANDEZ 2022-4-19

0 个投票

Hi everyone
I want to simulate Q-learning agent for control inverted pendulum in simulink (with Q-table) (just for ilustrative example)
I've picked the rlsimplependulumModel.slx predefined in matlab.
This is my code
mdl = 'rlSimplePendulumModel';
open_system(mdl)
obsInfo = rlNumericSpec([3 1]); % vector of 3 observations: sin(theta), cos(theta), d(theta)/dt
actInfo = rlFiniteSetSpec([-2 0 2]); % 3 possible values for torque: -2 Nm, 0 Nm and 2 Nm
obsInfo.Name = 'observations';
actInfo.Name = 'torque';
agentBlk = [mdl '/RL Agent'];
env = rlSimulinkEnv(mdl,agentBlk,obsInfo,actInfo);
env.ResetFcn = @(in)setVariable(in,'theta0',pi,'Workspace',mdl);
Ts = 0.05; % simulation time
Tf = 20; % sample time
% Fix the random generator seed for reproducibility
rng(0)
%% To create a Q-learning agent:
%1 Create a critic using an rlQValueRepresentation object.
qTable = rlTable(obsInfo, actInfo);
qRepresentation = rlQValueRepresentation(qTable, obsInfo, actInfo);
qRepresentation.Options.LearnRate = 0.99;
%% 2 Specify agent options using an rlQAgentOptions object.
agentOpts = rlQAgentOptions;
agentOpts.DiscountFactor = 0.99;
agentOpts.EpsilonGreedyExploration.Epsilon = 0.9;
agentOpts.EpsilonGreedyExploration.EpsilonDecay = 0.01;
%% 3 Create the agent using an rlQAgent object.
qAgent = rlQAgent(qRepresentation,agentOpts);
%% Training Algorithm
% rlQAgentOptions.
trainOpts = rlTrainingOptions;
trainOpts.MaxStepsPerEpisode = ceil(Tf/Ts);
trainOpts.MaxEpisodes = 2000;
trainOpts.StopTrainingCriteria = "AverageReward";
trainOpts.StopTrainingValue = -740;
trainOpts.ScoreAveragingWindowLength = 5;
trainingStats = train(qAgent,env,trainOpts);
AND THIS IS THE ERROR MESSAGE
Error using rlTable/validateInput (line 131)
Input must be a scalar rlFiniteSetSpec.
Error in rlTable (line 51)
validateInput(obj, ObservationInfo)
Error in qlearningpendulum (line 30)
qTable = rlTable(obsInfo, actInfo);
any suggestions?
Tuong Nguyen
Tuong Nguyen 2022-10-7

0 个投票

I think to use tabular Q learning, your observation has to be discrete and finite. That means your obsInfo has to be rlFiniteSetSpec(allStates), where in "allStates" you list out all the possible observations. See https://www.mathworks.com/help/reinforcement-learning/ref/rltable.html for the rlTable and https://www.mathworks.com/help/reinforcement-learning/ref/rl.util.rlfinitesetspec.html for the rlFiniteSetSpec.

类别

在 帮助中心 和 File Exchange 中查找有关 Applications 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by