How can I extract a trained RL Agent's network's weights and biases?

23 次查看（过去 30 天）

显示更早的评论

rakbar 2020-3-26

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/513136-how-can-i-extract-a-trained-rl-agent-s-network-s-weights-and-biases

评论：轩 2024-1-5

采纳的回答： Anh Tran

在 MATLAB Online 中打开

How can I extract a trained RL Agent's network's weights and biases?

My network is:

statePath = [
    imageInputLayer([numObservations 1 1], 'Normalization', 'none', 'Name', 'state')
    fullyConnectedLayer(NumNeuron, 'Name', 'CriticStateFC1')
    reluLayer('Name', 'CriticRelu1')
    fullyConnectedLayer(NumNeuron, 'Name', 'CriticStateFC2')];
actionPath = [
    imageInputLayer([1 1 1], 'Normalization', 'none', 'Name', 'action')
    fullyConnectedLayer(NumNeuron, 'Name', 'CriticActionFC1')
    reluLayer('Name', 'ActorRelu1')
    fullyConnectedLayer(NumNeuron, 'Name', 'CriticActionFC2')];
commonPath = [
    additionLayer(2,'Name', 'add')
    reluLayer('Name','CriticCommonRelu')
    fullyConnectedLayer(1, 'Name', 'output')];
criticNetwork = layerGraph(statePath);
criticNetwork = addLayers(criticNetwork, actionPath);
criticNetwork = addLayers(criticNetwork, commonPath);    
criticNetwork = connectLayers(criticNetwork,'CriticStateFC2','add/in1');
criticNetwork = connectLayers(criticNetwork,'CriticActionFC2','add/in2');
% set some options for the critic
criticOpts = rlRepresentationOptions('LearnRate',learing_rate,...
                                      'GradientThreshold',1);
% create the critic based on the network approximator
critic = rlQValueRepresentation(criticNetwork,obsInfo,actInfo,...
    'Observation',{'state'},'Action',{'action'},criticOpts);
agent = rlDQNAgent(critic,agentOpts)
trainingStats = train(agent,env,trainOpts);

After training, I'd like to get the network's trained weights and biases.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Anh Tran 2020-3-27

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/513136-how-can-i-extract-a-trained-rl-agent-s-network-s-weights-and-biases#answer_422395

编辑：Anh Tran 2020-3-27

在 MATLAB Online 中打开

You can get the parameters from the trained's critic representation for DQN agent. In MATLAB R2020a, see getLearnableParameters and getCritic functions (function name changes a bit since R2019b). You can follow similar steps to get the actor's parameters from actor-based agent like DDPG or PPO.

critic = getCritic(agent);
criticParams = getLearnableParameters(critic);

6 个评论
显示 4更早的评论隐藏 4更早的评论

Francisco Serra 2023-12-14

@rakbar @Dmitriy Ogureckiy have you found a way of getting the weights after each training episode?

轩 2024-1-5

@Francisco Serra I have the same need. I find a silly method: save the agent after each episode and use "getLearnableParameters" to print the parameter of each agent.

请先登录，再进行评论。

更多回答（0 个）

请先登录，再回答此问题。

类别

AI and Statistics Deep Learning Toolbox Applications Autonomous and Control Systems Reinforcement Learning

在 Help Center 和 File Exchange 中查找有关 Reinforcement Learning 的更多信息

标签

产品

Reinforcement Learning Toolbox

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Translated by