How to save pretrained DQN agent and extract the weights inside the network?

Question

Kuan Yi Li 2024-8-28

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2148334-how-to-save-pretrained-dqn-agent-and-extract-the-weights-inside-the-network

编辑： praguna manvi 2024-8-29

The following is part of the program. I want to know how to extract the weight values from the trained DQN network.

DQNnet = [
    imageInputLayer([1 520 1],"Name","ImageFeatureInput","Normalization","none")
    fullyConnectedLayer(1024,"Name","fc1")
    reluLayer("Name","relu1")
    % fullyConnectedLayer(512,"Name","fc2")
    % reluLayer("Name","relu2")
    fullyConnectedLayer(14,"Name","fc3")
    softmaxLayer("Name","softmax")
    classificationLayer("Name","ActionOutput")];
ObsInfo = getObservationInfo(env);
ActInfo = getActionInfo(env);
DQNOpts = rlRepresentationOptions('LearnRate',0.0001,'GradientThreshold',1,'UseDevice','gpu');
DQNagent = rlQValueRepresentation(DQNnet,ObsInfo,ActInfo,'Observation',{'ImageFeatureInput'},'ActionInputNames',{'BoundingBox Actions'},DQNOpts);
agentOpts = rlDQNAgentOptions(...
    'UseDoubleDQN',true ...
    ,'MiniBatchSize',256);
agentOpts.EpsilonGreedyExploration.Epsilon = 1;
agent = rlDQNAgent(DQNagent,agentOpts);
%% Agent Training
% Training options
trainOpts = rlTrainingOptions(...
    'MaxEpisodes', 100, ...
    'MaxStepsPerEpisode', 100, ...
    'Verbose', true, ...
    'Plots','training-progress',...
    'ScoreAveragingWindowLength',400,...
    'StopTrainingCriteria','AverageSteps',...
    'StopTrainingValue',1000000000,...
    'SaveAgentDirectory', pwd + "\agents\");
% Agent training
trainingStats = train(agent,env,trainOpts);

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

praguna manvi 2024-8-28

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2148334-how-to-save-pretrained-dqn-agent-and-extract-the-weights-inside-the-network#answer_1506584

编辑：praguna manvi 2024-8-29

在 MATLAB Online 中打开

Hi @Kuan Yi Li,

For saving and loading pretrained “DQN” agent, you could use “load” and “save” functions as:

doTraining = false; 
if doTraining 
    % Train the agent. 
    trainingStats = train(agent,env,trainOpts); 
    save myagent.mat agent 
else 
    % Load the pretrained agent for the example. 
    load("MATLABCartpoleDQNMulti.mat","agent") 
end 