To monitor the change in weights of the actor network at every step during training in the example "Tune PI Controller Using Reinforcement Learning", you can use a custom callback function to record the model weights at each step.
You can create a custom callback function to extract and log the weights of the actor network which will be called at each training step.
global weightsHistory;
weightsHistory = [];
% Custom callback function to log actor weights
function stop = logActorWeights(agent, ~, ~)
global weightsHistory;
actor = getActor(agent);
parameters = getLearnableParameters(actor);
% Append the current weights to the history
weightsHistory = [weightsHistory; parameters{1}(:)'];
stop = false;
end
"logActorWeights" function is defined to extract the learnable parameters (weights) of the actor network and append them to "weightsHistory".
Then, add the callback function to the training options
trainOpts = rlTrainingOptions(...
MaxEpisodes=maxepisodes, ...
MaxStepsPerEpisode=maxsteps, ...
ScoreAveragingWindowLength=100, ...
Verbose=false, ...
Plots="training-progress", ...
StopTrainingCriteria="AverageReward", ...
StopTrainingValue=-355, ...
Callbacks=@logActorWeights); % Add the callback function
Hope this answers your query.
