Hi @浩文, to monitor the actor network's loss during reinforcement learning training in MATLAB, you can log and visualize the loss values using built-in tools. Here's a typical approach that works well when you're using train with an RL agent:
First, create a trainingProgressMonitor and an rlDataLogger to track training progress:
monitor = trainingProgressMonitor();
logger = rlDataLogger(monitor);
Then, define a callback function to log the actor loss. This function will be triggered after each learning step:
logger.AgentLearnFinishedFcn = @logActorLoss;
function dataToLog = logActorLoss(data)
dataToLog.ActorLoss = data.ActorLoss;
end
When setting up your training options, include the logger:
trainOpts = rlTrainingOptions( ...
MaxEpisodes = 1000, ...
Logger = logger, ...
Plots = "training-progress");
Train your agent with:
train(agent, env, trainOpts);
After training, you can visualize the logged data using the Reinforcement Learning Data Viewer:
openTrainingDataViewer(logger.LogFileName);
From the viewer, select the ActorLoss variable to view the trend over time. This gives you insight into whether the actor network is being updated effectively during training.
If you're using a custom training loop instead of the built-in train function, you can manually log the actor loss values at each training step and plot them yourself using standard MATLAB plotting functions. Hope this helps!
