In MATLAB, when using the reinforcement learning toolbox and the train function, there is an option to save the agent when certain criteria are met. The SaveAgentCriteria and SaveAgentValue options allow you to specify when to save the agent based on the training progress. However, the train function by default saves not only the agent but also the training statistics, which can be quite large and lead to the error you're experiencing if the files become too big.
As of my knowledge cutoff date in early 2023, there is no built-in option in the train function to save only the agent without the training statistics. However, you can work around this limitation by implementing a custom training loop where you manually save the agent when your criteria are met.
Here's an example of how you might implement a custom training loop and save only the agent:
% Assuming you have your environment 'env' and agent 'agent' already set up
% Training options
maxEpisodes = 10000;
maxSteps = 500;
thresholdReward = 100;
% Loop over episodes
for episode = 1:maxEpisodes
% Reset the environment and the agent
observation = reset(env);
episodeReward = 0;
% Loop over steps
for step = 1:maxSteps
% Generate action from the agent
action = agent.act(observation);
% Simulate the environment with the action
[nextObservation, reward, isDone, info] = env.step(action);
% Accumulate episode reward
episodeReward = episodeReward + reward;
% Save experience tuple (S, A, R, S')
experience = {observation, action, reward, nextObservation, isDone};
agent.remember(experience);
% Learn from the experience tuple
if mod(step, agent.AgentOptions.ExperienceBufferLength) == 0
agent.learn();
end
% Update the observation
observation = nextObservation;
% Check for episode termination
if isDone
break;
end
end
% Save the agent if the reward threshold is met
if episodeReward >= thresholdReward
saveAgentOnly(agent, episode); % Custom function to save the agent
end
end
% Custom function to save only the agent
function saveAgentOnly(agent, episodeNumber)
filename = sprintf('savedAgents/AgentEpisode%d.mat', episodeNumber);
agentToSave = agent; % Copy the agent to a new variable
save(filename, 'agentToSave');
end
In this custom training loop, you manually handle the training process and save the agent to a file using the save function. The saveAgentOnly function is a custom function that you define to save only the agent object without the training statistics.
