This is a known limitation of the TrainingProgressMonitor in MATLAB's Deep Learning Toolbox: once the monitor is stopped (i.e., monitor.Stop == true), it cannot be restarted or resumed directly. This typically happens when you save and reload your workspace from a checkpoint.
Here are two practical workarounds:
1. You can instantiate a new TrainingProgressMonitor after loading your checkpoint and continue logging from where you left off as shown below:
% After loading checkpoint
monitor = trainingProgressMonitor;
% Reinitialize monitor settings
monitor.Info = ["Epoch", "Iteration"];
monitor.Metrics = ["Loss", "Accuracy"];
monitor.XLabel = "Iteration";
% Optionally, set the starting point
monitor.Progress = previousProgress; % if you saved this
You will need to manually track and re-log any previous metrics if you want continuity in the visual plot.
2. If you saved the monitor’s data (e.g., metrics, progress values) before checkpointing, you can replot or merge them into a new monitor:
% Load previous metrics
load('checkpoint.mat', 'previousLoss', 'previousAccuracy');
% Create new monitor
monitor = trainingProgressMonitor;
monitor.Metrics = ["Loss", "Accuracy"];
% Re-log previous metrics
for i = 1:length(previousLoss)
recordMetrics(monitor, i, ...
Loss=previousLoss(i), ...
Accuracy=previousAccuracy(i));
end
This won’t “resume” the monitor per se, but it gives you a visual continuation.
To make merging easier, consider saving your metrics separately during training:
lossHistory(end+1) = currentLoss;
accuracyHistory(end+1) = currentAccuracy;
Then, when restarting, you can reconstruct the monitor’s state.
