Log file of minimization done in parallel using batch
4 次查看(过去 30 天)
显示 更早的评论
Hi,
I'm working on a problem that has an outer minimization that searches over one set of parameters, and an inner minimization that for every iteration of the outer one, solves n systems of equations, where each system depends on the parameter values of the iteration of the outer loop. As all systems of the inner loop are independent of each other, I can use the parallel features of MATLAB to speed up the outer minimization. This work perfectly when using the 'local' profile in my laptop and I save a lot of time by using PARFOR in the inner minimization, compared with the same situation without parallelizing the inner loop.
However, I need to submit this as a batch job to a remote cluster, and I would like to be able to monitor the job. In particular, I would like to see the value of the objective function at each iteration, as it is shown in my laptop when 'Display','iter' is specified in options.
I have tried using
job = batch('script', 'matlabpool', 4,'CaptureDiary',true)
However, using
diary(job)
while the job is running returns nothing. It seems that I'll have to wait to see the output of diary() until the job is finished. Is there any way to observe what I want? I would like to have a log file as the one that is generated when I run the script as a batch job without using the parallel features.
Thanks,
Fernando
0 个评论
采纳的回答
Edric Ellis
2012-7-9
编辑:Edric Ellis
2020-11-24
At this time, it's not possible to get the diary output from a batch job until after the job has completed execution. You could manually write status output to a file using fopen/fprintf/fclose.
EDIT
Recent versions of Parallel Computing Toolbox do support incremental updates to the diary of a batch job. For example:
c = parcluster('local');
j = batch(c, 'for idx = 1:5, disp(idx), pause(2), end');
% Wait for the job to start running.
wait(j, 'running');
% Print the diary in a loop.
while ~strcmp(j.State, 'finished')
fprintf('Diary at time: %s\n', string(datetime));
diary(j);
pause(2);
end
Results in output like this:
Diary at time: 24-Nov-2020 13:57:01
Warning: The diary of this batch job might be incomplete because the job is still running.
--- Start Diary ---
1
2
3
--- End Diary ---
更多回答(0 个)
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Simple Batch Processing 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!