Can a parfor loop be restarted?

Question

Tyler Warner 2018-6-8

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/404737-can-a-parfor-loop-be-restarted

回答： OCDER 2018-6-11

I am processing large amounts of files. Each file might take thirty minutes to an hour. Here's the code snippet on the high level:

parfor firstfile:lastfile
  [morestuff1, morestuff2] = runfunction(stuff1, stuff2);
end

Problem is that every night I leave it running on the server, something wrong happens and the parfor doesn't complete. I want to be able to catch a loop that fails for any reason and restart it later. Here's what I am proposing as the solution:

files = firstfile:lastfile;
while length(files) > 0
  parfor currentfile = files
    try
      [morestuff1, morestuff2] = runfunction(stuff1, stuff2);
      files(files == currentfile) = [];
    catch
      % not sure what to do here
    end
  end
end

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

OCDER 2018-6-11

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/404737-can-a-parfor-loop-be-restarted#answer_324222

在 MATLAB Online 中打开

Perhaps you can save the input variables used for your runfunction into a separate cell array. Try this for example:

ErrorFile = cell(1, 1000);
parfor k = 1:1000
    stuff1 = rand(1);
    stuff2 = rand(1);
    try
        assert(stuff1 > stuff2, 'OOPS: Error in this function!'); %Pretend this is the "runfunction". If there's an error, record the input variables.
    catch Msg
        ErrorFile{k} = {Msg, k, stuff1, stuff2}; %Save the iteration #, variables, and error message 
    end
end
ErrorFile(cellfun(@isempty, ErrorFile)) = []; %This stores variables required to redo parfor with errors
cellfun(@(x) display(x{1}), ErrorFile); %Display all the error messages