Parallel for loop problem

Question

Simone Fiumi 2022-8-12

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1778395-parallel-for-loop-problem

评论： Simone Fiumi 2022-8-12

I'm running the following code in parallel. The variables of interes are w_b and alpha_b. The problem is that if I pause at a random time and then quit debugging the variables computed up to now are not saved. If i do that without the parallel cicle but with a simple for loop the variables are saved. For example, let's say I stop after 1 hour and I have 5 variables saved using a standard for loop, while with a parfor loop there are no saved variables (they are still equal to 0 as preallocated).

Nsim=50;
alpha_b=zeros(Nsim,1);
w_b=zeros(Nsim,1);
parfor k=1:Nsim
    
    seq=zeros(size(M,1),size(M,2));
    for j=1:size(M,1)
    idx=find(M(j,:)==0,1)-1;
    seq(j,:)=myhmmgenerate(idx,size(M,2),P_est,E,M(j,1));
    end
    sequence_s=seq_to_sequence(seq); 
    jumps_s=make_jumps(sequence_s);
    [Y_btstrp,N_btstrp]=Y_N(jumps_s);
    P_start_b=Pstart_nd(seq,2,2,Y_btstrp,N_btstrp); 
    [LL, ~, P, obsmat, nrIterations]=dhmm_em(seq, prior, P_start_b, E,N_btstrp,Y_btstrp);  
    [w_b(k),alpha_b(k)]=parameters(P,ne,nn,N_btstrp,Y_btstrp,1); % VARIABLES OF INTEREST
   
    
end

2 个评论
显示无隐藏无

Jeffrey Clark 2022-8-12

@Simone Fiumi, there is no guarenteed order of processing of the parfor indexs. Are you sure you didn't get 5 or more values somewhere (but the same somewhere) in each of w_b and alpha_b vectors? If Nsim is large the run might have filled in end-(0:4:20) and you might not have noticed.

Simone Fiumi 2022-8-12

I don't know if that happened or not. Right now it is running and since I don't want to risk I'll wait for it to end completely so that I won't encounter this problem. Thanks.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Raymond Norris 2022-8-12

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1778395-parallel-for-loop-problem#answer_1025760

在 MATLAB Online 中打开

@Simone Fiumi think of the code running in the parfor as running on some entirely different machine. There is no "debugging" per se. So pausing/stopping the parfor terminates the work entiring including the ability to checkpoint variables.

What you could consider doing is creating a data queue that sends data back to the MATLAB client. The client (outside of the parfor) could view the data queue. For instance:

function sf
D = parallel.pool.DataQueue;
afterEach(D,@(data)logdata(data))
parfor idx = 1:10
    ...
    [wb, ab] = parameters(P,ne,nn,N_btstrp,Y_btstrp,1);
    % Send data back to client
    send(D,[wb ab])
    w_b(k) = wb;
    alpha_b(k) = ab;
end
end
function logdata(D)
% Evaluate a snapshot of wb, ab
end