reentering a parallel pool
1 次查看(过去 30 天)
显示 更早的评论
Hi everyone, I try to parallel function evaluation on workers. Suppose I have 10 different vectors that will be evaluated through same function. I want to retrieve outputs when they become ready. Also, when the certain number of outputs is reached, I want to evaluate the retrieved outputs through a different function without waiting to fetch all of the results. Moreover, when the unretrived data of the agents become ready, I want to use it on the same fc2 function. I want to incomplete parfeval futures to use when they finished on the same fc2 function. Also, It will be a continuing process. Whenever 5 of the retrieved results become ready, they will be used as an input for fc2 function and output of the fc2 function will be input for parfeval function again and again. I wonder is it possible or not? Here is my code.
while itr<itrmax
for j=1:nbagents
f(j) =parfeval(p,@evaluate,1,agent(j));
end
counter=0;
finishedjobs=[];
for idx = 1:nbagents
[completedIdx,value] = fetchNext(f);
finishedjobs=[finishedjobs completedIdx];
agent(completedIdx)=value;
counter=counter+1;
if counter==5
break;
end
end
for i=1:counter
output(i)=agent(finishedjobs(i))
end
target=fc2(output(i));
itr=itr+1;
end
0 个评论
回答(1 个)
Sean de Wolski
2020-6-10
Look at using a DataQueue to transmit intermediate data from one parfeval future that's running continuously.
web(fullfile(docroot, 'parallel-computing/parallel.pool.dataqueue.html'))
5 个评论
Sean de Wolski
2020-6-11
I don't think that's possible with parfeval, unfortunately. It can be done with spmd, though that's synchronous. Maybe one of the Parallel Devs will chime in.
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Parallel Computing Fundamentals 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!