Correct fetchNext index after clearing FevalFuture object

Question

Nicola Dalla Pozza 2019-10-14

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/485186-correct-fetchnext-index-after-clearing-fevalfuture-object

回答： Edric Ellis 2019-10-15

I am using parfeval to run code asynchronously on workers. The running time varies a lot between workers depending on some parameters that in the following code I represent with 'm'.

When I collect the results with fetchNext, I then process them with idx, in particular retrieving m(idx). At the same time, I need to clear the last retrieved FevalFuture object to save memory.

In Parfeval - Memory consumption piling up - Clear output data? it is suggest to simply assign [ ] to the FevalFuture object. However, it seems to me that doing this mix up the indices idx of the objects, which I need to further process the data. See for instance this mve

N = 10;
m = 1:10;
for idx = N:-1:1
    f(idx) = parfeval(@(m,N) m*rand(N), 1, m(idx), N);
end
total = 0;
for idx = 1:N
    [idx, data] = fetchNext(f);
    idx
    f(idx) = []; % This line here
    
    % process data with idx, m(idx) ..
end

If you comment the line indicated the code will work fine, but if I clear f(idx) the just-used idx could be reused and I would process the data with the wrong m(idx).

How is it possible to get the correct index idx with which f(idx) was called in the fist loop?

Thanks in advantage

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Edric Ellis 2019-10-15

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/485186-correct-fetchnext-index-after-clearing-fevalfuture-object#answer_396431

在 MATLAB Online 中打开

One way you can deal with this is by making the same modification to m as you proceed. Something like this perhaps:

N = 10;
m = 1:10;
for idx = N:-1:1
    f(idx) = parfeval(@(m,N) m*rand(N), 1, m(idx), N);
end
for idx = 1:N
    [fidx, data] = fetchNext(f);
    % Get the value out of "m" corresponding to the completed future
    mval = m(fidx);
    % Delete corresponding elements of both "f" and "m"
    f(fidx) = [];
    m(fidx) = [];
    % Work with the particular value of "m" and the result
    disp([mval, sum(data(:))]);
end