Slice called broadcast variable in parfor

20 次查看（过去 30 天）

Mariella Dreißig 2019-12-17

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/497060-slice-called-broadcast-variable-in-parfor

评论： Mariella Dreißig 2019-12-18

I have a code I want to parallelize, something like:

parfor i = 1:length(all_idx(:,1))
    combis = nchoosek(1:k,2);
    
    results = zeros(length(combis(:,1)),1);
    for j = 1:length(combis(:,1))
        results(j) = fun(X(:,all_idx(i,combis(j,1))), X(:,all_idx(i,combis(j,2))));
    end
    
    ... 
end  

X and all_idx are broadcast variables, especially X is a huge matrix and produces a significant overhead. I read something about slicing the variables, the computation results are strored in (variables in which I write), but is there a way to somehow slice the variables which are read within the loop? As far as I understand, the whole matrices X and all_idx have to be sent to every worker in order to extract the columns needed, or did I miss something?

2 个评论
显示无隐藏无

Edric Ellis 2019-12-18

It looks from your code very much like each iteration of your outer parfor loop needs the whole value of X available. In this case, it is unavoidable that X must be sent separately to each worker.

"Slicing" variables is good - when it works - i.e. when your parallelism can be expressed as operations over independent "slices" of your data

Mariella Dreißig 2019-12-18