Avoid broadcasting an array in parfor

2022 2 20
1 个回答
回答已采纳
更新时间：2022 2 20
2 次查看（30 天）

0 个投票

Hi, I have a matrix (A) and I want to access certain columns of this matrix in for loop. I need some help avoiding the broadcasting the the entire matrix A in the loop when I use parfor. Any help would be useful. I have provided simple working example which is a part of larger code.
A = rand(100,5);
combinations = randi([1 5],100,3);
parfor i=size(combinations,1)
current_comb = combinations(i,:);
current_profile = A(:,current_comb);
end

请先登录，再回答此问题。

 采纳的回答

Raymond Norris
Raymond Norris 2022-2-20

0 个投票

As a side note, I suspect that
parfor i=size(combinations,1)
should be more like
parfor i=1:size(combinations,1)
as the original is a scalar and therefore, the for-loop is a single iteration.
In your example, combinations is a sliced variable; however, A needs to be a broadcast. Using the loop variable, i, MATLAB knows a prior how to break up combinations to each of the workers. On the other hand, indexing into A is not known until we know the value of current_comb (at runtime). To provide this flexibleness, MATLAB needs to send all of A as a broadcast variable.
It might help to look at ticBytes and tocBytes to get a sense of how much data is being sent back and forth.

3 个评论
显示 1更早的评论 隐藏 1更早的评论

dipak sanap
dipak sanap 2022-2-20
Hi, Raymond thanks for the response. And yes, it was a typo on my side while defining loop iteration number. Since, the size of A is not that big, I can create another variable which holds the combinations of A in another matrix which can later be used in parfor. Below, is the solution, I could think of. Please, suggest if this is correct.
A = rand(100,5);
combinations = randi([1 5],100,3);
A_combs = zeros(size(A,1),3,size(combinations,1));
for i=1:size(combinations,1)
A_combs(:,:,i) = A(:,combinations(i,:));
end
parfor i=1:size(combinations,1)
current_comb = combinations(i,:);
cuurent_profile = A_combs(:,:,i);
end
Raymond Norris
Raymond Norris 2022-2-20
That's the right idea. Now you're able to slice A_combs. The only caveat is that you've doubled your memory size on the client side (with A_combs). Mind the typo in cuurent_profile.
One consideration is how much of this can be created within the parfor loop. You've given pseudo code, but if you're really using rand, you might generate your combinations in the parfor, eliminating the need to send it to the workers.
dipak sanap
dipak sanap 2022-2-20
Hi, thanks Raymond. Yeah, it was just a pesudo code. I dont actully use rand to generate combinations and they can not be done in par loop. But, thanks I just wanted to make sure that creating A_combs was correct approch. I would go ahead with this strategy.

请先登录，再进行评论。

更多回答（0 个）

类别

在 帮助中心 和 File Exchange 中查找有关 Loops and Conditional Statements 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by