reduction variables and evaluation order

Question

Bruno Luong 2022-8-8

1
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1775995-reduction-variables-and-evaluation-order

评论： Raymond Norris 2022-8-12

This the doc on can read

"A reduction variable accumulates a value that depends on all the iterations together, but is independent of the iteration order. MATLAB allows reduction variables in parfor-loops."

Among the operations one have

% X = X * expr

And we all know for matrix, multiplcation order matters. So why X itcan be considered as valid reduction variable using "*"?

However when I test it it looks like parfor magically multiply in the right order as showed in this code:

A=rand(2,2,10);
% B = A(:,:,1)*A(:,:,2)*...A(:,:,10)
B=1;
for k=1:size(A,3)
    B = B*A(:,:,k);
end
B
B = 2×2
    0.1810    0.3914
    0.1442    0.3118
C=1;
parfor k=1:size(A,3)
    pause(0.1*rand()); % This will make the order of each worker more or less random
    C = C*A(:,:,k);
end
Starting parallel pool (parpool) using the 'local' profile ...
Connected to the parallel pool (number of workers: 2).
C
C = 2×2
    0.1810    0.3914
    0.1442    0.3118
norm(B-C)/norm(B)
ans = 3.3254e-16

Do I missread the doc? There is a descrption in the doc about what works are done by worker and what by client and it is not totally clear to me. Is there any joint+fork occurs when such variable is updated? Order matter or not? Can someone shed a light?

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Matt J 2022-8-8

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1775995-reduction-variables-and-evaluation-order#answer_1023190

编辑：Matt J 2022-8-8

在 MATLAB Online 中打开

First remember that each worker is assigned some consecutive subset of loop iterations. The worker will execute the iterations within its assigned subset in the original, consecutive order. Therefore, each partial sequence of reduction operations will have a predictable result.

Then, once all iterations are complete, the partial reductions are post-consolidated. I suspect that the order in which reduction variables are post-consolidated is also done to preserve the original ordering (and why not, since the processing is sequential anyway at this point).

Note therefore that concatenation works, too, even though we can clearly see that the timing of each iteration's execution is all over the place:

C=[];
parfor k=1:10
    pause(0.1*rand());
    disp("k="+k)
    C=[C,k];
end
Starting parallel pool (parpool) using the 'local' profile ...
Connected to the parallel pool (number of workers: 2).
k=1
k=3
k=4
k=7
k=10
k=2
k=5
k=6
k=8
k=9
C
C = 1×10
     1     2     3     4     5     6     7     8     9    10

8 个评论
显示 6更早的评论隐藏 6更早的评论

Matt J 2022-8-8

That means the reduction variables required the operator to be associative, which is more relax than commutative (order)?

It is recommended here that the operation be associative,

https://www.mathworks.com/help/parallel-computing/reduction-variable.html#bragj2e-1

Raymond Norris 2022-8-12

When the client sends the instructions/data out, it recalls the order for assignment back to the variable on the lefthand side. This doesn't mean the values will be sorted (as in the case of the clock example). It means that MATLAB knew when the 4th value returned, where to put it. Even if it was assigned after the 5th (which could have been the 1st iteration to evaluate in the 2nd group).

请先登录，再进行评论。