tall array within a parfor?
显示 更早的评论
Consider the following scenario:
ds = datastore(...);
dst = tall(ds);
parfor ...
% warning here: dst is a broadcast variable. This might result in unnecessary communication overhead.
chunk = gather(dst(:, idx));
end
Given the fact that dst is a tall array, why MATLAB warns about overhead? In fact, due to the lazy nature of tall arrays, above parfor is quite memory friendly.
So, what's the reason of the warning in case of tall arrays (pleast note I'm aware of communication overhead issue in case of other data types)?
Thanks!
采纳的回答
You've generated idx in a way that doesn't have a simple dependence on the loop variable. This warning is not present if you are simply parfor-looping over the columns.
dst=tall(rand(5,10));
parfor i=1:size(dst,2)
chunk = gather( dst(:, i) );
end
Because of whatever you're doing, the variable is not sliced. Because it is not sliced, you may have overhead due to two or more parpool workers trying to access the same memory at the same time. If you had been writing into the tall array, you would have to be even more careful. Parallel write operations into the same memory location can have unpredictable effects.
更多回答（0 个）
类别
在 帮助中心 和 File Exchange 中查找有关 Parallel for-Loops (parfor) 的更多信息
Translated by 
