What happens when a parallel worker finishes early?

5 次查看（过去 30 天）
Tyler Warner
Tyler Warner 2017-7-20
评论： Tyler Warner 2017-7-28
Hi everyone,
I have a benchmark code running where I test the time it takes from start to finish. I noticed that the workers are not synchronized. What will happen when one worker finishes early? This is a parfor loop of 20 workers on a 20-core server.
Good day!

请先登录，再回答此问题。

采纳的回答

Edric Ellis
Edric Ellis 2017-7-21
When MATLAB runs a parfor loop, it attempts to split the iterations of the loop across the workers to keep them all busy simultaneously. However, this is not always possible (especially if you have only a very small number of loop iterations, and the iterations take different amounts of time). So, towards the end of the parfor loop, some of the workers will be idle waiting for the last workers to finish.
  4 个评论
显示 2更早的评论
Edric Ellis
Edric Ellis 2017-7-21
编辑：Edric Ellis 2017-7-21
To expand on my answer: we don't simply divide up the parfor loop iterations across the workers - we split the iterations in differing sized groups such that there are (generally) around 3 "intervals" (groups of loop iterations) per worker. Subsequent intervals get dispatched to workers as they complete operating on their intervals - this is a relatively simple but effective means of load-balancing. We don't currently provide any way for you to tailor this scheduling, and you're right we don't try and subdivide late-finishing intervals to see if we can get other workers to operate on them.
The current scheme provides a trade-off between communication overhead and late-finishing. There are always going to be pathological cases where no scheduling system can work well. Consider the following silly example:
parpool('local', 3);
parfor idx = 1:3
if idx == 3
pause(100);
end
out(idx) = rand();
end
There is no possible scheduling that we could use for this case to keep all the workers busy at the same time.
If you want full control over the parallel scheduling, you can use parfeval.

请先登录，再进行评论。

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Parallel for-Loops (parfor) 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by