How is the work distributed in parfor loop?

Question

Luqman Saleem 2024-2-15

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2082618-how-is-the-work-distributed-in-parfor-loop

评论： Walter Roberson 2024-2-17

In a scenario with a parfor loop consisting of 10000 iterations distributed over 10 workers, is each worker initially assigned 10000/10=1000 tasks at the beginning of the parfor loop, or does each worker receive one task initially and subsequently get assigned new tasks upon completion?

Specifically, I am encountering a situation where approximately 500 iterations of a parfor loop (of total 10000 iterations) are computationally heavy while the remaining ~9500 iterations are computationally lighter. When running this parfor loop, I notice that the execution is initially extremely fast, with around 90% completion within 5 minute. However, the remaining 10% takes significantly longer, up to 20 hours. My suspicion is that the computationally heavy iterations are being assigned to a single worker, which handles them sequentially.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Edric Ellis 2024-2-16

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2082618-how-is-the-work-distributed-in-parfor-loop#answer_1410178

If you know in advance which iterations are the time-consuming ones, then you could consider running those together as a separate parfor loop, and the default iteration partitioning described by @Walter Roberson will probably do a decent job of keeping all workers busy.

If you do not know in advance, then there are two strategies that you could use, and maybe you could use both together:

Shuffle the order of the iterations using randperm or similar to try and end up with the long-running iterations in different sub-ranges.
Use parforOptions to force small sub-ranges.

2 个评论
显示无隐藏无

Luqman Saleem 2024-2-17

在 MATLAB Online 中打开

Thank you. I have a general idea of which iterations are computationally intensive. My current parfor loop structure is as follows:

parfor ix = 1:10000
    % code
end

Specifically, I have identified the iterations from ix = 2000 to 2500 and from ix = 7000 to 7500 as computationally heavy. Could you please provide a sample code illustrating how I can instruct the parfor loop to distribute these iterations among multiple workers?

Walter Roberson 2024-2-17

For example, set the RangePartitionMethod to [50*ones(1,(1950-0)/50), 20*ones(1,(2500-2000)/50), 50*ones(1,(7000-2500)/50)), 20*ones(1,(7500-7000)/20), 50*ones(1,(10000-7500)/50)]

or something like that.

请先登录，再进行评论。

Answer 2

Walter Roberson 2024-2-15

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2082618-how-is-the-work-distributed-in-parfor-loop#answer_1409658

You can control how parfor divides up the iterations by using parforOptions() RangePartitionMethod https://www.mathworks.com/help/parallel-computing/parforoptions.html#mw_5eb7f106-9fa3-45da-a7ae-6f5b0db4bef0

The default is "auto", which divides most iterations up into chunks, then divides most of the remaining iterations into smaller chunks, then hands out the remaining iterations one at a time.