Nested parfor loop slows down with each iteration

6 次查看（过去 30 天）
Marc
Marc 2024-6-13
回答： Jasvin 2024-6-13
I have the following (simplified) code example with a nested outer parfor and inner for loop. Variables are sliced. It takes about 30 minutes for the first 10 outer iterations (running on 10 workers). The second round (iterations 10-20) takes more than an hour and it keeps increasing for later iterations. What is wrong here?
function my_function()
n_iterations_v = cell(1,50);
n_iterations_v = init_iterations(); % each element is an integer
parameters_v = cell(1,50);
parameters_v = init_params() % element i is a cell array with n_iterations_v{i} elements (where each element is a struct)
% outer loop
parfor (k = 1:50)
results_v = cell(1,n_iterations_v{k});
% inner loop
for i=1:n_iterations_v{k}
results_v{i} = some_other_function(parameters_v{k}{i});
end
save(filename, "-fromstruct", struct("results_v",results_v));
end
end

请先登录，再回答此问题。

回答（1 个）

Jasvin
Jasvin 2024-6-13
Have a couple of suggestions on how to investigate further.
Can you measure whether the same behaviour is observed for a regular "for" loop as well? If that's the case then it would imply that this is expected behaviour and parallelization can only do so much and the performance has to do more with the business logic of your code instead.
I also found this StackOverflow answer that suggests to use "parpool('threads')" which could be applicable in your case since you too seem to have a decent amount of data along with the copy operation involved.
https://stackoverflow.com/a/69955780
You could also create a technical support case with the MATLAB team by contacting support@mathworks.com

类别

在 Help Center 和 File Exchange 中查找有关 Parallel for-Loops (parfor) 的更多信息

标签

产品


版本

R2024a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by