problem performing multiple, independent tasks simultaneously in parfor
显示 更早的评论
Here's a parfor code snippet that's intended to search for 2 different file types and load a file simultaneously, but fails due to different output types:
% run 3 independent things simultaneously to save time
parfor n 1:3
switch n
case 1
output{n} = dir('pathname\**\*.txt'); % this takes a long time
case 2
output{n} = dir('pathname\**\*.csv'); % this takes a long time
case 3
output{n} = load('pathname\matlab.mat'); % this also takes a long time
end
end
Here's how I'd prefer to write this function, which doesn't crash and runs MUCH faster than each function separately, but variables a, b, and c are not saved.
% run 3 independent things simultaneously to save time
parfor n 1:3
switch n
case 1
a = dir('pathname\**\*.txt'); % this takes a long time
case 2
b = dir('pathname\**\*.csv'); % this takes a long time
case 3
c = load('pathname\matlab.mat'); % this also takes a long time
end
end
I'm not I/O limited, these processes run simultaneously much faster. Run time with the second snippet is the same as the single longest operation. How can I run either method of code snippet with the outputs saved?
3 个评论
Walter Roberson
2022-7-26
parfor n = 1:3
output{1} and output{2} would be directory information, output{3} would be a struct containing one field for each variable loaded from the matlab.mat file.
Bruno Luong
2022-7-26
编辑：Bruno Luong
2022-7-26
Redargless it does't work (I don't know why just by reading your code)
Your parfor doesn't do any real computation, just reading, so it is mostly limited by your perepherical HW (reading speed and transfert data through the bus). Not sur parfor will not in contrary to what you would expect slow down even more, since the caching is worse.
David L.
2022-7-26
采纳的回答
That code that assigns to output{n} will not fail due to different output types. All three of the outputs (for the code you posted) would be struct(), and besides you are outputting a cell array.
parfor n = 1:3
switch n
case 1
output{n} = dir('*.txt'); % this takes a long time
case 2
output{n} = dir('*.csv'); % this takes a long time
case 3
output{n} = struct('this', 5, 'that', 7); % this also takes a long time
end
end
output
Have you considered using parfeval?
F(1) = parfeval(@() dir('*.txt'), 1);
F(2) = parfeval(@() dir('*.csv'), 1);
F(3) = parfeval(@() struct('this', 5, 'that', 7), 1);
wait(F)
a = fetchOutputs(F(1))
b = fetchOutputs(F(2))
c = fetchOutputs(F(3))
更多回答（0 个）
类别
在 帮助中心 和 File Exchange 中查找有关 Loops and Conditional Statements 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
