How can I parallelize a script full of functions for faster implementation?

1 次查看（过去 30 天）
Emmanuel
Emmanuel 2017-6-4
评论： KSSV 2017-6-4
I have the following script:
%A loop will come herethat reads the variable "name"
name = 'RD 771Jun03'
%%Extracting
BinSearch_right(name)
BinSearch_left(name)
BinSearch_joints(name)
BinSearch_image(name)
BinSearch_sk(name)
It is all the BinSearch_*(name)* statements above are functions that extract files. This code takes atleast around 3 minutes to run for 1 file and I have 100 such files.
How can I parallelize this script such that all the BinSearch* statements are running parallely? I have tried reading about parpool but have failed to understand it's implementation. Can you please give me a good lead on how to go about it? Is it wise to call these functions in pythonm?
  1 个评论
KSSV
KSSV 2017-6-4
编辑：KSSV 2017-6-4
How many cores yo have in your processor? Also you will get help on your function if you copy the code here. Code might need optimization.

请先登录，再进行评论。

请先登录，再回答此问题。

回答（1 个）

Walter Roberson
Walter Roberson 2017-6-4
You have two choices:
1)
parpool();
parfor K = 1 : length(filenames)
name = filenames{K};
BinSearch_right(name)
BinSearch_left(name)
BinSearch_joints(name)
BinSearch_image(name)
BinSearch_sk(name)
end
The above does the files in parallel, but the work for any one file is done in series in one worker. If one file takes less time than the others, then parfor would simply move on to the next file for that worker.
2)
parpool(5);
for K = 1 : length(filenames)
name = filenames{K};
parfor worker = 1 : 5
switch worker
case 1; BinSearch_right(name)
case 2; BinSearch_left(name)
case 3; BinSearch_joints(name)
case 4; BinSearch_image(name)
case 5; BinSearch_sk(name)
end
end
end
The above does the work on each file in parallel. If one of the routines takes less time than the others, then the worker would be left idle until the longest step was completed. As you are passing in file names rather than file contents, the workers would probably have to "complete" about reading the file content in.
I would point out that reading in a file can be the slowest part of processing (file systems are always slower than memory), so you should consider rewriting your functions so that they take the content of the image rather than the filename of the image.

类别

在 Help Center 和 File Exchange 中查找有关 Parallel for-Loops (parfor) 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by