Run scripts in parallel, one core each
显示 更早的评论
Dear all:
I’m looking for a way to run scripts simultaneously, one core each, on my local matlabpool. This is similar to the question “Run scripts in parallel on multiple workers (distributed job)” asked by “Igor Braverman” on 31 Oct 2012 but he did not get an answer to his question.
Say I have a script0.m that generates large outputs. Then script1.m, script2.m and script3.m are going to accept script0.m’s outputs as input, independently of one another, and perform their own functions. These scripts do not lend themselves well to parallelism, so I would just rather execute them in parallel.
Igor Braverman suggested using batch, say:
j1=batch(‘script1’);
j2=batch(‘script2’);
j3=batch(‘script3’);
with each batch using one core each but he found that MATLAB wants to wait for the first batch to finish before starting the next one, and I am finding the same problem. What’s the best way to go about this problem?
Thank you, cpicke
2 个评论
Edric Ellis
2015-8-20
This should work and run those scripts in parallel. Just after you've issued the 3 batch commands, what is the output of:
disp([j1, j2, j3])
and
disp(j3.Parent)
cpicke
2016-1-6
回答（1 个）
Walter Roberson
2015-8-19
Either spmd or parfor should work for that provided that you use functions instead of scripts.
funs = {@script1, @script2, @script3}
parfor K = 1 : length(funs)
outputs{K} = funs{K}(inputs);
end
2 个评论
Walter Roberson
2016-1-6
When you use "for" then if you have calculations with sufficiently large matrices, the calculations might be handed over to the BLAS or LAPACK libraries, which are highly optimized and multithreaded. The calculations might thus use all of your cores. However, when you use parfor, again those optimized libraries might be called, but they would be restricted to one core. Running in explicit parallel does not necessarily mean faster.
Also, parfor workers do not have access to the graphics hardware, so they would need to do all of their drawing in software.
Doing I/O on multiple large files in parallel can be slower than sequentially due to contention for the drive heads and drive caches. However, if the files are smaller, then typically it can be more efficient to read multiple files in multiple threads and return the data for processing.
It is possible that your process might benefit from using parfeval() to run processes that do the I/O and return ready data, with your master thread grabbing the next available set of data and doing the processing sequentially on the sets of data. Also, depending on what you are doing, a mapreduce approach might work for you.
类别
在 帮助中心 和 File Exchange 中查找有关 Parallel for-Loops (parfor) 的更多信息
2016-1-6
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
