How can I run a function in parallel and return the compiled output of the function?

Question

Anthony 2016-6-16

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/290321-how-can-i-run-a-function-in-parallel-and-return-the-compiled-output-of-the-function

编辑： Anthony 2016-7-1

I have a long cell array (thousands) of full file paths, and a function that has been written to pull data from each file path. Normally, I would just send the cell array of file paths to this function, and it would return the data. Because the size of my cell array is so large, this can take a long time, sometimes a couple hours or more. I was wondering if there is any way to have MATLAB call this function in parallel with the file paths sent in groups, and return the results in the same way as without the parallel function processing?

I'll try to give an example to better explain what I want to do.

Current method:

filepaths = {'filepath1','filepath2','filepath3',.....}

results = datapullfunction(filepaths)

Desired method:

filepaths = {'filepath1','filepath2','filepath3',.....}

--send 1/12th of filepaths to datapullfunction 12 times in parallel (I have a 12 core processor) and have it return the same results as the current method--

_____________

Currently I'm trying to use batch:

c = parcluster();

j = batch(c,'datapullfunction',1,filepaths);

results = fetchOutputs(j)

but results returns an error saying my input to datapullfunction "is not a recognized option." I know that it is a recognized option because if I send datapullfunction(filepaths) it works just fine.

Thanks!

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

sam0037 2016-6-24

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/290321-how-can-i-run-a-function-in-parallel-and-return-the-compiled-output-of-the-function#answer_226601

Hi,

One way to achieve this would be to create a set of jobs and submit to your parallel cluster for execution. Refer to the link below to know more about running independent jobs on a cluster:

http://www.mathworks.com/help/distcomp/program-independent-jobs.html

http://www.mathworks.com/help/distcomp/program-independent-jobs-on-a-local-cluster.html

3 个评论
显示 1更早的评论隐藏 1更早的评论

Walter Roberson 2016-7-1

http://www.mathworks.com/help/distcomp/createtask.html

{inputargs} A row cell array specifying the input arguments to be passed to the function F. Each element in the cell array will be passed as a separate input argument. If this is a cell array of cell arrays, a task is created for each cell array.

Anthony 2016-7-1