How to send files from worker to client in MDSC?

Question

raym 2018-9-9

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/418206-how-to-send-files-from-worker-to-client-in-mdsc

评论： raym 2018-9-10

采纳的回答： Edric Ellis

在 MATLAB Online 中打开

In the client I opened several workers by MDSC with MJS cluster "MyMJScluster" using multiple computers as node PC.

parpool('MyMJScluster')

Then I run a function which has a spmd block.

spmd
...
end

During the running of job, several files are generated on the local hard drive of each node PC. These files are under the same path of each node, such as "D:\data1\output\*.*", but filenames are different and unpredicatble, but could be dir from worker PC.

I want these files to transfered to the client under the same path to pool them together and do further combined small-scale analysis of these files at client.

Is there a way to do such a transfer? The addAttachedFiles only transfer the specified files from client to workers, but not the reverse.

Thanks.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Edric Ellis 2018-9-10

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/418206-how-to-send-files-from-worker-to-client-in-mdsc#answer_336102

在 MATLAB Online 中打开

There's no facility to transfer files from the workers back to the client. If you can, I would suggest reading the files into memory on the workers, and then transferring the variables back from your spmd block as Composite objects. Something like

spmd
  ... do stuff ...
  myData = loadAllOutputs('D:\data1\output\*.*');
end
% Pull all the data back from the workers
localData = myData(:);
% localData is now a cell array of length 'numlabs'
% containing all the data loaded on each worker