How can I open a worker pool on a remote machine?

Question

Amit Rotem 2019-4-9

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/455290-how-can-i-open-a-worker-pool-on-a-remote-machine

评论： Jason Ross 2019-4-10

采纳的回答： Jason Ross

在 MATLAB Online 中打开

Can MATLAB send calculations to remote workers through ssh?

I want to achieve the following behavior:

% connect to remote machines via ssh
%  and open 4 workers on each machine (i.e. total pool of 8 worker)
RemoteParpool('location',{'local','userA@address1','userB@address2'},'PoolSize',[2 3 4])
% distribute some work over 9 workers (6 remote & 2 local) and return an answer to the Workspace variable 'MyArray'
parfor j=1:10
    MyArray(j)=rand; % do stuff
end

The remote (and local) machines all have MATLAB with Parallel Computing Toolbox and run on Linux. And I'm using passphraseless ssh login with shared keys.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Jason Ross 2019-4-9

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/455290-how-can-i-open-a-worker-pool-on-a-remote-machine#answer_369787

This is the purpose of MDCE / MATLAB Parallel Server. It allows you to run the MATLAB job scheduler on a collection of machines, or integrate with an existing scheduler installation (Slurm, LSF, PBS, Torque, Grid Engine, etc). You submit the job to the remote scheduler using Parallel Computing Toolbox and then the job scheduler starts the MATLAB sessions and runs the job.

The documentation for MATLAB Parallel Server can be found here.

2 个评论
显示无隐藏无

Amit Rotem 2019-4-10

Thanks for the prompt answer! However, this seems overly complicated for my needs, I don't think I need a scheduler - I'll be the only one using the cluster. Is there a simpler solution in the case of a single client? (with Mathematica I can achieve this with just a few lines of code)

Jason Ross 2019-4-10

No, when you go off the single machine (local scheduler) you definitely need some kind of scheduler listening.

We do offer a means of parallelized computation on Amazon EC2, where all the scheduler setup happens on Amazon -- but that's not local resources, and it does require some other setup.

请先登录，再进行评论。