Is there an easy way to find out which workers are running on the same host in a Generic Cluster job so I can efficiently allgather?

Question

Frank Moore-Clingenpeel 2022-8-30

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1790930-is-there-an-easy-way-to-find-out-which-workers-are-running-on-the-same-host-in-a-generic-cluster-job

评论： Frank Moore-Clingenpeel 2022-8-30

Say I have the following script which submits a job to a Generic parallel cluster, which has procsPerNode=2:

%myScript.m
c=parcluster('MyGenericClusterProfile');
j=createCommunicatingJob(c,'Type','spmd');
j.NumWorkersRange=[4 4];
createTask(j,@mySpmdFunction,0,{});
submit(j);

What this will do, is reques 2 nodes from my cluster, each of which will individually run 2 MATLAB workers in paralel, which alltogether will run mySpmdFunction as though it was launched within an spmd statement (so they can do stuff like labSend to communicate and use labindex to get an id, etc).

My question is, is there any way for the nodes to know which other workers are 'local'--i.e., which ones reside on the same piece of hardware versus which ones are remote? A way to use reflection to find this information is preferred, but if that's not available will MATLAB consistently assign workers to nodes sequentially (so then workers 1 and 2 will always share a node and workers 3 and 4 will always share a node in the example)? If there's no way to inquire what workers share nodes, is there a way to inquire and find the GenericCluster the workers are running on so I can find the procsPerNode property?

For that matter, is there a built-in allgather function? I'm really only investigating this to implement my own allgather from scratch...

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Edric Ellis 2022-8-30

You can use gop to perform general MPI-style all-reduce operations, and a special case of that is gcat which can operate as an MPI-style gather or all-gather. For example:

parpool("local");
Starting parallel pool (parpool) using the 'local' profile ...
Connected to the parallel pool (number of workers: 2).
spmd
    x = gcat(labindex);
end
disp(x{1})
     1     2