How to Pin Workers to Specific Cores

Question

Jonathan 2015-4-8

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/196618-how-to-pin-workers-to-specific-cores

评论： Ayad Anwer 2019-9-6

I would like to know how to pin parpool workers to specific cores. I have a dual socket system, so each socket has faster access to certain resources, like GPUs.

Therefore, say I have 8 workers in my pool and 8 GPUs. I want to pin each worker to a specific core on my dual socket system.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Edric Ellis 2015-4-8

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/196618-how-to-pin-workers-to-specific-cores#answer_174409

在 MATLAB Online 中打开

What OS are you using? If Windows, you could adapt the code from this answer on Stack Overflow. One way to pick the affinitization is to use an spmd block after opening the pool and using labindex to choose the CPU.

On Linux, you can use this approach to set affinity. I would adapt that approach and use the undocumented feature('getpid') means of accessing the current process' PID, something like this

spmd
  pid = feature('getpid');
  system(sprintf('taskset -pc %d %d', labindex-1, pid));
end

3 个评论
显示 1更早的评论隐藏 1更早的评论

Steve Lantz 2017-11-10

在 MATLAB Online 中打开

Here's a function that does the trick on Windows 7. It is based on the Stack Overflow link above.

function pin_process_to_core(coreidx,totalcores)
    proc = System.Diagnostics.Process.GetCurrentProcess();
    procbit = int32(2^(coreidx-1));  % will become mask
    fprintf('For coreidx %2d, procbit is %s\n',...
        coreidx,dec2bin(procbit,totalcores));
    proc.ProcessorAffinity = System.IntPtr(procbit);
end

In typical usage, the argument coreidx corresponds to the labindex for each worker, so the function is meant to be called from (e.g.) within a spmd block.

The above calls to the .NET library actually fail if they are not enclosed in a function. If they appear directly within the spmd, MATLAB can't decide whether it's looking at a variable or a function, and it throws an error.

You can comment out the fprintf if you don't want to look at the masks that are set for each worker.

This trick can improve performance quite a bit if there are many cores and many workers, but only a few tasks. It prevents the few workers/tasks from roaming around from core to core, resulting in continual trashing of the caches.

Ayad Anwer 2019-9-6