What happens if I create a parpool with more workers than the number of cores the OS has assigned me?

4 次查看(过去 30 天)
I requested a machine from our internal cluster requesting for a interactive session on a machine with 64 cores. After my job got processed, I received the requested session. I then opened a VNC connection to this session through which I could then start MATLAB. 
Problem:
When I execute the following command on my MATLAB Command Window, I see that MATLAB is restricted to only use 16 cores instead of the 64 I expected:
maxNumCompThreads()
ans = 16
Questions:
  1. Is there a way to force MATLAB to use all the available cores on the machine?
  2. What happens if I call parpool('Processes',64) inside this MATLAB? Does the restriction apply to those workers as well?

采纳的回答

MathWorks Support Team
编辑:MathWorks Support Team 2024-12-6
  1. The given output indicates that the OS or the scheduling software is restricting MATLAB from using all available cores on the machine. MATLAB cannot control this behavior. However, you might be able to remove any restrictions by updating your environment (docker/ cluster/ scheduler) settings that are restricting the number of cores allocated to a process.
  2. If the 'Processes' profile was updated to use NumWorkers = 64 on this machine and parpool was then called Parallel Computing Toolbox would start 64 workers but they would be restricted to just the resources the OS has assigned. This would likely lead to resource contention. *
*One exception to this situation is on high core count Windows machines with > 64 cores where an OS restriction may indicate the presence of "processor groups". In this will not limit the ability to start process parpools larger than 64 workers and the NumWorkers in the 'Processes' profile can be safely adjusted using these steps.

更多回答(0 个)

类别

Help CenterFile Exchange 中查找有关 Parallel Computing Fundamentals 的更多信息

产品


版本

R2024a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by