Parfor is running much slower on a 18-core PC than a 10-core PC

4 次查看（过去 30 天）

Mingyang Sun 2021-8-26

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1440924-parfor-is-running-much-slower-on-a-18-core-pc-than-a-10-core-pc

评论： Mingyang Sun 2021-9-2

I am currently running a compute-intensive financial trading test; the test has 503 independent loops. I have two PCs in my office, one has 10-core Intel i9-10900k @ 3.7GHz with 128GB DDR4; the other one has 18-core Intel i9-7980XE @ 3.4GHz with 128GB DDR4 as well.

In order to speed up the whole processing, the first 10-core PC runs loop 1-to-189 in parfor; the second 18-core PC runs loop 190-to-503 in parfor.

After one-day running, I found the 10-core PC finished 25 loops, the 18-core PC only finished 4 loops. I found very curious, can anyone know the reason of this problem?

P.S. 10-core PC running MATLAB 2020b with latest updates with MOSEK 9.2.35

18-core PC running MATLAB 2021a with latest updates with MOSEK 9.2.36

I also checked the taskmgr.exe, unlike the 10-core PC, I found in 18-core PC, 10 matlab tasks are in one group, the other 10 tasks are seperated....

6 个评论
显示 4更早的评论隐藏 4更早的评论

Walter Roberson 2021-8-28

parfor has communications overheads, and if you give each node too little to do, then you can end up spending more time communicating than you get back from splitting the work. So it is possible for progress to get slower as you add more workers. It would be interesting to restrict both machines to (say) four cores and see what their performance is.

The studies I looked at a number of years ago suggested that 4 to 6 cores was the "sweet spot" for a variety of tasks -- at least with the hardware of those days. There are tasks for which the communications might be comparatively small and so net performance can improve as you add cores, even into the tens of thousands, but most code has to be specially written for that kind of situation.

Mingyang Sun 2021-9-2

When using the MOSEK (interior-point or integer programming) inside parfor loop, it is better to turn off the multi-threads in MOSEK, otherwise, by default, MOSEK will using all cores.

param.MSK_IPAR_NUM_THREADS = 1;

param.MSK_IPAR_INTPNT_MULTI_THREAD = 'MSK_OFF';

After I turn off the multi-threads in MOSEK, parfor-loop is now approx. 17 times faster than for-loop!

请先登录，再进行评论。

请先登录，再回答此问题。