About the efficiency of parallel computing

Question

Halil Eyyuboglu 2018-10-1

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/421646-about-the-efficiency-of-parallel-computing

编辑： Stephen23 2018-10-3

Hello ; We have a parallel setup in our university on a Dell computer which has 56 cores (all within the same machine). When we start the Admin Center, we see the number of cores as 56, but the count for number of workers is 112.

We have run different m files (both simple test and our actual jobs) with this configuration mainly using parfor loops.

The reduction in computation time is around 20 times.

Is this reasonable ?

We were expecting (taking into account the number of core) a computation time reduction around fifty times.

Your kind comments would be appreciated.

Halil Eyyuboglu

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

OCDER 2018-10-1

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/421646-about-the-efficiency-of-parallel-computing#answer_339273

Read this:

https://www.mathworks.com/help/optim/ug/improving-performance-with-parallel-computing.html

It depends on the communication overhead. If you have to send and collect data from workers frequently, you'll get less than Num_of_Workers reduction in computation time. In some cases (for small jobs), parallel computing is actually slower than single-core jobs. The best way to set this up is to ensure each worker has a good chunk of processing to do and only when it's done, send data once back to the main terminal.

Note that if each worker is trying to read/write data to hard drive, then this data accessing speed could be the limiting factor and even having 10000 workers may not help. To speed things up, first find the bottleneck in your code using normal for-loops and by using the profiler.

https://www.mathworks.com/help/matlab/ref/profile.html

2 个评论
显示无隐藏无

Stephen23 2018-10-3

在 MATLAB Online 中打开

Halil Eyyuboglu's "Answer" moved here and formatted correctly:

Thanks for the kind reply. Yes you are right, the coordination of the workers to send data and collect results would simetimes overshadow the computation time. But I think my case is quite straight forward. To clarify the point I have included below the piece of code I use for test. As far as I understand here, there is almost no communication cost, no need for coordination between workers. Yet, although I have 56 cores, I get only a reduction of 20 times compared with the case of replacing parfor by for (on the same machine). As I said in my previous correspondence, my expectation is at least 50 times. The same applies to my actual m file I run.

I would be quite happy, if I can receive some comments to the following points

1) When I run the admin center, for 56 cores, 112 workers are assingned, can I change to 56 workers, how ? Would this bring any further reductions in computation time ?

2) Is there something wrong with the test code I have given below. Or rather can you suggest a simple test in order to clearly see the benefit of many workers and the (linear) relationship between the number of workers and reduction in computation time.

I would appreciate your kind reply

    clear;clc;clf reset;close all
  Nmax = 100;tic
  for io = 1:10
  parfor im = 1:Nmax
      %im
      for im1 = 1:Nmax
  zm = cos(rand([Nmax,Nmax,21]));
      end;end;end
  toc

Stephen23 2018-10-3

编辑：Stephen23 2018-10-3

"As far as I understand here, there is almost no communication cost"

All parallel computations have an overhead, which is not just due to communication, but to actually setting up the parallel pool, splitting up the tasks, and combing the data afterwards. This is not a trivially small time, and cannot be disregarded.

"Is this reasonable ?"

Probably, it depends on your calculation, which you told us nothing about.

" As I said in my previous correspondence, my expectation is at least 50 times"

Then your expectation does not reflect the reality of how parallel computing actually works. Search this forum to know more: this topic has been discussed many times before:

https://www.mathworks.com/matlabcentral/answers/416003-parfor-slower-than-for

https://www.mathworks.com/matlabcentral/answers/389191-parfor-is-slower-than-for#answer_310685

https://www.mathworks.com/matlabcentral/answers/21478-slow-execution-of-parfor-loops-due-to-communication-overhead-load-static-data-into-worker-workspace

https://www.mathworks.com/matlabcentral/answers/32460-parfor-slower-than-for-for-simple-tutorial-example

https://www.mathworks.com/matlabcentral/answers/179468-trying-to-use-parfor-but-it-is-very-slow

https://www.mathworks.com/matlabcentral/answers/75166-why-is-parfor-slower-than-for

This is also explained in the documentation:

https://www.mathworks.com/help/distcomp/interactively-run-a-loop-in-parallel.html

请先登录，再进行评论。