How to transfer date from gpu to workplace faster?

Question

Han F 2021-1-6

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/710293-how-to-transfer-date-from-gpu-to-workplace-faster

评论： Walter Roberson 2021-1-10

Hi everyone,

There is bottleneck in my codes.

I use CUDA to accelerate some parts of my algorithm, and the results should be returned workplace for following calculation.

I use gather(), firstly, to transfer data from gpu to workplace. However, the profiler shows that execution time of gather() (1000s) account for almost 1/3 of total time (3000s). My question is any stratgies can help to improve data transfer process?

ALso, I posted a similar question https://www.mathworks.com/matlabcentral/answers/703592-why-does-the-gather-function-only-take-around-0-001-seconds-at-command-window-while-1s-in-a-loo . But I still confuse about how gputimeit() or wait(gpuDevice) work?

Thank you in advance!

3 个评论
显示 1更早的评论隐藏 1更早的评论

Han F 2021-1-6

Hi Walter, thank you for your comment. Following profiler is the latest result. Line 649 and 651 speed 650 seconds and total time is 2900s. [dk1,...,dk10] is a 1000*9 matrix.

Do you mean there is no way to reduce the time spend waiting for the GPU to finish ?

Joss Knight 2021-1-10

Can you explain how you expect to reduce the time spent waiting for the GPU to finish? The GPU has to finish computing a result before it can return a result! The only way to make that go faster is to get a faster GPU - or maybe write your own CUDA code, if you think you can implement a faster algorithm.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Walter Roberson 2021-1-6

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/710293-how-to-transfer-date-from-gpu-to-workplace-faster#answer_592228

在 MATLAB Online 中打开

These are the strategies to reduce the time spent in gather():

do less complex operations on the GPU so that the GPU finishes the task faster
Use smaller output matrices -- ideally only a scalar
Use a faster GPU with faster memory and faster I/O bus
Use faster memory on you CPU
don't use the GPU for operations that would complete about as quickly on the CPU. For example, even though addition is faster on the GPU, if you use large arrays, most of the time is spent transfering the data to the GPU and the results back from the GPU

There is no

gpuprefs(gpudevice, 'gather_rate', 'ludicrous_speed')

You have to wait for the GPU to finish executing the task given to it, and the driver has to do a DMA operation to get the data back.

Oh yes, there is one more strategy to reduce the time spent in gather():

use Linux instead of Windows. The Windows drivers are less efficient (for reasons having to do with the architecture requirements that Windows places.)