GPU computations slowing down

2 次查看（过去 30 天）
Peter
Peter 2017-1-5
评论： Joss Knight 2017-1-8
Within a loop, my code carries out some iterative solving procedure on the GPU. With every loop iteration, the time taken by the solving procedure increases. There doesn't seem to be a memory leak: gpuDevice consistently reports approximately the following:
TotalMemory: 1.2885e+10
AvailableMemory: 5.6773e+09
However, when I pause the loop for a while by means of a debug stop, after relaunching again, computations are up to speed again, after which they start slowing down again...
I'm running matlab2016b on Windows 10, using a Nvidia Titan X (Pascal) GPU. Any suggestions for this problem?
  3 个评论
显示 1更早的评论
Jan
Jan 2017-1-5
Without seeing the code or knowing any details, it is hard to guess, what's going on. Did you control the temperature of the graphics board?
Peter
Peter 2017-1-6
thanks Jan, I already started to be suspicious about temperature and monitored it. After launching the matlab script, temperature starts rising, GPU performance starts being capped by Temperature. (See performance monitor screen shot below) After reaching 95(!) degrees, the crash occurred again. So we do have strong indications for a cooling issue... What does this mean?
  • a) I got myself a wrong system set-up, need better cooling ?
  • b) this GPU card doesn't suit my needs, perhaps go for one of the Keplers (more expensive for similar memory)?

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Peter
Peter 2017-1-6
Monitoring the GPU performance revealed that most probably the temperature is causing the issue: Slowing down of performance goes with rising of temperature and performance is capped by temperature.
Crash of the GPU occurred when GPU reached 95 degrees...
  1 个评论
Joss Knight
Joss Knight 2017-1-8
Most people who report issues like this are simply seeing an issue of asynchronous execution. They see fast times until the kernel execution queue is full, and then they start seeing the actual complete computation times.
To avoid this try using wait(gpuDevice) to add synchronisation points to your code.

请先登录，再进行评论。

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 GPU Computing 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by