What happen to the CUDA cache mem?

Question

fpexp 2017-12-14

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/372851-what-happen-to-the-cuda-cache-mem

评论： Joss Knight 2018-7-18

Hello there. I am a newbie with the GPU computing with Matlab, so apologize if the question sounds silly. I am trying to optimise some computation I am doing with the GPU. I believe it is well configured. I am doing some testing to understand how the GPU reacts to different commands and choose the best programming strategy. I have incurred in the following thing. I would appreciate some elucidation about the mechanism by which this feature happens. I am running a Geforce GTX 1080 ti. I do the following:

A = rand([100 100 100 100 10],'single','gpuArray')
tic,permute(A,[3 2 1 5 4]),wait(M.SelectedDevice()),toc

(trying to see how long does it take to permute a matrix)

now, if I ask the parallel.gpu.GPUDeviceManager.instance.SelectedDevice().AvailableMemory (read the available memory), then I can run a permute again. However, if I run two consecutive permute, I get the following

Error using gpuArray/permute Out of memory on device. To view more detail about available memory on the GPU, use 'gpuDevice()'. If the problem persists, reset the GPU by calling 'gpuDevice(1)'.

WHY?

2 个评论
显示无隐藏无

Walter Roberson 2017-12-14

Have you tried calling gather() after the permute?

fpexp 2017-12-14

nope, in fact the result is not stored anywhere. I would have expected the RAM area to be released immediately

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Joss Knight 2017-12-19

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/372851-what-happen-to-the-cuda-cache-mem#answer_296904

The result is stored as the variable ans, which means you have less memory the second time round.

4 个评论
显示 2更早的评论隐藏 2更早的评论

giovanni esposito 2018-7-18

编辑：giovanni esposito 2018-7-18

在 MATLAB Online 中打开

hence, for example this code shall free all gpus memory ad the end of each loop, correct ? I try to do this but memory is still busy at the end of each loop.

clear all
RefreshGPU = 100;
NW =  gpuDeviceCount;
nw = 1:NW;
poolobj = gcp('nocreate'); % If no pool, do not create new one.
if isempty(poolobj)
    ParObj = parpool('local',NW);
else
    delete(gcp);
      ParObj = parpool('local',NW);
  end
a=rand(NW,1e5);
Nloop = 1e5;
for kk=1:Nloop    
    spmd
        b = somefunction(a(labindex,:)); % this function do something on GPUs
    end    
    clear b
end

Joss Knight 2018-7-18

No, you are calling clear b on the client. You need to do it inside the SPMD block.

请先登录，再进行评论。

Answer 2

Jeffrey Daniels 2018-3-12

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/372851-what-happen-to-the-cuda-cache-mem#answer_309729

FYI - For anyone else having similar problems, I get similar errors when I run too many workers. The GPU is being shared by each of the CPU workers and if you have too large or too many GPU matricies you will run out of memory on the GPU. One solution is to open the Cluster Profile Manager from the Parallel menu and reduce the number of workers in your Cluster Profile.