GPU memory usage for Hadamard product

2024 7 12
4 个回答
更新时间：2024 7 18
17 次查看（30 天）

1 个投票

I have a GPU with 48 Gb of RAM.
I have a large matrix A (complex single 45927x45927, gpuArray) taking 16 Gb of my GPU.
I get to the following Hadamard product with a vector B (complex single 45927x1, gpuArray) with approximately 28 Gb of free RAM on the GPU:
C = A .* (B.')
But this throws an "out of memory on the device" error, despite there being enough memory for C (also 16 Gb). Why is this happening? Does MATLAB implicitly convert B to a full matrix? Is there a way to get this multiplication to work on the GPU within the available memory? (maybe this has been fixed in later releases of MATLAB?)

8 个评论
显示 6更早的评论 隐藏 6更早的评论

Walter Roberson
Walter Roberson 2024-7-14
I seem to recall that additional memory is needed for complex arrays, due to differences in the layout of complex vectors.
Walter Roberson
Walter Roberson 2024-7-14
The rule of thumb is that you should avoid having arrays that are more than 1/3 of the GPU memory: otherwise the creation of temporary variables for calculation runs the risk of overflowing memory.
Walter Roberson
Walter Roberson 2024-7-14
You could try
N = size(A,1);
C = zeros(N, N, 'gpuArray');
for K = 1 : N
C(:,K) = A(:,K) .* B(1,K);
end
Thanks for the suggestion. I guess you meant B(K, 1) ?
I tried:
C = zeros(size(A), "like", B);
for n = 1 : length(B)
C(:, n) = A(:, n) .* B(n);
end
This works, technically, but it becomes extremely slow and kind of defeats the purpose of using the GPU...
(The first line is the fastest way I found to directly initialize a complex single array on the GPU)
Matt J
Matt J 2024-7-17
编辑：Matt J 2024-7-18
But the fact that it works without giving you out-of-memory error messages is important. It indicates that the problems you are having are because of temporary memory allocation, and not because MATLAB is failing to allocate space for the final result C.
Matt J
Matt J 2024-7-18
编辑：Matt J 2024-7-18
What happens if you do other operations, like,
C=A+A
C=A+B
C=gpuArray.zeros(size(A))+1j*gpuArray.ones(size(A));

请先登录，再进行评论。

请先登录，再回答此问题。

回答（4 个）

Sivsankar
Sivsankar 2024-7-12

0 个投票

Hi,
I believe that you are probably limited by the size of RAM of your GPU. I can suggest some ideas that you can implement which may work.
  • Upgrade to the latest version of MATLAB.
  • Pre-allocate the memory for the resultant gpuArray matrix C.
  • Clear any existing gpuArray that may take up the RAM.
  • Reset GPU device to release memory.
You can leverage this MATLAB answer that highlights a similar issue.
https://www.mathworks.com/matlabcentral/answers/504069-gpu-out-of-memory
You can also make use of this MathWorks documentation that explains on how to resolve such “out of memory” errors.
https://www.mathworks.com/help/matlab/matlab_prog/resolving-out-of-memory-errors.html
Hope this works for you. Thanks.

1 个评论
显示 -1更早的评论 隐藏 -1更早的评论

Thanks for the suggestions. I tried clearing and pre-allocating, but MATLAB still seemingly attempts to allocate 3 * 16 Gb for the operation, even though it should take only 2 * 16 Gb. Is it meant to be that way? Otherwise, I will try with a newer version of MATLAB to see if this has been fixed.

请先登录，再进行评论。

Matt J
Matt J 2024-7-14

0 个投票

This is just a guess, but possibly the GPU memory is fragmented. I.e., maybe your available 28GB is broken into blocks that are smaller than 16GB and therefore Matlab cannot find enough space to allocate C.
Catalytic
Catalytic 2024-7-14
编辑：Catalytic 2024-7-14

0 个投票

Does MATLAB implicitly convert B to a full matrix?
When you say this, are you implying that issparse(B)=true? If so, then it is possible that C could consume up to 32 GB.

2 个评论
显示 无 隐藏 无

Matt J
Matt J 2024-7-14
I don't think that would be possible. If B were sparse, you would get a different error if you tried to multiply it with a single matrix.
No, I improperly meant "full" as in matrix instead of vector, sorry for the confusion. I was wondering whether B is repeated from 45927x1 to 45927x45927, that could explain the memory requirement.

请先登录，再进行评论。

Joss Knight
Joss Knight 2024-7-18
编辑：Joss Knight 2024-7-18

0 个投票

I can't reproduce this.
You say you have 48Gb of GPU memory available...have you checked this? Try running gpuDevice and looking at the output. Maybe you have other variables taking up space. Maybe you have another MATLAB running and using the GPU?
Also, did it work the first time you ran it but not the second? If C already exists then you would need space for A, C and the new C all at once, and you would run out of memory.

3 个评论
显示 1更早的评论 隐藏 1更早的评论

I get there with more or less 28 Gb free...
I now tried with another GPU with 24 Gb of total RAM and smaller matrix size (~8 Gb), but I still get the same error.
This is a copy of the command window when I type some commands when hitting a breakpoint right before the operation:
473 C = A .* (B.');
>> gpuDevice
ans =
CUDADevice with properties:
Name: 'NVIDIA A10'
Index: 1
ComputeCapability: '8.6'
SupportsDouble: 1
DriverVersion: 12.2000
ToolkitVersion: 11
MaxThreadsPerBlock: 1024
MaxShmemPerBlock: 49152
MaxThreadBlockSize: [1024 1024 64]
MaxGridSize: [2.1475e+09 65535 65535]
SIMDWidth: 32
TotalMemory: 2.3942e+10
AvailableMemory: 1.4382e+10
MultiprocessorCount: 72
ClockRateKHz: 1695000
ComputeMode: 'Default'
GPUOverlapsTransfers: 1
KernelExecutionTimeout: 0
CanMapHostMemory: 1
DeviceSupported: 1
DeviceAvailable: 1
DeviceSelected: 1
>> (numel(A) * 32 * 2 / 8) / 1024 / 1024 / 1024 % [Gb]
ans =
8.0181
>> (numel(B) * 32 * 2 / 8) / 1024 / 1024 % [Mb]
ans =
0.2503
>> (gpuDevice().AvailableMemory) / 1024 / 1024 / 1024 % [Gb]
ans =
13.3945
And when I hit continue, the error happens:
Error using .*
Out of memory on device. To view more detail about available memory on the GPU, use 'gpuDevice()'. If the problem persists, reset the GPU by
calling 'gpuDevice(1)'.
Joss Knight
Joss Knight 2024-7-18
Your gpuDevice output seems to indicate you have quite an old version of MATLAB. What version are you on?
Joss Knight
Joss Knight 2024-7-18
Never mind. I can't reproduce this in any old version, on Linux or Windows.
Just to be sure, make sure your script is standalone and is the only thing you run after you've started MATLAB. It should contain no other computation than creating A and B and then computing C, so for instance:
n = 45927;
proto = gpuArray(single(1i));
A = randn(n,'like',proto);
B = randn(n,1,'like',proto);
C = A.*(B.');
gpuDevice
whos

请先登录，再进行评论。

类别

在 帮助中心 和 File Exchange 中查找有关 Parallel and Cloud 的更多信息

产品

版本

R2021a

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by