"Maximum variable size allowed on the device is exceeded." when using predict on GPU

50 次查看（过去 30 天）
hanspeter
hanspeter 2024-10-30，9:58
评论： hanspeter 2024-11-3，13:17
Hello there,
I have a neural net and want to do a prediction using predict. When I do that I get the error
Error using dlnetwork/predict
Execution failed during layer(s) 'fc1'.
Caused by:
Error using parallel.internal.gpu.mtimesAdd
Maximum variable size allowed on the device is exceeded.
Afaik it's because on GPU Matlab only supports 2^32 elements in one array. Whats the best way to circumvent this?
I know I can disable the GPU completely by setting CUDA_VISIBLE_DEVICES to -1, but I still want to use it in other functions.
The data I pass into predict is a normal array and not a gpuarray.
Is there a way to force predict to use CPU?
Cheers

请先登录，再回答此问题。

采纳的回答

Joss Knight
Joss Knight 2024-11-1，15:39
编辑：Joss Knight 2024-11-1，15:41
Is there a way to force predict to use CPU?
Yes. Pass data in as an in-memory array. If your data is on the device, you may need to call gather.
z = predict(net, gather(x));
If this does not work it will be because some of your network weights are on the GPU. So you may need to call gather on the network as well:
net = dlupdate(@gather, net);
net.State = dlupdate(@gather, net.State);
It's much more likely, however, that you're just passing too much data into your network. If you are trying to call predict on a large dataset, consider using minibatchpredict instead, to process the data in batches.

更多回答（1 个）

Shivam Lahoti
Shivam Lahoti 2024-10-30，11:22
Hi hanspeter,
I understand you are encountering a “Maximum variable size allowed on the device is exceeded" error when using predict on a GPU.
As you mentioned, the current maximum number of elements allowed in a gpuArray is intmax('int32'), or 2^31. This is because CUBLAS passes array lengths as int (int32_T). This is a known limitation and is mentioned in the documentation link below:
https://www.mathworks.com/help/parallel-computing/gpuarray.html#mw_6d599a79-d350-4ffd-8b2c-0b4b353f858b
Moreover, none of the following can exceed intmax('int32'):
  • The number of elements of a dense array.
  • The number of nonzero elements of a sparse array.
  • The size in any given dimension. For example, zeros(0,3e9,'gpuArray') is not allowed.
I hope this helps you understand the issue.
Regards,
Shivam

类别

在 Help Center 和 File Exchange 中查找有关 Image Data Workflows 的更多信息

标签

产品


版本

R2023b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by