CUDN_STATUS_BAD_PARAM:While running CNN in Matlab
6 次查看(过去 30 天)
显示 更早的评论
I am currently trying to train and test a network in matlab using cnn using gpu. I have gpu -GTX 1070 and CUDA 8.0 and cudnn 5.1. I followed the example of object detection using deep learning provided by mathworks from the link https://www.mathworks.com/help/vision/examples/object-detection-using-deep-learning.html
After I run the following lines:
% Train a network.
Net1 = trainNetwork(trainingImages, trainingLabels, layers, opts);
It is giving me the trained network as output. But after I run the following line
YTest = predict(Net1, X_test);
it is giving me following error
Error using nnet.internal.cnngpu.convolveForward2D
Unexpected error calling cuDNN: CUDNN_STATUS_BAD_PARAM.
Error in nnet.internal.cnn.layer.Convolution2D/doForward (line 218)
Z = nnet.internal.cnngpu.convolveForward2D( ...
Error in nnet.internal.cnn.layer.Convolution2D/forwardNormal (line 195)
Z = this.doForward(X,this.Weights.Value,this.Bias.Value);
Error in SeriesNetwork/predict (line 139)
Y(:,:,:,i) = predictNetwork.predict(X);
Any help will be appreciated. Thank you
5 个评论
Patrick Marmaroli
2019-4-23
Hello, while it worked fine up to now, I just experienced the same error. Impossible to make it work again. I don't know how to figure it out. Any help is welcolmed.
Training on single GPU.
Initializing input data normalization.
|========================================================================================|
| Epoch | Iteration | Time Elapsed | Mini-batch | Mini-batch | Base Learning |
| | | (hh:mm:ss) | Accuracy | Loss | Rate |
|========================================================================================|
Error using trainNetwork (line 165)
Unexpected error calling cuDNN: CUDNN_STATUS_BAD_PARAM.
Here are my GPU properties
gpuDevice
ans =
CUDADevice with properties:
Name: 'Quadro M620'
Index: 1
ComputeCapability: '5.0'
SupportsDouble: 1
DriverVersion: 10.1000
ToolkitVersion: 10
MaxThreadsPerBlock: 1024
MaxShmemPerBlock: 49152
MaxThreadBlockSize: [1024 1024 64]
MaxGridSize: [2.1475e+09 65535 65535]
SIMDWidth: 32
TotalMemory: 2.1475e+09
AvailableMemory: 1.5266e+09
MultiprocessorCount: 4
ClockRateKHz: 1017500
ComputeMode: 'Default'
GPUOverlapsTransfers: 1
KernelExecutionTimeout: 1
CanMapHostMemory: 1
DeviceSupported: 1
DeviceSelected: 1
From the training data side, everything seems correct....
K>> size(XTrain)
ans =
13 7 1 304
K>> size(YTrain)
ans =
304 1
K>> sum(sum(sum(isnan(XTrain))))
ans =
0
K>> sum(isnan(double(YTrain)))
ans =
0
Thank you for your suggestions.
Patrick Marmaroli
2019-4-26
Just a quick update about my last comment, in my case the error came from a bad design of the max pool layer.
As shown below my mistake was to apply a 3x3 max pool onto an input image of 4x2.
Unfortunately this error hasn't been detected by the DLN Analyzer (analyzeNetwork).
![Clipboard03.jpg](https://www.mathworks.com/matlabcentral/answers/uploaded_files/216727/Clipboard03.jpeg)
回答(0 个)
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Image Data Workflows 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!