Custom Neural Network Sample Code Fails on GPU

Question

Zachary David 2017-9-5

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/355324-custom-neural-network-sample-code-fails-on-gpu

编辑： Zachary David 2017-9-12

The sample architecture laid out in this guide https://www.mathworks.com/help/nnet/ug/create-and-train-custom-neural-network-architectures.html

Runs on CPU but fails on GPU.

The extra steps I added which are required to run efficiently on GPU

xg = nndata2gpu(X);
yg = nndata2gpu(Y);

and then either net = configure(net,X,T); OR using the [Xs,Xi,Ai,Ys,EWs,shift] = preparets(net,X,Y); method and sending all of the results to the gpu.

The problem appears to be when there are multiple input vectors of differing lengths. I have confirmed that this fails on simpler examples of similar type.

If there's no workaround this appears to be an issue preventing multiple input recurrent networks from running on the gpu. (I can try hacking a solution where every entry is a separate input vector, but this increases complexity and possibly might not work in all situations)

The Exception is: Dimensions of matrices being concatenated are not consistent.

Trace:

Error using cat
Dimensions of matrices being concatenated are not consistent.
Error in cell2mat (line 75)
            m{n} = cat(2,c{n,:});
Error in nnGPU.pc (line 52)
xoffset = cell2mat(xoffset');
Error in nncalc.preCalcData>iPreCalcDataForNNDATA2GPU (line 43)
    data.Pc = mode.pc(net,data.X,data.Xi,data.Q,data.TS,hints);
Error in nncalc.preCalcData (line 7)
    data = iPreCalcDataForNNDATA2GPU(net,data,doPc);
Error in nncalc.setup1>setupImpl (line 176)
  calcData = nncalc.preCalcData(matlabMode,matlabHints,net,data,doPc,doPd,calcHints.doFlattenTime);
Error in nncalc.setup1 (line 16)
[calcMode,calcNet,calcData,calcHints,net,resourceText] = setupImpl(calcMode,net,data);
Error in nncalc.setup (line 7)
[calcMode,calcNet,calcData,calcHints,net,resourceText] = nncalc.setup1(calcMode,net,data);
Error in network/train (line 357)
    [calcLib,calcNet,net,resourceText] = nncalc.setup(calcMode,net,data);

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Amanjit Dulai 2017-9-8

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/355324-custom-neural-network-sample-code-fails-on-gpu#answer_280782

在 MATLAB Online 中打开

Instead of using nndata2gpu to prepare your data for GPU training, you can use the 'useGPU' flag. Just change the call to train in the example to the following:

net = train(net,X,T, 'useGPU', 'yes');

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Zachary David 2017-9-12

编辑：Zachary David 2017-9-12

Thanks Amanjit, however the size of the data sets (200 sequences of 7000 entries) I'm using in practice prohibit this method from being practical. In fact, using a batch size of 1 with a sequence length of 7000 never completes.

(this is on a Titan Xp with "computation only" CUDA settings enabled.)

请先登录，再进行评论。