Matlab Shallow Network Mini Batch Training

Question

Lynn Marciano 2018-7-12

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/410036-matlab-shallow-network-mini-batch-training

评论： Jingjun Liu 2019-11-23

Hello, I have been training my data through the patternnet provided by matlab and really like it's functionality and I've been seeing great results using it. I have a problem however, I want to start investigating all the functions that can be adjusted under the hood of the default patternnet, but I have such a large data size, that even though I'm connected to a cluster, my model takes about 10 hours at minimum to train. I know there are capabilities with training on the GPU but after several attempts, it says I have no memory for training. I know having a minibatch might be able to compensate for this, but I'm not entirely sure if I have to create a datastore for the minibatch to be effective. If anyone has input a minibatch into the shallow network inputs and trained on GPU, please give me some insight on the right direction to go with this. Thanks in advance.

4 个评论
显示 2更早的评论隐藏 2更早的评论

Ilya Makarov 2019-1-12

在 MATLAB Online 中打开

Hi Lynn,

In my experience, it's impossible to use mini-batch training using shallow networks (patternnet) - Matlab seems to use the whole dataset as a single batch. In case you have millions of training vectors, you'll probably get an error like "Out-of-memory" or so.

There's a possibility to use an incremental training but that's not the way we wish to proceed.

In my experience the only way to go is to create a deep network for classifying sequences and then re-configure it to behave like a deep multilayer perceptron. This is how it works (at least, on my machine). Suppose you have 1M training vectors (each vector has N dimensions)

1) create a deep network for sequence classification (suppose it will have 2 hidden layers with 1024 neurons per layer, Rectified Linear Unit as an output for any hidden layer and the output classification layer with 100 classes (outputs))

layers = [...
    sequenceInputLayer(N)
    
    fullyConnectedLayer(1024)
    relyLayer
    
    fullyConnectedLayer(1024)
    relyLayer
    
    fullyConnectedLayer(100)
    softmaxLayer
    classificationLayer]

2) Choose needful training options next

options = trainingOptions(%insert your parameters here
          'MiniBatchSize',500, ...
           'MaxEpochs',100, ...
           %insert your parameters here)

Now the network we're creating will train parameters using mini-batches of 500 vectors each for 100 epochs maximum

3) prepare your data. For doing this create a cell massive inputs{1000000,1}, where the i-th cell (== inputs{i,1}) will have exactly one input training vector (the whole cell massive will have 1M cells in total)

After that create a cell array of class labels labels{1000000,1}, where the i-th cell (==labels{i,1}) will have exactly one categorical label for the given input training vector of the same index (please don't forget to make a label categorical)

4) train your network

net=trainNetwork(inputs,labels,layers,options)

Hope it helps.

sayak nag 2019-3-15

Please help I am following your advice but it seems that despite whatever I specify as my mini-batch the network is training in batch mode i.e. no of iterations per epoch is 1.

Jingjun Liu 2019-11-23

The mini-batch does not work for sequenceInputLayer. That's what I found.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Greg Heath 2018-12-20

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/410036-matlab-shallow-network-mini-batch-training#answer_353218

If you have a huge dataset, it is often rewarding to just randomly divide it into m subsets. Then design with 1 and test on m-1. If the subsets are sufficiently large. it is not necessary to use m-fold cross-validation. However, you may want to design more than one net.

Hope this helps.

Thank you for formally accepting my answer

Greg

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

Matlab Shallow Network Mini Batch Training

4 个评论
显示 2更早的评论隐藏 2更早的评论

回答（1 个）

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

Matlab Shallow Network Mini Batch Training

4 个评论 显示 2更早的评论隐藏 2更早的评论

回答（1 个）

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

4 个评论
显示 2更早的评论隐藏 2更早的评论

0 个评论
显示 -2更早的评论隐藏 -2更早的评论