classify large data deep learning out of memory

1 次查看（过去 30 天）
michael scheinfeild
回答： michael scheinfeild 2021-2-1
hi i have many movies as in example
https://www.mathworks.com/help/deeplearning/ug/classify-videos-using-deep-learning.html;jsessionid=39b9f3a2fe4ea2b09aa92f076d0f
it run out of memory for me
so i did few steps
1) save each video feature + label in mat files in two folders train and validation
1.mat
{[1024×92 single] [chew]}
2.mat
{[1024×80 single] [run]}
2) define datastore
TrainStore = fileDatastore(trainFolder,'ReadFcn',@load,'FileExtensions','.mat');
ValidationStore = fileDatastore(validationFolder,'ReadFcn',@load,'FileExtensions','.mat');
so for example
data1 = read(ValidationStore);
result in :
sequences: {[1024×122 single] [talk]}
3) define options :
options = trainingOptions('adam', ...
'MiniBatchSize',miniBatchSize, ...
'InitialLearnRate',1e-4, ...
'GradientThreshold',2, ...
'Shuffle','every-epoch', ...
'ValidationData',ValidationStore, ...
'ValidationFrequency',numIterationsPerEpoch, ...
'Plots','training-progress', ...
'Verbose',false);
get Error using trainingOptions (line 288)
The value of 'ValidationData' is invalid. The datastore used for 'ValidationData' must return a 2-column table or an M-by-2 cell array.
4) try to train
[netLSTM,info] = trainNetwork(TrainStore,layers,options);
dont work works
Error using trainNetwork (line 170)
Invalid training data. Responses must be nonempty.
documentation :
net = trainNetwork(ds,layers,options) trains a network using the datastore ds. For networks with multiple inputs, use this syntax with a combined or transformed datastore.
so probably issue of validation ...
TrainStore
TrainStore =
FileDatastore with properties:
Files: {
' ...\MLCODE\PROJECTS\DL\ClassifyPython\VideoClassify\Train\10.mat';
' ...\MLCODE\PROJECTS\DL\ClassifyPython\VideoClassify\Train\11.mat';
' ...\MLCODE\PROJECTS\DL\ClassifyPython\VideoClassify\Train\12.mat'

请先登录，再回答此问题。

采纳的回答

michael scheinfeild

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Image Data Workflows 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by