Deep Learning Memory issues for BiLstm mat files
2 次查看(过去 30 天)
显示 更早的评论
Hi,
I have a huge dataset of over 20GB of numercial time-series data stored in files as shown below:
x1_1 = double(23*921600)
...
I have about 250 of these mat files. and the label or target of these are single row categorical array that shows 0 and 1.
y1_1= categorical(1*921600)
...
I loaded 5 of each file and was able to train and classiy them using a BiLSTM Netowrok of 3 hidden layers with over 300 neurorns. but now i want to run all of them. My example is very similar to this example, I also tried this which was no help. I know that i should use a datastore of some kind but i tried most[File,Tall] of them and couldn't solve the issue. Any ideas how to solve this issue? Thank you.
0 个评论
回答(1 个)
Divya Gaddipati
2019-7-19
Hi,
You could use the Custom Mini-batch Datastore which uses the function sequenceDatastore that reads data from the specified folder and obtains labels from the subfolder names.
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Image Data Workflows 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!