Feed data into Neural Networks file-by-file

2 次查看（过去 30 天）
Le Anh
Le Anh 2019-10-31
回答： Joss Knight 2019-11-1
Currently for training my neural network, I need to load a huge dataset and it costs almost available RAM in the computer. That's why the training step (using GPU) is very slow. The pseudo-code is following:
Loading and pre-processing dataset (300 files)
Training Neural Network
So how can I don't have to load all at one time, and will be loaded in time the neural network is being trained, like this:
For file in Dataset
Loading and pre-processing file
Training Neural Network
or
Training Neural network
At some epoch:
Loading and Pre-processing a file in dataset
Continue to train.
Does MATLAB currently support this feature?

请先登录，再回答此问题。

回答（1 个）

Joss Knight
Joss Knight 2019-11-1
Datastores are designed for precisely this purpose. It may be that you're after an imageDatastore processed by a transform.

类别

在 Help Center 和 File Exchange 中查找有关 Image Data Workflows 的更多信息

标签

产品


版本

R2019b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by