How to train a network and reconstructing the dataset on the fly

1 次查看（过去 30 天）
matan dud
matan dud 2022-10-23
回答： Sanjana 2023-6-8
Hello there!
I want to train a pretrained CNN, using image data that i create myself (based on saved data), because of the amount, i want to create the images on-the-fly, while runing the the training. and i'm not sure how to read and use
i.e, i have a few folders containing >1600 *.mat files of amplitude and phases of my image. but the dataset i want to train on is a calculation based on those *.mat files. using the calculations, the dataset can grow even uo to a million images.
since i can't save million images, how can i create smaller datasets and train/validate the network in one go?
do someone has an example of using mini-batches etc.?
thanks a lot!

请先登录，再回答此问题。

回答（1 个）

Sanjana
Sanjana 2023-6-8
Hi,
I understand that you want to train the network using mini-batches and looking for a way to manipulate dataset on the fly while training.
One possible solution is to use custom training loop.
Please refer to the following link for a reference to write custom training loop in MATLAB,
https://www.mathworks.com/help/deeplearning/ug/train-network-using-custom-training-loop.html
Please refer to the following example, for understanding manipulating data within a training loop and usage of mini batches for training,
https://www.mathworks.com/matlabcentral/fileexchange/76860-human-pose-estimation-with-deep-learning
Hope this helps!

类别

在 Help Center 和 File Exchange 中查找有关 Image Data Workflows 的更多信息

标签

产品


版本

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by