Where and with how much data is the initial normalization done when training a network

1 次查看(过去 30 天)
When I start a network training (say VGG16) there is an initial step of data normalization. I was expecting all the dataset to load on the RAM, perform the corresponding preprocessing (done by the CPU) and later on, whenever it is needed, compose each batch, do the required additional preprocessing over the batch (done again by the CPU) and then transfer it to the GPU memory. However, seeing the behaviour while running my procedure, I can see that the RAM load does not go so high when doing the aforementioned data normalization (zeromean in this case). Why is this? Am I missing something? Is this initial preprocessing done per batch and stored again in the main memory or something like that?
Thank you so much in advance,
(By the way, sorry if this is a so basic question).
  1 个评论
yanqi liu
yanqi liu 2021-12-21
编辑:yanqi liu 2021-12-21
yes,sir,i aslo has this confused, may be the imageDatastore and augmentedImageDatastore use on cpu to prepare data,and then make them to gpu and train

请先登录,再进行评论。

回答(1 个)

Sahil Jain
Sahil Jain 2021-12-23
Hi Iker, the mean and standard deviation statistics are calculated iteratively by computing them for each mini-batch and then merging them with the running statistics. Since it's done batch-wise, not all the data is loaded onto the RAM so RAM usage isn't high. These calculated statistics are then used by the imageInputLayer to perform normalization.

类别

Help CenterFile Exchange 中查找有关 Image Data Workflows 的更多信息

产品


版本

R2021b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by