Where and with how much data is the initial normalization done when training a network

Question

IKER OTXOA DE LATORRE 2021-12-20

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1614655-where-and-with-how-much-data-is-the-initial-normalization-done-when-training-a-network

回答： Sahil Jain 2021-12-23

When I start a network training (say VGG16) there is an initial step of data normalization. I was expecting all the dataset to load on the RAM, perform the corresponding preprocessing (done by the CPU) and later on, whenever it is needed, compose each batch, do the required additional preprocessing over the batch (done again by the CPU) and then transfer it to the GPU memory. However, seeing the behaviour while running my procedure, I can see that the RAM load does not go so high when doing the aforementioned data normalization (zeromean in this case). Why is this? Am I missing something? Is this initial preprocessing done per batch and stored again in the main memory or something like that?

Thank you so much in advance,

(By the way, sorry if this is a so basic question).

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

yanqi liu 2021-12-21

编辑：yanqi liu 2021-12-21

yes，sir，i aslo has this confused， may be the imageDatastore and augmentedImageDatastore use on cpu to prepare data，and then make them to gpu and train

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Sahil Jain 2021-12-23

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1614655-where-and-with-how-much-data-is-the-initial-normalization-done-when-training-a-network#answer_860680

Hi Iker, the mean and standard deviation statistics are calculated iteratively by computing them for each mini-batch and then merging them with the running statistics. Since it's done batch-wise, not all the data is loaded onto the RAM so RAM usage isn't high. These calculated statistics are then used by the imageInputLayer to perform normalization.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

Where and with how much data is the initial normalization done when training a network

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

回答（1 个）

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

Where and with how much data is the initial normalization done when training a network

1 个评论 显示 -1更早的评论隐藏 -1更早的评论

回答（1 个）

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

0 个评论
显示 -2更早的评论隐藏 -2更早的评论