Forward function with frozen batch normalization layers

Question

Imola Fodor 2024-2-26

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2087088-forward-function-with-frozen-batch-normalization-layers

回答： Shivansh 2024-4-2

In my application i have both batch normalization and dropout, and i would like to perform MC dropout with the forward function, and ideally i would freeze the parameters TrainedMean and TrainedVariance for the batch normalization layers, but i cannot seem to understand is it possible. I have the bn layers after conv layers, and the dropout after the recurrent layer in my net. Thank you in advance

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Imola Fodor 2024-2-28

actually another problem is that i get an error for using the forward function, simply putting as input the same arguments as for the predict function..

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Shivansh 2024-4-2

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2087088-forward-function-with-frozen-batch-normalization-layers#answer_1434751

在 MATLAB Online 中打开

Hi Imola!

It seems you are facing issues with freezing the parameters of Batch normalization layers and the forward function for the network. For the first case, you can achieve this by setting the 'OffsetLearnRateFactor' and 'ScaleLearnRateFactor' as 0. This will ensure that the "Trained Mean" and "Trained variance" are not updated during the training. You can refer to the following related MATLAB answer for freezing the weights:

https://www.mathworks.com/matlabcentral/answers/548100-how-can-i-freeze-specific-weights-of-neural-network-model.

For the second query regarding the "forward" function, the "forward" function doesn't take the same inputs as the "predict" function. The "forward" function needs input to be a formatted dlarray. You can convert the current input array to a formatted dlarray using the following command:

dlA = dlarray(input,"format");

A sample example using wNet network