Folding batch normalization into preceding convolution

15 次查看（过去 30 天）
Kai Tan
Kai Tan 2019-2-17
Trying to port a trained MATLAB CNN models to another framework. I would like to get rid of batch norm (BN) layers by folding the parameters into the preceding convolution layers. I use the following formulation:
% layer is BatchNorm layer
m = layer.TrainedMean;
v = layer.TrainedVariance;
offset = layer.Offset;
scale = layer.Scale;
ep = layer.Epsilon;
% adjust the weights:
for j = 1:size(scale, 3) % the number of output channels
denom = sqrt(v(1,1,j)+ep);
% adjust the 4D convolution weights
new_w(:,:,:,j) = scale(1,1,j)*w(:,:,:,j)/denom;
% adjust the convolution biases
new_b(1,1,j) = scale(1,1,j)*(b(1,1,j)-m(1,1,j))/denom + offset(1,1,j);
end
However, when I compare the outputs of the BN layer to the outputs of the convolution layer, where the BN parameters were folded into, I am getting different results.
Left (BN layer), Right (Conv Layer with BN parameters folded).
untitled.jpg
Has anyone successfully done this?

请先登录，再回答此问题。

回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Image Data Workflows 的更多信息

标签

产品


版本

R2018a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by