Activations of freezed layers are different between before/after training, why?

Question

ntinoson 2018-6-29

1
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/408072-activations-of-freezed-layers-are-different-between-before-after-training-why

评论： Amanjit Dulai 2018-8-28

I follow the example "transfer-learning-using-googlenet" where, the last 3 layers ('loss3-classifier','prob','output') are replaced with 3 new ones. Then I 'freeze' the first 141 layers (that is up to and including 'pool5-drop_7x7_s1'):

layers(1:141) = freezeWeights(layers(1:141));
lgraph = createLgraphUsingConnections(layers,connections);

Then I follow fine-tuning.

Since 'pool5-7x7_s1' is BEFORE 'pool5-drop_7x7_s1', I would expect that the following two vectors were the same:

b_orig=  activations(net_orig, I, 'pool5-7x7_s1');
b_tune=  activations(net_tune, I, 'pool5-7x7_s1');

but they aren't!... Any idea why?

p.s. I also tried the activation of several other layers BEFORE 'pool5-drop_7x7_s1', and I got different vectors.... 'I' is an image, 'net_orig=googlenet;', and 'net_tune' is the resulting net after tuning.

2 个评论
显示无隐藏无

conngame 2018-7-15

I have the same problem using alexnet. Any explanations to this question?

ntinoson 2018-7-17

I did also try other CNNs, and same result. Activation of freezed layers after fine-tuning is different to that before fine-tuning (for the same input image of course). If anyone comes up with an explanation, drop a line!

请先登录，再进行评论。

请先登录，再回答此问题。