Semantic Segmentation - How many layers to replace in transfer learning?

Question

awezmm 2019-5-1

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/459530-semantic-segmentation-how-many-layers-to-replace-in-transfer-learning

评论： Guy Reading 2019-9-27

Im doing semantic segmentation using Resnet-18 with Deeplab v3+ (https://www.mathworks.com/help/vision/examples/semantic-segmentation-using-deep-learning.html)

However, I want to retain on progressively harder tasks and want to use transfer learning. How many of the final layers should I be replacing? How do I figure out how many layers I have?

On the analyzeNetwork it says I have 101 but I am using Resnet - 18 which I thought had much less?

4 个评论
显示 2更早的评论隐藏 2更早的评论

Tohru Kikawada 2019-5-11

编辑：Tohru Kikawada 2019-5-11

Philip, you'll need to try this on R2019a since the example has been revised to use DeepLab v3 instead of SegNet in the latest vresion.

Guy Reading 2019-9-27

I've been doing a bit of digging on the resnet18() <-> DeepLab V3+ connection, in this link MATLAB writes:

"This example creates the Deeplab v3+ network with weights initialized from a pre-trained Resnet-18 network"

But then we might ask, "how do we use the weights trained on one network to be used in another? Won't they be meaningless relative to Deeplab?

Then this link writes:

"The latest implementation of DeepLab supports multiple network backbones, like MobileNetv2, Xception, ResNet-v1, PNASNET and Auto-DeepLab."

So I guess we can treat Deeplab V3+ as some form of extension of resnet18 and thus can use the weights.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Tohru Kikawada 2019-5-2

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/459530-semantic-segmentation-how-many-layers-to-replace-in-transfer-learning#answer_373280

Did you see helperDeeplabv3PlusResnet18.m which is attached to the example as a supporting file? The supporting function might be helpful to create your own transfer learning network.

4 个评论
显示 2更早的评论隐藏 2更早的评论

awezmm 2019-5-7

I have already looked at the helperDeeplabv3PlusResnet18.m and said in my previous comment that is want not helpful...

Tohru Kikawada 2019-5-7

Sorry for the confusion. Someone asked me where the file can be found.

请先登录，再进行评论。

Answer 2

Guy Reading 2019-9-23

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/459530-semantic-segmentation-how-many-layers-to-replace-in-transfer-learning#answer_393087

在 MATLAB Online 中打开

Did you make any progress on this, @awezmm? I'm looking to do the same thing as you, with Resnet-18, too, and I got stuck at the same point as you so Googled & found your question here!

So far I've followed the (adapted for resnet) instructions of this tutorial:

https://uk.mathworks.com/videos/deep-learning-with-matlab-transfer-learning-in-10-lines-of-matlab-code-1487714838381.html

So for resnet that'd be:

%% load a pre-trained CNN
pretrainedFolder = fullfile(tempdir,'pretrainedNetwork');
pretrainedNetwork = fullfile(pretrainedFolder,'deeplabv3plusResnet18CamVid.mat'); 
data = load(pretrainedNetwork); 
net = data.net;
layers = net.Layers

This shows me all 101 layers for resnet. For me, personally, I'd like to classify 2 things (background or object) so I've edited the final layer to give me 2 things, but I'm pretty sure I need to do more layers and unsure which ones:

%% Modify the network to use 2 categories
layers(101) = pixelClassificationLayer; % note, in the example he uses classificationLayer as it's not semantic seg

& now I'm stuck! I'll comment r.e. letting you know any progress I've made... I'm going to look into the structure of resnet more, now, to get a better understanding of what I need to change and how...

5 个评论
显示 3更早的评论隐藏 3更早的评论

Guy Reading 2019-9-23

在 MATLAB Online 中打开

Okay I've got a network with no errors showing when analysing it with analyzeNetwork(), I'll see if this has actually adapted the network correctly tomorrow, though, when I feed it some data. I couldn't use the function noted in the previous comment, connectLayers, as it doesn't work with a DAGNetwork:

%% load a pre-trained CNN
pretrainedFolder = fullfile(tempdir,'pretrainedNetwork');
pretrainedNetwork = fullfile(pretrainedFolder,'deeplabv3plusResnet18CamVid.mat'); 
data = load(pretrainedNetwork); 
net = data.net;
layers = net.Layers;
lgraph = layerGraph(net);
%% Modify the network to use 2 categories
numClasses = 2;
objectCategories = categorical({'Lump','Background'});
newScorerLayer = convolution2dLayer(1,numClasses, ...
    'Name', 'NewScorer',...
    'NumChannels', 256,...
    'WeightLearnRateFactor', 1,...
    'BiasLearnRateFactor', 1);
newdec_ups2Layer = transposedConv2dLayer(8, 2, ...
    'Stride', 4, ...
    'Cropping', 2, ...
    'Name', 'newdec_upsample2',...
    'NumChannels', 2);
newDec_crop2Layer = crop2dLayer('centercrop', ...
    'Name', 'dec_crop2'); % crop layer requires connections
newSoftmaxLayer = softmaxLayer('Name', 'newSoftmax-out');
newLabelsLayer = pixelClassificationLayer('Name', 'newLabels',...
    'Classes', objectCategories);
% taken from: https://uk.mathworks.com/help/deeplearning/examples/train-deep-learning-network-to-classify-new-images.html
lgraph = replaceLayer(lgraph, 'scorer', newScorerLayer);
lgraph = replaceLayer(lgraph, 'dec_upsample2', newdec_ups2Layer);
lgraph = replaceLayer(lgraph, 'dec_crop2', newDec_crop2Layer);
lgraph = replaceLayer(lgraph, 'softmax-out', newSoftmaxLayer);
lgraph = replaceLayer(lgraph, 'labels', newLabelsLayer);
% check we have something that looks okay
analyzeNetwork(lgraph);

Guy Reading 2019-9-24

Oh right! So I looked to replace the layers which are specific to the amount of classes we want to categorise into. So layers 97:101 refer to layers which have a dimension set to 11, which was the original amount of classes:

Which is why I chose those. But I'm not 100% certain! Have you got this working for you at all? / what's the intuition behind picking the last layer, only?

Guy Reading 2019-9-26

To all that are reading: the above method worked for me & I'm starting to get labelled images from my model. I'm still not sure about which layers to freeze, if there are any suggestions with that I'd be interested to hear!

请先登录，再进行评论。

Semantic Segmentation - How many layers to replace in transfer learning?

4 个评论
显示 2更早的评论隐藏 2更早的评论

回答（2 个）

4 个评论
显示 2更早的评论隐藏 2更早的评论

5 个评论
显示 3更早的评论隐藏 3更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

Semantic Segmentation - How many layers to replace in transfer learning?

4 个评论 显示 2更早的评论隐藏 2更早的评论

回答（2 个）

4 个评论 显示 2更早的评论隐藏 2更早的评论

5 个评论 显示 3更早的评论隐藏 3更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

4 个评论
显示 2更早的评论隐藏 2更早的评论

4 个评论
显示 2更早的评论隐藏 2更早的评论

5 个评论
显示 3更早的评论隐藏 3更早的评论