Transfer Learning on Unet: Image Input Size not matching for layers

5 次查看（过去 30 天）
Hridayi
Hridayi 2020-3-11
评论： sab kara 2020-8-9
Hello,
I am trying to retrain a couple layers of the U-net architecture with new data. However, some of the layers have a different input size and therefore, are giving me an error. How do I change the input to the layers without messing up the U-net architecture?
load SimNet1.mat
Net2 = SimNet1; %renaming U-Net
analyzeNetwork(Net2)
plot(Net2)
layers = Net2.Layers;
lgraph = layerGraph(layers)
lgraph = connectLayers(lgraph,'Encoder-Stage-1-ReLU-2','Decoder-Stage-4-DepthConcatenation/in2')
lgraph = connectLayers(lgraph,'Encoder-Stage-2-ReLU-2','Decoder-Stage-3-DepthConcatenation/in2')
lgraph = connectLayers(lgraph,'Encoder-Stage-3-ReLU-2','Decoder-Stage-2-DepthConcatenation/in2')
lgraph = connectLayers(lgraph, 'Encoder-Stage-4-ReLU-2','Decoder-Stage-1-DepthConcatenation/in2')
figure;
plot(lgraph)
%Real US Directories - Transfer Learning Images (260)
segDir = fullfile(%Fire Location); % Segmentations
USDir = fullfile(%File Location); % Labels = US Images
imds = imageDatastore(USDir); %DataStore of input training images - ultrasound images
classNames = ["bone","background"]; %labels
labelIDs = [1 0];
pxds = pixelLabelDatastore(segDir,classNames,labelIDs);
larray = [convolution2dLayer([1 1], 2,'NumChannels',64,'NumFilters',2,'Name','NewFinalConvLayer')];
lgraph = replaceLayer(lgraph,'Final-ConvolutionLayer',larray);
larray2 = pixelClassificationLayer('Name','NewPixelClassificationLayer','Classes',["bone" "background"]);
lgraph = replaceLayer(lgraph,'Segmentation-Layer',larray2);
larray3 = softmaxLayer('Name','NewSoftMaxLayer');
lgraph = replaceLayer(lgraph,'Softmax-Layer',larray3);
%Error is occuring for the next two layers
larray4 = [convolution2dLayer([3 3], 2,'NumChannels',128,'NumFilters',64,'Name','NewDecoderStage41layer')];
lgraph = replaceLayer(lgraph,'Decoder-Stage-4-Conv-1',larray4);
larray5 = [convolution2dLayer([3 3], 2,'NumChannels',64,'NumFilters',64,'Name','NewDecoderStage42Layer')];
lgraph = replaceLayer(lgraph,'Decoder-Stage-4-Conv-2',larray5);
figure;
plot(lgraph)
options = trainingOptions('adam','InitialLearnRate', 3e-4, ...
'MaxEpochs',100,'MiniBatchSize',15, ...
'Plots','training-progress','Shuffle','every-epoch');
ds = pixelLabelImageDatastore(imds,pxds) %returns a datastore based on input image data(imds - US images)
%and pxds (required network output - segmentations)
TLNet7 = trainNetwork(ds,lgraph,options)
save TLNet7
  2 个评论
Srivardhan Gadila
Srivardhan Gadila 2020-3-16
@Hridayi can you provide the values of the imageSize, numClasses & 'EncoderDepth' of your above U-Net network and also can you attach the SimNet1.mat file?
Hridayi
Hridayi 2020-3-16
@Srivardhan Gadila
These are the parameters used to train SimNet1:
imageSize = [256 256 1]; %Size of images being used
numClasses = 2; %Binary images so 2
encoderDepth = 4; %How many layers will there be in Unet apart from input and output
Unfortunately, I can't attach the file because it exceeds 5 MB even when I put it in a zipped folder.
I've also tried another way by freezing the weights of previous layers but that method does not give me good segmentations. It also won't allow me to change the PixelClassificationLayer or Softmax layer as they don't have any weights to change. Below is my code for that method.
I'm not sure which of these methods is better but I'm only getting results from my original method. However, by directly replacing the layers, I can't change other layers of the network which take different input sizes since it's a U-net which changes size at different encoder and decoders.
load SimNet3.mat %Same parameters as SimNet1 only difference is this network has more images
trainedNet = SimNet3; %renaming U-Net
layers = trainedNet.Layers;
connections = trainedNet.Connections;
layers(1:55) = FreezeWeights(layers(1:55)); %freeze up to the replaced layers
lgraph = CreateLgraphUsingConnections(layers,connections);
%FinalConvLayer
a = lgraph.Layers(56,1); %copy the layer
am = mean(reshape(a.Weights,1,[]));%find the mean of the weights
a.Weights = (2*am)*rand(size(a.Weights))- am; %replace layer with random matrix centered around the mean of the pretrained layer
am = mean(reshape(a.Bias,1,[])); % find mean of the bias
a.Bias = (2*am)*rand(size(a.Bias))- am;%replace layer with random matrix centered around the mean of the pretrained laye
lgraph = replaceLayer(lgraph,a.Name,a);%replace FinalConv
clear a am;
%Real US Directories - Transfer Learning Images (260 images)
segDir = fullfile('Directory'); % Segmentations
USDir = fullfile('Directory'); % Labels = US Images
imds = imageDatastore(USDir); %DataStore of input training images - ultrasound images
classNames = ["bone","background"]; %labels
labelIDs = [1 0];
pxds = pixelLabelDatastore(segDir,classNames,labelIDs); %Has Ground Truth pixel data on training images-Seg images are pixel labeled (only have 255/0
options = trainingOptions('adam','InitialLearnRate', 5e-4, ...
'MaxEpochs',100,'MiniBatchSize',15, ...
'Plots','training-progress','Shuffle','every-epoch');
ds = pixelLabelImageDatastore(imds,pxds); %returns a datastore based on input image data(imds - US images)
%and pxds (required network output - segmentations)
TLNet13 = trainNetwork(ds,lgraph,options)
save TLNet13

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Srivardhan Gadila
Srivardhan Gadila 2020-3-17
Based on the information on imageSize, numClasses & 'EncoderDepth', for the Net2 in your question add the value 'same' for the 'padding' Name-Value pair argument as follows for the NewFinalConvLayer, NewDecoderStage41layer & NewDecoderStage42Layer:
larray = [convolution2dLayer([1 1], 2,'NumChannels',64,'NumFilters',2,'Name','NewFinalConvLayer','Padding','same')];
lgraph = replaceLayer(lgraph,'Final-ConvolutionLayer',larray);
larray2 = pixelClassificationLayer('Name','NewPixelClassificationLayer','Classes',["bone" "background"]);
lgraph = replaceLayer(lgraph,'Segmentation-Layer',larray2);
larray3 = softmaxLayer('Name','NewSoftMaxLayer');
lgraph = replaceLayer(lgraph,'Softmax-Layer',larray3);
%Error is occuring for the next two layers
larray4 = [convolution2dLayer([3 3], 2,'NumChannels',128,'NumFilters',64,'Name','NewDecoderStage41layer','Padding','same')];
lgraph = replaceLayer(lgraph,'Decoder-Stage-4-Conv-1',larray4);
larray5 = [convolution2dLayer([3 3], 2,'NumChannels',64,'NumFilters',64,'Name','NewDecoderStage42Layer','Padding','same')];
lgraph = replaceLayer(lgraph,'Decoder-Stage-4-Conv-2',larray5);
analyzeNetwork(lgraph)
This would maintain the same U-Net structure but with the replaced layers and should no longer cause the mentioned error.

更多回答（1 个）

Samia OUKIL
Samia OUKIL 2020-6-1
How to segment color image(Skin lesion) with Unet and transfert learning?
Hello, I am a beginner in deep learning! So,I have a medical image database (Skin lesion) to segment with U-net and transfer learning! For that, I downloaded " u-net-release-2015-10-02.tar.gz"(https://lmb.informatik.uni-freiburg.de/people/ronneber/u-net/) but I don't know how to segment my database!
Please, if you can help and guide me to segment with Unet and how to use transfer learning?
  3 个评论
显示 1更早的评论
sab kara
sab kara 2020-8-9
slt if you find eny thing about this theme please give me a code i need it

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Semantic Segmentation 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by