Mini-batch loss not changing for FCN

2 次查看（过去 30 天）
filipendo125
filipendo125 2018-1-30
评论： Filip Kiniorski 2018-2-23
I have a Neural Network based on Fully Convolutional Neural Network for semantic segmentation ( FCN) and try to train it using Pascal-Context data. However, instead of 450 classes provided by the dataset, I only want to capture 12 + background.
I prepared a dataset with 7000+ images and label maps, and followed the tutorial on Semantic Segmentation ( Semantic Segmentation), and followed every step similarly to the one presented.
However, when I train my neural network my Mini-batch loss remains constant (checked until end of first 3 epochs, then tried to reduce the amount of images, but still the same value) and doesn't change, even if I change learning rate or other parameters. Is there something wrong with the training process?
So I tried to reduce the number of images to 400, and what I am getting is in picture below. It doesn't improve at all.
Below is my code:
% MAIN FUNCTION %
outputFolder= fullfile('imade');
imgDir = fullfile('imade','images');
imds = imageDatastore(imgDir);
I = readimage(imds, 1);
% I = histeq(I);
% figure
% imshow(I)
labelDir = fullfile('imade','labels');
classes = [
"bottle"
"chair"
"diningtable"
"person"
"pottedplant"
"sofa"
"tvmonitor"
"ground"
"wall"
"floor"
"keyboard"
"ceiling"
"background"
];
valueSet = {
[230 25 75]
[60 180 75]
[255 225 25]
[0 130 200]
[245 130 48]
[145 30 180]
[128 128 0]
[210 245 60]
[250 190 190]
[0 128 128]
[170 110 40]
[128 0 0]
[0 0 0]
};
pxds = pixelLabelDatastore(labelDir,classes,valueSet);
% show sample image with overlay
C = readimage(pxds, 1);
cmap = camvidColorMap;
B = labeloverlay(I,C,'ColorMap',cmap);
figure
imshow(B)
pixelLabelColorbar(cmap,classes)
% calculate frequency of class pixels
tbl = countEachLabel(pxds);
% resize images and labels to desired format
imageFolder = fullfile(outputFolder,'imagesReszed',filesep);
imds = resizeCamVidImages(imds,imageFolder);
labelFolder = fullfile(outputFolder,'labelsResized',filesep);
pxds = resizeCamVidPixelLabels(pxds,labelFolder);
% partition dataset into test and train
[imdsTrain, imdsTest, pxdsTrain, pxdsTest] = partitionCamVidData(imds,pxds);
% create FCN net
imageSize = [225 300];
numClasses = numel(classes);
lgraph = fcnLayers(imageSize,numClasses,'type','16s');
st = fullfile('imade','checkPoint');
% adjust based on occurrence of each class
imageFreq = tbl.PixelCount ./ tbl.ImagePixelCount;
classWeights = median(imageFreq) ./ imageFreq;
pxLayer = pixelClassificationLayer('Name','labels','ClassNames', tbl.Name, 'ClassWeights', classWeights);
lgraph = removeLayers(lgraph, 'pixelLabels');
lgraph = addLayers(lgraph, pxLayer);
lgraph = connectLayers(lgraph, 'softmax' ,'labels');
% set training parameters
options = trainingOptions('sgdm', ...
'Momentum', 0.9, ...
'InitialLearnRate', 1e-3, ...
'L2Regularization', 0.0005, ...
'MaxEpochs', 100, ...
'MiniBatchSize', 1, ...
'Shuffle', 'every-epoch', ...
'VerboseFrequency', 2);
datasource = pixelLabelImageSource(imdsTrain,pxdsTrain);
doTraining = true;
if doTraining
[net, info] = trainNetwork(datasource,lgraph,options);
else
data = load (strcat(st,filesep,'convnet_checkpoint__4607__2018_01_27__21_25_03.mat'));
war = data.net;
[sp, info] = trainNetwork(datasource,war.Layers,options);
end
% Define the colormap used by CamVid dataset
function cmap = camvidColorMap()
cmap = [
230 25 75
60 180 75
255 225 25
0 130 200
245 130 48
145 30 180
128 128 0
210 245 60
250 190 190
0 128 128
170 110 40
128 0 0
0 0 0
];
% Normalize between [0 1].
cmap = cmap ./ 255;
end
% Add a colorbar to the current axis. The colorbar is formatted
% to display the class names with the color.
function pixelLabelColorbar(cmap, classNames)
colormap(gca,cmap)
% Add colorbar to current figure.
c = colorbar('peer', gca);
% Use class names for tick marks.
c.TickLabels = classNames;
numClasses = size(cmap,1);
% Center tick labels.
c.Ticks = 1/(numClasses*2):1/numClasses:1;
% Remove tick mark.
c.TickLength = 0;
end
% Partition CamVid data by randomly selecting 60% of the data for training. The
% rest is used for testing.
function [imdsTrain, imdsTest, pxdsTrain, pxdsTest] = partitionCamVidData(imds,pxds)
% Set initial random state for example reproducibility.
rng(0);
numFiles = numel(imds.Files);
shuffledIndices = randperm(numFiles);
% Use 60% of the images for training.
N = round(0.60 * numFiles);
trainingIdx = shuffledIndices(1:N);
% Use the rest for testing.
testIdx = shuffledIndices(N+1:end);
% Create image datastores for training and test.
trainingImages = imds.Files(trainingIdx);
testImages = imds.Files(testIdx);
imdsTrain = imageDatastore(trainingImages);
imdsTest = imageDatastore(testImages);
% Extract class and label IDs info.
classes = pxds.ClassNames;
labelIDs = 1:numel(pxds.ClassNames);
% Create pixel label datastores for training and test.
trainingLabels = pxds.Files(trainingIdx);
testLabels = pxds.Files(testIdx);
pxdsTrain = pixelLabelDatastore(trainingLabels, classes, labelIDs);
pxdsTest = pixelLabelDatastore(testLabels, classes, labelIDs);
end
% Resize images to desired image size.
function imds = resizeCamVidImages(imds, imageFolder)
if ~exist(imageFolder,'dir')
mkdir(imageFolder)
else
imds = imageDatastore(imageFolder);
return; % Skip if images already resized
end
reset(imds)
while hasdata(imds)
% Read an image.
[I,info] = read(imds);
% Resize image.
I = imresize(I,[225 300]);
% Write to disk.
[~, filename, ext] = fileparts(info.Filename);
imwrite(I,[imageFolder filename ext])
end
imds = imageDatastore(imageFolder);
end
% Resize pixel label data to desired image size.
function pxds = resizeCamVidPixelLabels(pxds, labelFolder)
classes = pxds.ClassNames;
labelIDs = 1:numel(classes);
if ~exist(labelFolder,'dir')
mkdir(labelFolder)
else
pxds = pixelLabelDatastore(labelFolder,classes,labelIDs);
return; % Skip if images already resized
end
reset(pxds)
while hasdata(pxds)
% Read the pixel data.
[C,info] = read(pxds);
% Convert from categorical to uint8.
L = uint8(C);
% Resize the data. Use 'nearest' interpolation to
% preserve label IDs.
L = imresize(L,[225 300],'nearest');
% Write the data to disk.
[~, filename, ext] = fileparts(info.Filename);
imwrite(L,[labelFolder filename ext])
end
labelIDs = 1:numel(classes);
pxds = pixelLabelDatastore(labelFolder,classes,labelIDs);
end

请先登录，再回答此问题。

回答（1 个）

Birju Patel
Birju Patel 2018-2-23
编辑：Birju Patel 2018-2-23
This is a R2017b bug in fcnLayers.
I've attached a patch for this issue. Here are the instructions for patching R2017b:
1. Download the attached patch, fcnpatch17b_2_23_2018.zip, and save it to your root MATLAB directory. The following command returns the root MATLAB directory.
% Root MATLAB directory
matlabroot
2. Execute the following MATLAB commands from the root MATLAB directory.
unzip fcnpatch17b_2_23_2018.zip
3. Restart matlab and execute the following command:
rehash toolboxcache
NOTE: This bug has been fixed in the upcoming release, R2018a.
  1 个评论
Filip Kiniorski
Filip Kiniorski 2018-2-23
That is great testing it right now.
I also have a question regarding implementation. Original papers states that channels in the image should be shifted from RGB to BGR, is that performed in FCN layers, or does it need to be done separately?

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Semantic Segmentation 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by