MATLAB Answers

stopsigns RCNN Bounding BOX understanding

8 views (last 30 days)
Matpar
Matpar on 3 Feb 2020
Commented: Matpar on 23 Mar 2020 at 13:40
Hi Professionals,
I am trying to really understand what is happening with this piece of code, I would like to understand where or if the feature extraction is passed into the RCNNObjectDetector to increase the accuracy of the classification and confidence of the bounding box!
My question may seem silly but this is confusing me for some time now I am having a challenge trying to move forward!
This the simple code that is in operation for the stopsign example which work fine!
clc
clear
close all
clearvars
load('rcnnStopSigns.mat', 'stopSigns', 'layers')
imDir = fullfile(matlabroot, 'toolbox', 'vision', 'visiondata','stopSignImages');
addpath(imDir);
options = trainingOptions('sgdm','MiniBatchSize', 32,'InitialLearnRate', 1e-6,'MaxEpochs', 10);
rcnn = trainRCNNObjectDetector(stopSigns, layers, options, 'NegativeOverlapRange', [0 0.3]);
img = imread('stopSignTest.jpg');
[bbox, score, label] = detect(rcnn, img, 'MiniBatchSize', 32)
[score, idx] = max(score)
bbox = bbox(idx, :)
annotation = sprintf('%s: (Confidence = %f)', label(idx), score)
detectedImg = insertObjectAnnotation(img, 'rectangle', bbox, annotation);
figure
imshow(detectedImg)
rmpath(imDir);
I am doing the same exercise using transfer learning using alexnet for a small gun dataset and the bounding box will not appear properly around the region of interest!
I tweaked and fine tuned the epochs and mini batch values and I even lowered the weights and still this is causing a problem somewhere that my knowledge is to limited to uncover!
Off course there are things in that simple code ABOVE that are not displayed within the code and does not demosntrate how they should be done!
I followed the example step by step watching the workspace and then running my code to see what happens which is the same thing!
I would like it very much please if a professional can assist me to understand what I am doing incorrectly and how I should move forward!
This process is critical for me to understand, pretty pretty pretty please can someone assist me!
I have attached the data that is used in my initial example!
Can someone tell me what I am doing incorrectly as this is identical to that of the RCNN Stop Sign Example on MathLab
clc
clear
close all
clearvars
%% Step 1 Deploying deepNetworkDesigner to edit Layers physically
imDir = fullfile(matlabroot, 'toolbox', 'vision', 'visiondata','gunsGT');
addpath(imDir);
load('LGTruths')
%% Step 2 Specifying Image Amount In Specified Folder
%The number of 1 first dimension i.e rows 2 second dimension ie column
%3 number of element is the 3rd dimension for 3d image
total_images = size(LGTruths,1);
%% Step 3 Accessing Content of Folder TrainingSet Using Datastore
imds =imageDatastore(imDir,'IncludeSubFolders',true,'LabelSource','Foldernames');
%% Step 4 Setting Output Function(images may have size variation resizing for consistency with pretrain net)
imds.ReadFcn=@(loc)imresize(imread(loc),[227,227]);
%% Step 5 Counting Images In Each Category "If not equal this will create issues"
tbl=countEachLabel(imds);
%% Step 6 Making Category The Same Number Of Images If Number Is Uneven May Cause Issues
minSetCount=min(tbl{:,2})
%% Step 7 Splitting Inputs Into Training and Testing Sets
% Measuring Elapse Time With TIC & TOC Function Working Together
tic
[imdsTrain,imdsValidation] = splitEachLabel(imds,0.7,'randomized'); size(imdsTrain)
toc
%% Step 8 Loading Pretrained Network
%Trained on 1million+ images/classify images into 1000 object categories
net = alexnet
% Display Alexnet architecture & network layer details
% analyzeNetwork(net) % Display Alexnet architecture & network layerdetails
%% Step 9 Altering InputSize Of 1st Layer/
% Alexnet Image requirements is 277 width 277 height by 3 colour channels
%Displays the input size of Alexnet
inputSize =net.Layers(1).InputSize
%% Step 10 Counting Total Number Of Images Including Subfolders **IF AVAILABLE**
imgTotal = length(imds.Files);
%% Step 11 Displaying Multiple Randomized Images Within The Dataset
% a = 4;
% b = 4;
% n = randperm(imgTotal, a*b);
% figure(),
% Idx = 1;
% for j=1:a
% for k=1:b
% img=readimage(imds,n(Idx));
% subplot(a,b,Idx)
% imshow(img);
% Idx=Idx+1;
% end
% end
%% Step 12 Replacing Final Layer/Last 3 Configure For 1000 classes
% Finetuning these 3 layers for new classification
% Extracting all Layers except the last 3
layersTransfer = net.Layers(1:end-3)
%% Step 13 Specifying Image Categories/Clases:
% Lowering the weights encourages the network to learn faster
numClasses = numel(categories(imdsTrain.Labels))
Tlayers = [
layersTransfer
fullyConnectedLayer(numClasses,'Name','fc8','WeightLearnRateFactor',5,'BiasLearnRateFactor',5)
softmaxLayer('Name','SoftmaxL')
classificationLayer('Name','ClassifL')]
%% Step 14 Training The Network
% Resizing images in datastore to meet Alexnet's size requirements
% Utilising Augmented Data Store for automatic resizing of training images
% Augmented Data Store Prevents Over Fitting By Randomly Flipping Along The Vertical Axis
% Stopping the network from memorizing exact details of the training data
% Also Randomly Translates them up to 30 pixels horizontally & Vertically
pixelRange = [-30 30];
imageAugmenter = imageDataAugmenter( ...
'RandXReflection',true, ...
'RandXTranslation',pixelRange, ...
'RandYTranslation',pixelRange);
%% Step 15 Resizing Images, Assists With Preventing Overfitting
% Utilising Data Augmentation For Resizing Validation Data
% implemented without specifying overfit prevention procedures
% By not specifying these procedures the system will be precise via
% predicitons
augmentedTrainingSet = augmentedImageDatastore(inputSize ,imdsTrain,'ColorPreprocessing', 'gray2rgb')
augimdsValidation = augmentedImageDatastore(inputSize,imdsValidation,'ColorPreprocessing', 'gray2rgb')
%% Step 15 Specifying Training Options
% Keep features from earlier layers of pretrained networked for transfer learning
% Specify epoch training cycle, the mini-batch size and validation data
% Validate the network for each iteration during training.
% (SGDM)groups the full dataset into disjoint mini-batches This reaches convergence faster
% as it updates the network's weight value more frequently and increases the
% computationl speed
% Implementing **WITH** The RCNN Object Detector
opts = trainingOptions('sgdm',...
'Momentum',0.9,...
'InitialLearnRate', 1e-4,...
'LearnRateSchedule', 'piecewise', ...
'LearnRateDropFactor', 0.1, ...
'Shuffle','every-epoch', ...
'LearnRateDropPeriod', 8, ...
'L2Regularization', 1e-4, ...
'MaxEpochs', 10,...
'MiniBatchSize',8,...
'Verbose', true)
%% Step 16 Training network Consisting Of Transferred & New Layers.
netTransfer = trainNetwork(augmentedTrainingSet,Tlayers,opts);
%% Step 17 Classifying Validation Images Utilising Fine-tuned Network
[YPred,scores] = classify(netTransfer,augimdsValidation);
%% Step 18 Displaying 4 Validation Image Samples With Predicted Labels
rng();
idx = randperm(numel(imdsValidation.Files),4);
figure
for i = 1:4
subplot(2,2,i)
I = readimage(imdsValidation,idx(i));
imshow(I)
label = YPred(idx(i));
title(string(label));
end
%% Step 19 Calculating Validation Data Classification Accuracy (Accuracy Labels Predicted Accurately By Network)
YValidation = imdsValidation.Labels;
accuracy = mean(YPred == YValidation);
% Step 20 Training The R-CNN detector.
% Training can take a few minutes to complete.
% Step 20 Positive and Negative Overlap Range Controls Which Image Patch is Used
rcnn = trainRCNNObjectDetector(LGTruths,netTransfer, opts, 'NegativeOverlapRange', [0 0.3])
save('rcnn.mat', 'rcnn')
rng()
cellGT = table2cell(LGTruths);
%% Step 21 Displaying A Random Selection Of An Image With Bounding Boxes
i = randi(size(cellGT,1),1);
testimg = imread(cellGT{i,1});
gtGunLocation = cellGT{i,2};
[bboxes,score,label] = detect(rcnn,testimg,'MiniBatchSize',8);
% figure
% for i=1:size(bboxes,1)
%
% bbox = bboxes(i, :);
% annotation = sprintf('%s: (Confidence = %f)', label(i), score(i));
% Imgdetected = insertObjectAnnotation(testimg, 'rectangle', bbox,annotation);
% subplot(2,2,i)
% imshow(Imgdetected);
% drawnow;
% end
%
%% Step 22 Detecting The Gun With Bounding Boxes
[max_score, max_loc] = max(score);
max_bbox = bboxes(max_loc,:);
max_annotation = sprintf('%s: (Confidence = %f)', label(max_loc), max_score);
Imgdetected = insertObjectAnnotation(testimg, 'rectangle', max_bbox,max_annotation);
figure,
subplot(1,2,1);
imshow(Imgdetected);
title('Rcnn output');
subplot(1,2,2);
imshow(testimg);
% Specfying the details on the same image
hold on
rectangle('position', gtGunLocation);
hold off
title('My ground truth of gun location');
drawnow;

  2 Comments

Hayat  Bouchkouk
Hayat Bouchkouk on 23 Mar 2020 at 11:28
hi Matpar
are you find a solution for your project because i face a same problem,and i want to know how can i create .mat file like this one
plizz help me i'me really stuck
Matpar
Matpar on 23 Mar 2020 at 13:40
I understand the frustration,
  1. Get a dataset that you would like to work with
  2. open the labeller app in matlab I am working with 2019b
  3. https://www.youtube.com/watch?v=xMsX5cdqbKU
  4. export as a table to workspace you will see the export feature in the labeller app this is where you specify the name LGTruths
  5. when this is done go to the workspace and you will see the table created
  6. You can rename this file in the workspace as well
  7. Then call the file as in the code above

Sign in to comment.

Answers (0)

Sign in to answer this question.


Translated by