Automated image labelling with Attributes

6 次查看（过去 30 天）
Mathieu
Mathieu 2021-10-1
Dear all,
I'm trying to automate image labelling. I went throught several available examples (Examples), and I think I'm nearly done, but I can't automate the attributes.
My problem is the following, I want to detect goats in images and then, classify the goats depending on their color (white, red, brown and black).
I created one rectangle label, with one attribute, the color, taking value into {'white','red','brown','black'}
I manually labelled more than 3000 images and know I have trained CNNs able to do, approximately, the job for me. I want to label more images to possibly improve my CNN accuracy, but it's a shame I have to label new images manually, when I have a method that is correct at 90%
Using the example liste above, for automatic car and distance labelling, I was able to design a code that automatically detect the goats. To be more precise, if I comment everything that is related to the attributes, my code run and I have bounding boxes around the detected animals.
But when the codes about the attributes is un-comment, I have error message: "Expected automated labels to be a valid label from among the existing label definitions".
I can understand the error message, but I can't find why the definition of my label is not correct in my code ...
I hope somebody will have any idea. Best would be that MAtlab provides more examples on automated labelling :)
Below is my code:
% This class defines a template for creating a custom label automation
% algorithm, to be used in labeling apps (Image Labeler, Video Labeler and
% Ground Truth Labeler). To access help for this class, enter the following
% at the MATLAB command prompt:
%
% >> doc vision.labeler.AutomationAlgorithm
%
% For reference, see the following AutomationAlgorithm class:
%
% >> edit vision.labeler.PeopleDetectorACF
%
%
% To use this algorithm from within the Image Labeler, Video Labeler or
% Ground Truth Labeler App, follow the steps outlined below and complete
% the class definition. Then, save this file as follows.
%
% Create a +vision/+labeler folder within a folder that is already
% on the MATLAB path. For example, if the folder /local/MyProject is on
% the MATLAB path, then create a +vision/+labeler folder hierarchy as
% follows:
%
% projectFolder = fullfile('local','MyProject');
% automationFolder = fullfile('+vision','+labeler');
%
% mkdir(projectFolder, automationFolder)
%
% Saving the file to the package directory is required to use your custom
% algorithm from within the labeling apps. You can add a folder to the
% path using the ADDPATH function.
%
% Save your algorithm class in the folder created in step 1. Refresh the
% algorithm list from within the app to start using your custom algorithm.
classdef CabrisCouleurs < vision.labeler.AutomationAlgorithm
%----------------------------------------------------------------------
% Step 1: Define required properties describing the algorithm. This
% includes Name, Description and UserDirections.
properties(Constant)
% Name: Give a name for your algorithm.
Name = 'Detection des cabris et de leurs couleurs';
% Description: Provide a one-line description for your algorithm.
Description = 'Use it to detect cabris with Brinno time lapse cameras';
% UserDirections: Provide a set of directions that are displayed
% when this algorithm is invoked. The directions
% are to be provided as a cell array of character
% vectors, with each element of the cell array
% representing a step in the list of directions.
UserDirections = {...
['Automation algorithms are a way to automate manual labeling ' ...
'tasks. This AutomationAlgorithm is a template for creating ' ...
'user-defined automation algorithms. Below are typical steps' ...
'involved in running an automation algorithm.'], ...
['Run: Press RUN to run the automation algorithm. '], ...
['Review and Modify: Review automated labels over the interval ', ...
'using playback controls. Modify/delete/add ROIs that were not ' ...
'satisfactorily automated at this stage. If the results are ' ...
'satisfactory, click Accept to accept the automated labels.'], ...
['Change Settings and Rerun: If automated results are not ' ...
'satisfactory, you can try to re-run the algorithm with ' ...
'different settings. In order to do so, click Undo Run to undo ' ...
'current automation run, click Settings and make changes to ' ...
'Settings, and press Run again.'], ...
['Accept/Cancel: If results of automation are satisfactory, ' ...
'click Accept to accept all automated labels and return to ' ...
'manual labeling. If results of automation are not ' ...
'satisfactory, click Cancel to return to manual labeling ' ...
'without saving automated labels.']};
end
%---------------------------------------------------------------------
% Step 2: Define properties to be used during the algorithm. These are
% user-defined properties that can be defined to manage algorithm
% execution.
properties
%% Couleur du cabris
couleur
%% Detector
detector
%% Couleur classification
net
end
properties (Constant, Access = private)
% Flag to enable Distance attribute estimation automation
AutomatecouleurAttribute = true;
% Supported Distance attribute name.
% The label must have an attribute with the name specified.
SupportedcouleurAttribName = 'couleur';
end
properties (Access = private)
% Actual attribute name for distance
couleurAttributeName;
% Flag to check if attribute specified is a valid distance
% attribute
HasValidcouleurAttribute = false;
end
%----------------------------------------------------------------------
% Step 3: Define methods used for setting up the algorithm.
methods
% a) Use the checkLabelDefinition method to specify whether a label
% definition is valid for the algorithm. This method is invoked
% on each ROI and Scene label definition to determine whether it
% is valid for the specified algorithm.
%
% For more help,
% >> doc vision.labeler.AutomationAlgorithm.checkLabelDefinition
%
function isValid = checkLabelDefinition(~, labelDef)
isValid = labelDef.Type == labelType.Rectangle;
end
% b) Use the checkSetup method to specify whether the algorithm is
% ready and all required set up is complete. If your algorithm
% requires no setup from the user, remove this method.
%
% For more help,
% >> doc vision.labeler.AutomationAlgorithm.checkSetup
%
function isReady = checkSetup(algObj)
isReady = ~isempty(algObj.SelectedLabelDefinitions);
end
% c) Optionally, specify what settings the algorithm requires by
% implementing the settingsDialog method. This method is invoked
% when the user clicks the Settings button. If your algorithm
% requires no settings, remove this method.
%
% For more help,
% >> doc vision.labeler.AutomationAlgorithm.settingsDialog
%
function settingsDialog(algObj)
disp('Executing settingsDialog')
%--------------------------------------------------------------
% Place your code here
%--------------------------------------------------------------
end
end
%----------------------------------------------------------------------
% Step 4: Specify algorithm execution. This controls what happens when
% the user presses RUN. Algorithm execution proceeds by first
% executing initialize on the first frame, followed by run on
% every frame, and terminate on the last frame.
methods
% a) Specify the initialize method to initialize the state of your
% algorithm. If your algorithm requires no initialization,
% remove this method.
%
% For more help,
% >> doc vision.labeler.AutomationAlgorithm.initialize
%
function initialize(algObj, ~)
disp('Executing initialize on the first image frame')
% Couleur du cabris
algObj.couleur = algObj.SelectedLabelDefinitions.Name;
% Detector
load Detector_27_08.mat
algObj.detector = detectorc;
% Couleur classification
load ResNet_September2021.mat
algObj.net = net;
% Initialize parameters to compute color
if algObj.AutomatecouleurAttribute
initializeAttributeParams(algObj);
end
%-------------------------------------------------------------
% Place your code here
%--------------------------------------------------------------
end
function initializeAttributeParams(algObj)
% Initialize properties relevant to attribute automation.
% The label must have an attribute with name Distance and type
% Numeric Value.
hasAttribute = isfield(algObj.ValidLabelDefinitions, 'Attributes') && ...
isstruct(algObj.ValidLabelDefinitions.Attributes);
if hasAttribute
attributeNames = fieldnames(algObj.ValidLabelDefinitions.Attributes);
idx = find(contains(attributeNames, algObj.SupportedcouleurAttribName));
if ~isempty(idx)
algObj.couleurAttributeName = attributeNames{idx};
algObj.HasValidcouleurAttribute = validatecouleurType(algObj);
end
end
end
function tf = validatecouleurType(algObj)
% Validate the attribute type.
tf = isfield(algObj.ValidLabelDefinitions.Attributes, algObj.couleurAttributeName);% && ...
%isfield(algObj.ValidLabelDefinitions.Attributes.(algObj.couleurAttributeName), 'DefaultValue') && ...
%isnumeric(algObj.ValidLabelDefinitions.Attributes.(algObj.couleurAttributeName).DefaultValue);
end
% b) Specify the run method to process an image frame and execute
% the algorithm. Algorithm execution begins at the first image
% frame and is invoked on all image frames selected for
% automation. Algorithm execution can produce a set of labels
% which are to be returned in autoLabels.
%
% For more help,
% >> doc vision.labeler.AutomationAlgorithm.run
%
function autoLabels = run(algObj, I)
img = I;
disp('Executing run on image frame')
ordre = cell(4,2);
ordre{1,1} = 1:4;
ordre{1,2} = transpose(1:4);
for n = 2:4
col = [];
C = nchoosek(1:4,n);
for i = 1:size(C,1)
col = [col ; perms(C(i,:))];
end
ordre{n,2} = col;
lig = repmat(1:n,[size(col,1) , 1]);
ind = zeros(size(lig,1),n);
for i = 1:n
ind(:,i) = sub2ind([n , 4],lig(:,i),col(:,i));
end
ordre{n,1} = ind;
end
autoLabels = [];
[bbox , probad , class] = algObj.detector.detect(img);
[bbox , probad ] = selectStrongestBbox(bbox,probad,'NumStrongest',4);
% bbox = algObj.detector.detect(I);
if ~isempty(probad)
proba = [];
for j = 1:numel(probad)
[~ , p] = algObj.net.classify(imresize(imcrop(img,bbox(j,:)),[224 224]));
% proba = [proba ; probad(j).*(p')];
proba = [proba ; p];
end
n = numel(probad);
ind = ordre{n,1};
c = ordre{n,2};
s = 0;
for j = 1:n
s = s + proba(j,c(:,j))';
end
[~ , m] = max(s);
couleurnum = c(m,:);
autoLabels.Name = 'goat';
autoLabels.Type = labelType.Rectangle;
autoLabels.Position = bbox;
if (algObj.AutomatecouleurAttribute && algObj.HasValidcouleurAttribute)
attribName = algObj.couleurAttributeName;
% Attribute value is of type 'Numeric Value'
for z = 1:size(bbox,1)
autoLabels(z).(attribName) = algObj.net.Layers(end).Classes(couleurnum(z));
end
end
end
end
% c) Specify the terminate method to clean up state of the executed
% algorithm. If your method requires no clean up, remove this
% method.
%
% For more help,
% >> doc vision.labeler.AutomationAlgorithm.terminate
%
function terminate(algObj)
disp('Executing terminate')
%--------------------------------------------------------------
% Place your code here
%--------------------------------------------------------------
end
end
end

请先登录，再回答此问题。

回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Ground Truth Labeling 的更多信息

标签

产品


版本

R2021a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by