Create a Custom Feature Extractor

You can use the bag-of-features (BoF) framework with many different types of image features. To use a custom feature extractor instead of the default speeded-up robust features (SURF) feature extractor, use the CustomExtractor property of a bagOfFeatures object.

Example of a Custom Feature Extractor

This example shows how to write a custom feature extractor function for bagOfFeatures. You can open this example function file and use it as a template by typing the following command at the MATLAB^® command prompt:

edit('exampleBagOfFeaturesExtractor.m')

Step 1. Define the image sets.
Step 2. Create a new extractor function file.
Step 3. Preprocess the image.
Step 4. Select a point location for feature extraction.
Step 5. Extract features.
Step 6. Compute the feature metric.

Define the set of images and labels

Read the category images and create image sets.

setDir  = fullfile(toolboxdir('vision'),'visiondata','imageSets');
imds = imageDatastore(setDir,'IncludeSubfolders',true,'LabelSource',...
    'foldernames');

Create a new extractor function file

The extractor function must be specified as a function handle:

extractorFcn = @exampleBagOfFeaturesExtractor;
bag = bagOfFeatures(imgSets,'CustomExtractor',extractorFcn)

exampleBagOfFeaturesExtractor is a MATLAB function. For example:

function [features,featureMetrics] = exampleBagOfFeaturesExtractor(img)
...

You can also specify the optional location output:

function [features,featureMetrics,location] = exampleBagOfFeaturesExtractor(img)
...

The function must be on the path or in the current working folder.

Argument	Input/Output	Description
`img`	Input	Binary, grayscale, or truecolor image. The input image is from the image set that was originally passed into `bagOfFeatures`.
`features`	Output	A `binaryFeatures` object. An M-by-N numeric matrix of image features, where M is the number of features and N is the length of each feature vector. The feature length, N, must be greater than zero and be the same for all images processed during the `bagOfFeatures` creation process. If you cannot extract features from an image, supply an empty feature matrix and an empty feature metrics vector. Use the empty matrix and vector if, for example, you did not find any keypoints for feature extraction. Numeric, real, and nonsparse.
`featureMetrics`	Output	An M-by-1 vector of feature metrics indicating the strength of each feature vector. Used to apply the `'SelectStrongest'` criteria in `bagOfFeatures` framework. Numeric, real, and nonsparse.
`location`	Output	An M-by-2 matrix of 1-based [x y] values. The [x y] values can be fractional. Numeric, real, and nonsparse.

Preprocess the image

Input images can require preprocessing before feature extraction. To extract SURF features and to use the detectSURFFeatures or detectMSERFeatures functions, the images must be grayscale. If the images are not grayscale, you can convert them using the im2gray function.

grayImage = im2gray(I);

Select a point location for feature extraction

Use a regular spaced grid of point locations. Using the grid over the image allows for dense SURF feature extraction. The grid step is in pixels.

gridStep = 8;
gridX = 1:gridStep:width;
gridY = 1:gridStep:height;

[x,y] = meshgrid(gridX,gridY);

gridLocations = [x(:) y(:)];

You can manually concatenate multiple SURFPoints objects at different scales to achieve multiscale feature extraction.

multiscaleGridPoints = [SURFPoints(gridLocations,'Scale',1.6);
    SURFPoints(gridLocations,'Scale',3.2);
    SURFPoints(gridLocations,'Scale',4.8);
    SURFPoints(gridLocations,'Scale',6.4)];

Alternatively, you can use a feature detector, such as detectSURFFeatures or detectMSERFeatures, to select point locations.

multiscaleSURFPoints = detectSURFFeatures(I);

Extract features

Extract features from the selected point locations. By default, bagOfFeatures extracts upright SURF features.

features = extractFeatures(grayImage,multiscaleGridPoints,'Upright',true);

Compute the feature metric

The feature metrics indicate the strength of each feature. Larger metric values are assigned to stronger features. Use feature metrics to identify and remove weak features before using bagOfFeatures to learn the visual vocabulary of an image set. Use the metric that is suitable for your feature vectors.

For example, you can use the variance of the SURF features as the feature metric.

featureMetrics = var(features,[],2);

If you used a feature detector for the point selection, then use the detection metric instead.

featureMetrics = multiscaleSURFPoints.Metric;

You can optionally return the feature location information. The feature location can be used for spatial or geometric verification image search applications. See the Geometric Verification Using estimateGeometricTransform2D Function example. The retrieveImages and indexImages functions are used for content-based image retrieval systems.

if nargout > 2
    varargout{1} = multiscaleGridPoints.Location;
end