audioFeatureExtractor
Streamline audio feature extraction
Description
audioFeatureExtractor encapsulates multiple audio feature
extractors into a streamlined and modular implementation.
Creation
Description
aFE = audioFeatureExtractor()
aFE = audioFeatureExtractor(Name=Value)aFE using one or more name-value
arguments.
Properties
Object Functions
extract | Extract audio features |
setExtractorParameters | Set nondefault parameter values for individual feature extractors |
info | Output mapping and individual feature extractor parameters |
generateMATLABFunction | Create MATLAB function compatible with C/C++ code generation |
plotFeatures | Plot extracted audio features |
Examples
Extract Multiple Audio Features
Read in an audio signal.
[audioIn,fs] = audioread("Counting-16-44p1-mono-15secs.wav");Create an audioFeatureExtractor object that extracts the MFCC, delta MFCC, delta-delta MFCC, pitch, spectral centroid, zero-crossing rate, and short-time energy of the signal. Use a 30 ms analysis window with 20 ms overlap.
aFE = audioFeatureExtractor( ... SampleRate=fs, ... Window=hamming(round(0.03*fs),"periodic"), ... OverlapLength=round(0.02*fs), ... mfcc=true, ... mfccDelta=true, ... mfccDeltaDelta=true, ... pitch=true, ... spectralCentroid=true, ... zerocrossrate=true, ... shortTimeEnergy=true);
Call extract to extract the audio features from the audio signal.
features = extract(aFE,audioIn);
Use info to determine which column of the feature extraction matrix corresponds to the requested pitch extraction.
idx = info(aFE)
idx = struct with fields:
mfcc: [1 2 3 4 5 6 7 8 9 10 11 12 13]
mfccDelta: [14 15 16 17 18 19 20 21 22 23 24 25 26]
mfccDeltaDelta: [27 28 29 30 31 32 33 34 35 36 37 38 39]
spectralCentroid: 40
pitch: 41
zerocrossrate: 42
shortTimeEnergy: 43
Plot the detected pitch over time.
t = linspace(0,size(audioIn,1)/fs,size(features,1)); plot(t,features(:,idx.pitch)) title("Pitch") xlabel("Time (s)") ylabel("Frequency (Hz)")
Plot the zero-crossing rate over time.
plot(t,features(:,idx.zerocrossrate)) title("Zero-Crossing Rate") xlabel("Time (s)")
Plot the short-time energy over time.
plot(t,features(:,idx.shortTimeEnergy)) title("Short-Time Energy") xlabel("Time (s)")
Extract Features from Data Set
Create an audio datastore that points to audio samples included with Audio Toolbox®.
folder = fullfile(matlabroot,"toolbox","audio","samples"); ads = audioDatastore(folder);
Create an audioFeatureExtractor object to extract the mel spectrum, Bark spectrum, ERB spectrum, and linear spectrum from each audio file. Use the default analysis window and overlap length for the spectrum extraction.
aFE = audioFeatureExtractor(SampleRate=44.1e3, ... melSpectrum=true, ... barkSpectrum=true, ... erbSpectrum=true, ... linearSpectrum=true);
Call extract to extract the features from each audio file in the datastore. Specify SampleRateMismatchRule as "resample" to resample the audio files in the datastore if they do not match 44.1 kHz, the sample rate of the audioFeatureExtractor object. If you have Parallel Computing Toolbox™, specify UseParallel as true to read the files and extract the features in parallel.
specs = extract(aFE,ads,SampleRateMismatchRule="resample",UseParallel=true);The specs variable is a numFiles-by-1 cell array, where numFiles is the number of files in the datastore. Each element of the cell array is a numHops-by-numFeatures-by-numChannels array, where the number of hops and number of channels depends on the length and number of channels of the audio file, and the number of features is the requested number of features from the audio data.
numFiles = numel(specs)
numFiles = 37
[numHops1,numFeaturesFile1,numChanelsFile1] = size(specs{1})numHops1 = 1053
numFeaturesFile1 = 620
numChanelsFile1 = 1
[numHops2,numFeaturesFile2,numChanelsFile2] = size(specs{2})numHops2 = 1724
numFeaturesFile2 = 620
numChanelsFile2 = 4
Visualize Extracted Audio Features
Use plotFeatures to visualize audio features extracted with an audioFeatureExtractor object.
Read in an audio signal from a file.
[audioIn,fs] = audioread("Counting-16-44p1-mono-15secs.wav");Create an audioFeatureExtractor object that extracts the gammatone cepstral coefficients (GTCCs) and the delta of the GTCCs. Set the SampleRate property to the sample rate of the audio signal, and use the default values for the other properties.
afe = audioFeatureExtractor(SampleRate=fs,gtcc=true,gtccDelta=true);
Plot the features extracted from the audio signal.
plotFeatures(afe,audioIn)
Algorithms
The audioFeatureExtractor creates a feature extraction pipeline based on
your selected features. To reduce computations, audioFeatureExtractor reuses
intermediary representations and outputs some intermediate representations as features.
For example, to create an object that extracts the centroid of the Bark spectrum, the flux
of the Bark spectrum, the pitch, the harmonic ratio, and the delta-delta of the MFCC, specify
the audioFeatureExtractor as
follows.
aFE = audioFeatureExtractor( ... SpectralDescriptorInput="barkSpectrum", ... spectralCentroid=true, ... spectralFlux=true, ... pitch=true, ... harmonicRatio=true, ... mfccDeltaDelta=true)
aFE =
audioFeatureExtractor with properties:
Properties
Window: [1024×1 double]
OverlapLength: 512
SampleRate: 44100
FFTLength: []
SpectralDescriptorInput: 'barkSpectrum'
Enabled Features
mfccDeltaDelta, spectralCentroid, spectralFlux, pitch, harmonicRatio
Disabled Features
linearSpectrum, melSpectrum, barkSpectrum, erbSpectrum, mfcc, mfccDelta
gtcc, gtccDelta, gtccDeltaDelta, spectralCrest, spectralDecrease, spectralEntropy
spectralFlatness, spectralKurtosis, spectralRolloffPoint, spectralSkewness, spectralSlope, spectralSpread
To extract a feature, set the corresponding property to true.
For example, obj.mfcc = true, adds mfcc to the list of enabled features.
Note
Because audioFeatureExtractor reuses intermediary representations, the
features output from audioFeatureExtractor might not correspond with the
default configuration of features output by corresponding individual feature
extractors.
Extended Capabilities
Version History
Introduced in R2019b
You can also select a web site from the following list
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)
Asia Pacific
- Australia (English)
- India (English)
- New Zealand (English)
- 中国
- 日本Japanese (日本語)
- 한국Korean (한국어)