Feature Extraction and Deep Learning

Audio labeling, datastore, voice activity detection, MFCC, pitch, loudness

Audio Toolbox™ enables you to extract auditory features common to machine-learning and deep-learning tasks. Use Audio Labeler to interactively define and visualize ground-truth for audio datasets. Use audioDatastore to handle large collections of audio recordings for batch processing or machine and deep learning applications.

Apps

Audio Labeler

Define and visualize ground-truth labels

Functions

expand all

Audio I/O

audioDatastore Datastore for collection of audio files

Cepstral Analysis

`mfcc`	Extract mfcc, log energy, delta, and delta-delta of audio signal
`gtcc`	Extract gammatone cepstral coefficients, log-energy, delta, and delta-delta
`cepstralFeatureExtractor`	Extract cepstral features from audio segment

Data Augmentation

`audioDataAugmenter`	Augment audio data
`audioTimeScaler`	Apply time scaling to streaming audio
`shiftPitch`	Shift audio pitch
`stretchAudio`	Time-stretch audio

Domain Conversion

`erb2hz`	Convert from equivalent rectangular bandwidth (ERB) scale to hertz
`bark2hz`	Convert from Bark scale to hertz
`mel2hz`	Convert from mel scale to hertz
`hz2erb`	Convert from hertz to equivalent rectangular bandwidth (ERB) scale
`hz2bark`	Convert from hertz to Bark scale
`hz2mel`	Convert from hertz to mel scale

Filter Design

designAuditoryFilterBank Design auditory filter bank

Loudness

`integratedLoudness`	Measure integrated loudness and loudness range
`loudnessMeter`	Standard-compliant loudness measurements

Periodicity and Harmonicity

`harmonicRatio`	Harmonic ratio
`pitch`	Estimate fundamental frequency of audio signal

Segmentation

voiceActivityDetector Detect presence of speech in audio signal

Spectral Descriptors

`audioFeatureExtractor`	Streamline audio feature extraction
`spectralCentroid`	Spectral centroid for audio signals and auditory spectrograms
`spectralCrest`	Spectral crest for audio signals and auditory spectrograms
`spectralDecrease`	Spectral decrease for audio signals and auditory spectrograms
`spectralEntropy`	Spectral entropy for audio signals and auditory spectrograms
`spectralFlatness`	Spectral flatness for audio signals and auditory spectrograms
`spectralFlux`	Spectral flux for audio signals and auditory spectrograms
`spectralKurtosis`	Spectral kurtosis for audio signals and auditory spectrograms
`spectralRolloffPoint`	Spectral rolloff point for audio signals and auditory spectrograms
`spectralSkewness`	Spectral skewness for audio signals and auditory spectrograms
`spectralSlope`	Spectral slope for audio signals and auditory spectrograms
`spectralSpread`	Spectral spread for audio signals and auditory spectrograms

Transforms and Windows

`melSpectrogram`	Mel spectrogram
`kbdwin`	Kaiser-Bessel-derived window
`mdct`	Modified discrete cosine transform
`imdct`	Inverse modified discrete cosine transform

Blocks

Voice Activity Detector	Detect presence of speech in audio signal
Cepstral Feature Extractor	Extract cepstral features from audio segment
Loudness Meter	Standard-compliant loudness measurements

Topics

Label Audio Using Audio Labeler

Interactively define and visualize ground-truth labels for audio datasets.

Speech-to-Text Transcription

Perform speech-to-text transcription in MATLAB^® using third-party cloud-based APIs.

Spectral Descriptors

Overview and applications of spectral descriptors.

Featured Examples

Speech Command Recognition Using Deep Learning

Train a simple deep learning model that detects the presence of speech commands in audio. The example uses the Speech Commands Dataset [1] to train a convolutional neural network to recognize a given set of commands.

Open Script

Speaker Identification Using Pitch and MFCC

Demonstrates a machine learning approach to identify people based on features extracted from recorded speech. The features used to train the classifier are: pitch of the voiced segments of the speech, and the Mel-Frequency Cepstrum Coefficients (MFCC). This is a closed-set speaker identification - the audio of the speaker under test is compared against all the available speaker models (a finite set) and the closest match is returned.

Open Script

Cocktail Party Source Separation Using Deep Learning Networks

Isolate a speech signal using a deep learning network.

Open Live Script

Keyword Spotting in Noise Using MFCC and LSTM Networks

Identify a keyword in noisy speech using a deep learning network. In particular, the example uses a Bidirectional Long Short-Term Memory (BiLSTM) network and mel-frequency cepstral coefficients (MFCC).

Open Script

Denoise Speech Using Deep Learning Networks

Denoise speech signals using deep learning networks. The example compares two types of networks applied to the same task: fully connected, and convolutional.

Open Live Script

Voice Activity Detection in Noise Using Deep Learning

Detect regions of speech in a low signal-to-noise environment using deep learning. The example uses the Speech Commands Dataset to train a Bidirectional Long Short-Term Memory (BiLSTM) network to detect voice activity.

Open Script

Classify Gender Using LSTM Networks

Classify the gender of a speaker using deep learning. In particular, the example uses a Bidirectional Long Short-Term Memory (BiLSTM) network and Gammatone Cepstral Coefficients (gtcc), pitch, harmonic ratio, and several spectral shape descriptors.

Open Script

Spoken Digit Recognition with Wavelet Scattering and Deep Learning

Classify spoken digits using wavelet time scattering paired with a support vector machine and a deep convolutional network based on mel-frequency spectrograms.

Open Live Script

Music Genre Classification Using Wavelet Time Scattering

Classify the genre of a musical excerpt using wavelet time scattering and the audio datastore. In wavelet scattering, data is propagated through a series of wavelet transforms, nonlinearities, and averaging to produce low-variance representations of the data. These low-variance representations are then used as inputs to a classifier.

Open Script

Sequential Feature Selection for Speech Emotion Recognition

A typical workflow for feature selection applied to the task of speech emotion recognition. You begin by creating a baseline accuracy using common audio features (MFCC). You then augment your data set to diminish overfitting. Finally, you perform sequential feature selection to select a better feature set.

Open Script

Acoustic Scene Recognition Using Late Fusion

Create a multi-model late fusion system for acoustic scene recognition. The example trains a convolutional neural network (CNN) using mel spectrograms and an ensemble classifier using wavelet scattering. The example uses the TUT dataset for training and evaluation [1].

Open Script

Pitch Tracking Using Multiple Pitch Estimations and HMM

Perform pitch tracking using multiple pitch estimations, octave and median smoothing, and a Hidden Markov Model (HMM).

Open Live Script

Time-Frequency Masking for Harmonic-Percussive Source Separation

Time-frequency masking is the process of applying weights to the bins of a time-frequency representation to enhance, diminish, or isolate portions of audio.

Open Live Script