Main Content

Speech Command Recognition Code Generation with Intel MKL-DNN

This example uses:

Open Live Script

This example shows how to deploy feature extraction and a convolutional neural network (CNN) for speech command recognition on Intel® processors. To generate the feature extraction and network code, you use MATLAB® Coder™ and the Intel® Math Kernel Library for Deep Neural Networks (MKL-DNN). In this example, the generated code is a MATLAB executable (MEX) function, which is called by a MATLAB script that displays the predicted speech command along with the time domain signal and auditory spectrogram. For details about audio preprocessing and network training, see Train Deep Learning Network for Speech Command Recognition.

Prerequisites

  • The MATLAB Coder Interface for Deep Learning Libraries support package

  • Xeon processor with support for Intel Advanced Vector Extensions 2 (Intel AVX2)

  • Intel Math Kernel Library for Deep Neural Networks (MKL-DNN)

  • Environment variables for Intel MKL-DNN

For supported versions of libraries and for information about setting up environment variables, see Prerequisites for Deep Learning with MATLAB Coder (MATLAB Coder).

Streaming Demonstration in MATLAB

Use the same parameters for the feature extraction pipeline and classification as developed in Train Deep Learning Network for Speech Command Recognition.

Define the same sample rate the network was trained on (16 kHz). Define the classification rate and the number of audio samples input per frame. The feature input to the network is a Bark spectrogram that corresponds to 1 second of audio data. The Bark spectrogram is calculated for 25 ms windows with 10 ms hops.

fs = 16000; 
classificationRate = 20;
samplesPerCapture = fs/classificationRate;

segmentDuration = 1;
segmentSamples = round(segmentDuration*fs);

frameDuration = 0.025;
frameSamples = round(frameDuration*fs);

hopDuration = 0.010;
hopSamples = round(hopDuration*fs);

Create an audioFeatureExtractor object to extract 50-band Bark spectrograms without window normalization.

afe = audioFeatureExtractor( ...
    SampleRate=fs, ...
    FFTLength=512, ...
    Window=hann(frameSamples,"periodic"), ...
    OverlapLength=frameSamples - hopSamples, ...
    barkSpectrum=true);

numBands = 50;
setExtractorParameters(afe,"barkSpectrum",NumBands=numBands,WindowNormalization=false);

Load the pretrained convolutional neural network and labels.

load('SpeechCommandRecognitionNetwork.mat')
numLabels = numel(labels);
backgroundIdx = find(labels == 'background');

Define buffers and decision thresholds to post process network predictions.

probBuffer = single(zeros([numLabels,classificationRate/2]));
YBuffer = single(numLabels * ones(1, classificationRate/2)); 

countThreshold = ceil(classificationRate*0.2);
probThreshold = single(0.7);

Create an audioDeviceReader object to read audio from your device. Create a dsp.AsyncBuffer object to buffer the audio into chunks.

adr = audioDeviceReader(SampleRate=fs,SamplesPerFrame=samplesPerCapture,OutputDataType='single');
audioBuffer = dsp.AsyncBuffer(fs);

Create a dsp.MatrixViewer object and a timescope object to display the results.

matrixViewer = dsp.MatrixViewer( ...
    ColorBarLabel="Power per band (dB/Band)", ...
    XLabel="Frames", ...
    YLabel="Bark Bands", ...
    Position=[400 100 600 250], ...
    ColorLimits=[-4 2.6445], ...
    AxisOrigin="Lower left corner", ...
    Name="Speech Command Recognition Using Deep Learning");

timeScope = timescope( ...
    SampleRate=fs, ...
    YLimits=[-1 1], ...
    Position=[400 380 600 250], ...
    Name="Speech Command Recognition Using Deep Learning", ...
    TimeSpanSource="Property", ...
    TimeSpan=1, ...
    BufferLength=fs);

timeScope.YLabel = "Amplitude";
timeScope.ShowGrid = true;

Show the time scope and matrix viewer. Detect commands as long as both the time scope and matrix viewer are open or until the time limit is reached. To stop the live detection before the time limit is reached, close the time scope window or matrix viewer window.

show(timeScope)
show(matrixViewer)
timeLimit = 10;

tic
while isVisible(timeScope) && isVisible(matrixViewer) && toc < timeLimit
    %% Capture Audio
    x = adr();
    write(audioBuffer,x);
    y = read(audioBuffer,fs,fs-samplesPerCapture);
    
    % Compute auditory features
    features = extract(afe,y);
    auditory_features = log10(features + 1e-6);
    
    % Transpose to get the auditory spectrum
    auditorySpectrum = auditory_features';
    
    % Perform prediction
    score = predict(net,auditory_features);      
    [~,YPredicted] = max(score);
    
    % Perform statistical post processing
    YBuffer = [YBuffer(2:end),YPredicted];
    probBuffer = [probBuffer(:,2:end),score(:)];

    [YMode_idx, count] = mode(YBuffer);
    count = single(count);
    maxProb = max(probBuffer(YMode_idx,:));

    if (YMode_idx == single(backgroundIdx) || count < countThreshold || maxProb < probThreshold)
        speechCommandIdx = backgroundIdx;
    else
        speechCommandIdx = YMode_idx;
    end
    
    % Update plots
    matrixViewer(auditorySpectrum);
    timeScope(x);

    if (speechCommandIdx == backgroundIdx)
        timeScope.Title = ' ';
    else
        timeScope.Title = char(labels(speechCommandIdx));
    end
    drawnow
end

Hide the scopes.

hide(matrixViewer)
hide(timeScope)

Prepare MATLAB Code for Deployment

To create a function to perform feature extraction compatible with code generation, call generateMATLABFunction on the audioFeatureExtractor object. The generateMATLABFunction object function creates a standalone function that performs equivalent feature extraction and is compatible with code generation.

generateMATLABFunction(afe,"extractSpeechFeatures")

The HelperSpeechCommandRecognition supporting function encapsulates the feature extraction and network prediction process demonstrated previously. So that the feature extraction is compatible with code generation, feature extraction is handled by the generated extractSpeechFeatures function. So that the network is compatible with code generation, the supporting function uses the coder.loadDeepLearningNetwork (MATLAB Coder) function to load the network.

Use the HelperSpeechCommandRecognition function to perform live detection of speech commands.

show(timeScope)
show(matrixViewer)
timeLimit = 10;

tic
while isVisible(timeScope) && isVisible(matrixViewer) && toc < timeLimit
    x = adr();    
        
    [speechCommandIdx, auditorySpectrum] = HelperSpeechCommandRecognition(x);  
		
    matrixViewer(auditorySpectrum);
    timeScope(x);
   
    if (speechCommandIdx == backgroundIdx)
        timeScope.Title = ' ';
    else
        timeScope.Title = char(labels(speechCommandIdx));
    end
    drawnow
end

Hide the scopes.

hide(timeScope)
hide(matrixViewer)

Generate MATLAB Executable

Create a code generation configuration object for generation of an executable program. Specify the target language as C++.

cfg = coder.config('mex');
cfg.TargetLang = 'C++';

Create a configuration object for deep learning code generation with the MKL-DNN library. Attach the configuration object to the code generation configuration object.

dlcfg = coder.DeepLearningConfig('mkldnn');
cfg.DeepLearningConfig = dlcfg;

Call codegen (MATLAB Coder) to generate C++ code for the HelperSpeechCommandRecognition function. Specify the configuration object and prototype arguments. A MEX file named HelperSpeechCommandRecognition_mex is generated to your current folder.

codegen HelperSpeechCommandRecognition -config cfg -args {rand(samplesPerCapture, 1, 'single')} -profile -report -v
### Compiling function(s) HelperSpeechCommandRecognition ...
------------------------------------------------------------------------
[1/22] cl /c /Zp8 /GR /W3 /EHs /nologo /MD /D_CRT_SECURE_NO_DEPRECATE /D_SCL_SECURE_NO_DEPRECATE /D_SECURE_SCL=0   /DMATLAB_MEX_FILE  /DMATLAB_MEX_FILE /O2 /Oy- /DNDEBUG /fp:strict /utf-8 /I "." /I "C:\Users\bhemmat\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\bhemmat.Bdoc24a.j2459562\deeplearning_shared-ex90506783" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include\mkldnn" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\layer" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\onednn" /I ".\interface" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include" /I "." -DMODEL=HelperSpeechCommandRecognition_mex -DMW_NEEDS_VERSION_H /wd4251  HelperSpeechCommandRecognition_data.cpp /Fobuild\win64\HelperSpeechCommandRecognition_data.obj
HelperSpeechCommandRecognition_data.cpp
[2/22] cl /c /Zp8 /GR /W3 /EHs /nologo /MD /D_CRT_SECURE_NO_DEPRECATE /D_SCL_SECURE_NO_DEPRECATE /D_SECURE_SCL=0   /DMATLAB_MEX_FILE  /DMATLAB_MEX_FILE /O2 /Oy- /DNDEBUG /fp:strict /utf-8 /I "." /I "C:\Users\bhemmat\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\bhemmat.Bdoc24a.j2459562\deeplearning_shared-ex90506783" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include\mkldnn" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\layer" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\onednn" /I ".\interface" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include" /I "." -DMODEL=HelperSpeechCommandRecognition_mex -DMW_NEEDS_VERSION_H /wd4251  eml_int_forloop_overflow_check.cpp /Fobuild\win64\eml_int_forloop_overflow_check.obj
eml_int_forloop_overflow_check.cpp
[3/22] cl /c /Zp8 /GR /W3 /EHs /nologo /MD /D_CRT_SECURE_NO_DEPRECATE /D_SCL_SECURE_NO_DEPRECATE /D_SECURE_SCL=0   /DMATLAB_MEX_FILE  /DMATLAB_MEX_FILE /O2 /Oy- /DNDEBUG /fp:strict /utf-8 /I "." /I "C:\Users\bhemmat\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\bhemmat.Bdoc24a.j2459562\deeplearning_shared-ex90506783" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include\mkldnn" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\layer" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\onednn" /I ".\interface" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include" /I "." -DMODEL=HelperSpeechCommandRecognition_mex -DMW_NEEDS_VERSION_H /wd4251  rt_nonfinite.cpp /Fobuild\win64\rt_nonfinite.obj
rt_nonfinite.cpp
[4/22] cl /c /Zp8 /GR /W3 /EHs /nologo /MD /D_CRT_SECURE_NO_DEPRECATE /D_SCL_SECURE_NO_DEPRECATE /D_SECURE_SCL=0   /DMATLAB_MEX_FILE  /DMATLAB_MEX_FILE /O2 /Oy- /DNDEBUG /fp:strict /utf-8 /I "." /I "C:\Users\bhemmat\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\bhemmat.Bdoc24a.j2459562\deeplearning_shared-ex90506783" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include\mkldnn" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\layer" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\onednn" /I ".\interface" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include" /I "." -DMODEL=HelperSpeechCommandRecognition_mex -DMW_NEEDS_VERSION_H /wd4251  colon.cpp /Fobuild\win64\colon.obj
colon.cpp
[5/22] cl /c /Zp8 /GR /W3 /EHs /nologo /MD /D_CRT_SECURE_NO_DEPRECATE /D_SCL_SECURE_NO_DEPRECATE /D_SECURE_SCL=0   /DMATLAB_MEX_FILE  /DMATLAB_MEX_FILE /O2 /Oy- /DNDEBUG /fp:strict /utf-8 /I "." /I "C:\Users\bhemmat\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\bhemmat.Bdoc24a.j2459562\deeplearning_shared-ex90506783" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include\mkldnn" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\layer" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\onednn" /I ".\interface" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include" /I "." -DMODEL=HelperSpeechCommandRecognition_mex -DMW_NEEDS_VERSION_H /wd4251  HelperSpeechCommandRecognition_initialize.cpp /Fobuild\win64\HelperSpeechCommandRecognition_initialize.obj
HelperSpeechCommandRecognition_initialize.cpp
[6/22] cl /c /Zp8 /GR /W3 /EHs /nologo /MD /D_CRT_SECURE_NO_DEPRECATE /D_SCL_SECURE_NO_DEPRECATE /D_SECURE_SCL=0   /DMATLAB_MEX_FILE  /DMATLAB_MEX_FILE /O2 /Oy- /DNDEBUG /fp:strict /utf-8 /I "." /I "C:\Users\bhemmat\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\bhemmat.Bdoc24a.j2459562\deeplearning_shared-ex90506783" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include\mkldnn" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\layer" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\onednn" /I ".\interface" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include" /I "." -DMODEL=HelperSpeechCommandRecognition_mex -DMW_NEEDS_VERSION_H /wd4251  HelperSpeechCommandRecognition_terminate.cpp /Fobuild\win64\HelperSpeechCommandRecognition_terminate.obj
HelperSpeechCommandRecognition_terminate.cpp
[7/22] cl /c /Zp8 /GR /W3 /EHs /nologo /MD /D_CRT_SECURE_NO_DEPRECATE /D_SCL_SECURE_NO_DEPRECATE /D_SECURE_SCL=0   /DMATLAB_MEX_FILE  /DMATLAB_MEX_FILE /O2 /Oy- /DNDEBUG /fp:strict /utf-8 /I "." /I "C:\Users\bhemmat\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\bhemmat.Bdoc24a.j2459562\deeplearning_shared-ex90506783" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include\mkldnn" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\layer" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\onednn" /I ".\interface" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include" /I "." -DMODEL=HelperSpeechCommandRecognition_mex -DMW_NEEDS_VERSION_H /wd4251  log10.cpp /Fobuild\win64\log10.obj
log10.cpp
[8/22] cl /c /Zp8 /GR /W3 /EHs /nologo /MD /D_CRT_SECURE_NO_DEPRECATE /D_SCL_SECURE_NO_DEPRECATE /D_SECURE_SCL=0   /DMATLAB_MEX_FILE  /DMATLAB_MEX_FILE /O2 /Oy- /DNDEBUG /fp:strict /utf-8 /I "." /I "C:\Users\bhemmat\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\bhemmat.Bdoc24a.j2459562\deeplearning_shared-ex90506783" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include\mkldnn" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\layer" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\onednn" /I ".\interface" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include" /I "." -DMODEL=HelperSpeechCommandRecognition_mex -DMW_NEEDS_VERSION_H /wd4251  interface\_coder_HelperSpeechCommandRecognition_mex.cpp /Fobuild\win64\_coder_HelperSpeechCommandRecognition_mex.obj
_coder_HelperSpeechCommandRecognition_mex.cpp
[9/22] cl /c /Zp8 /GR /W3 /EHs /nologo /MD /D_CRT_SECURE_NO_DEPRECATE /D_SCL_SECURE_NO_DEPRECATE /D_SECURE_SCL=0   /DMATLAB_MEX_FILE  /DMATLAB_MEX_FILE /O2 /Oy- /DNDEBUG /fp:strict /utf-8 /I "." /I "C:\Users\bhemmat\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\bhemmat.Bdoc24a.j2459562\deeplearning_shared-ex90506783" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include\mkldnn" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\layer" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\onednn" /I ".\interface" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include" /I "." -DMODEL=HelperSpeechCommandRecognition_mex -DMW_NEEDS_VERSION_H /wd4251  interface\_coder_HelperSpeechCommandRecognition_api.cpp /Fobuild\win64\_coder_HelperSpeechCommandRecognition_api.obj
_coder_HelperSpeechCommandRecognition_api.cpp
[10/22] cl /c /Zp8 /GR /W3 /EHs /nologo /MD /D_CRT_SECURE_NO_DEPRECATE /D_SCL_SECURE_NO_DEPRECATE /D_SECURE_SCL=0   /DMATLAB_MEX_FILE  /DMATLAB_MEX_FILE /O2 /Oy- /DNDEBUG /fp:strict /utf-8 /I "." /I "C:\Users\bhemmat\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\bhemmat.Bdoc24a.j2459562\deeplearning_shared-ex90506783" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include\mkldnn" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\layer" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\onednn" /I ".\interface" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include" /I "." -DMODEL=HelperSpeechCommandRecognition_mex -DMW_NEEDS_VERSION_H /wd4251  sort.cpp /Fobuild\win64\sort.obj
sort.cpp
[11/22] cl /c /Zp8 /GR /W3 /EHs /nologo /MD /D_CRT_SECURE_NO_DEPRECATE /D_SCL_SECURE_NO_DEPRECATE /D_SECURE_SCL=0   /DMATLAB_MEX_FILE  /DMATLAB_MEX_FILE /O2 /Oy- /DNDEBUG /fp:strict /utf-8 /I "." /I "C:\Users\bhemmat\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\bhemmat.Bdoc24a.j2459562\deeplearning_shared-ex90506783" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include\mkldnn" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\layer" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\onednn" /I ".\interface" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include" /I "." -DMODEL=HelperSpeechCommandRecognition_mex -DMW_NEEDS_VERSION_H /wd4251  permute.cpp /Fobuild\win64\permute.obj
permute.cpp
[12/22] cl /c /Zp8 /GR /W3 /EHs /nologo /MD /D_CRT_SECURE_NO_DEPRECATE /D_SCL_SECURE_NO_DEPRECATE /D_SECURE_SCL=0   /DMATLAB_MEX_FILE  /DMATLAB_MEX_FILE /O2 /Oy- /DNDEBUG /fp:strict /utf-8 /I "." /I "C:\Users\bhemmat\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\bhemmat.Bdoc24a.j2459562\deeplearning_shared-ex90506783" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include\mkldnn" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\layer" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\onednn" /I ".\interface" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include" /I "." -DMODEL=HelperSpeechCommandRecognition_mex -DMW_NEEDS_VERSION_H /wd4251  Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\version\cpp_mexapi_version.cpp /Fobuild\win64\cpp_mexapi_version.obj
cpp_mexapi_version.cpp
[13/22] cl /c /Zp8 /GR /W3 /EHs /nologo /MD /D_CRT_SECURE_NO_DEPRECATE /D_SCL_SECURE_NO_DEPRECATE /D_SECURE_SCL=0   /DMATLAB_MEX_FILE  /DMATLAB_MEX_FILE /O2 /Oy- /DNDEBUG /fp:strict /utf-8 /I "." /I "C:\Users\bhemmat\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\bhemmat.Bdoc24a.j2459562\deeplearning_shared-ex90506783" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include\mkldnn" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\layer" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\onednn" /I ".\interface" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include" /I "." -DMODEL=HelperSpeechCommandRecognition_mex -DMW_NEEDS_VERSION_H /wd4251  interface\_coder_HelperSpeechCommandRecognition_info.cpp /Fobuild\win64\_coder_HelperSpeechCommandRecognition_info.obj
_coder_HelperSpeechCommandRecognition_info.cpp
[14/22] cl /c /Zp8 /GR /W3 /EHs /nologo /MD /D_CRT_SECURE_NO_DEPRECATE /D_SCL_SECURE_NO_DEPRECATE /D_SECURE_SCL=0   /DMATLAB_MEX_FILE  /DMATLAB_MEX_FILE /O2 /Oy- /DNDEBUG /fp:strict /utf-8 /I "." /I "C:\Users\bhemmat\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\bhemmat.Bdoc24a.j2459562\deeplearning_shared-ex90506783" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include\mkldnn" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\layer" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\onednn" /I ".\interface" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include" /I "." -DMODEL=HelperSpeechCommandRecognition_mex -DMW_NEEDS_VERSION_H /wd4251  HelperSpeechCommandRecognition_mexutil.cpp /Fobuild\win64\HelperSpeechCommandRecognition_mexutil.obj
HelperSpeechCommandRecognition_mexutil.cpp
[15/22] cl /c /Zp8 /GR /W3 /EHs /nologo /MD /D_CRT_SECURE_NO_DEPRECATE /D_SCL_SECURE_NO_DEPRECATE /D_SECURE_SCL=0   /DMATLAB_MEX_FILE  /DMATLAB_MEX_FILE /O2 /Oy- /DNDEBUG /fp:strict /utf-8 /I "." /I "C:\Users\bhemmat\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\bhemmat.Bdoc24a.j2459562\deeplearning_shared-ex90506783" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include\mkldnn" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\layer" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\onednn" /I ".\interface" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include" /I "." -DMODEL=HelperSpeechCommandRecognition_mex -DMW_NEEDS_VERSION_H /wd4251  extractSpeechFeatures.cpp /Fobuild\win64\extractSpeechFeatures.obj
extractSpeechFeatures.cpp
[16/22] cl /c /Zp8 /GR /W3 /EHs /nologo /MD /D_CRT_SECURE_NO_DEPRECATE /D_SCL_SECURE_NO_DEPRECATE /D_SECURE_SCL=0   /DMATLAB_MEX_FILE  /DMATLAB_MEX_FILE /O2 /Oy- /DNDEBUG /fp:strict /utf-8 /I "." /I "C:\Users\bhemmat\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\bhemmat.Bdoc24a.j2459562\deeplearning_shared-ex90506783" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include\mkldnn" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\layer" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\onednn" /I ".\interface" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include" /I "." -DMODEL=HelperSpeechCommandRecognition_mex -DMW_NEEDS_VERSION_H /wd4251  stft.cpp /Fobuild\win64\stft.obj
stft.cpp
[17/22] cl /c /Zp8 /GR /W3 /EHs /nologo /MD /D_CRT_SECURE_NO_DEPRECATE /D_SCL_SECURE_NO_DEPRECATE /D_SECURE_SCL=0   /DMATLAB_MEX_FILE  /DMATLAB_MEX_FILE /O2 /Oy- /DNDEBUG /fp:strict /utf-8 /I "." /I "C:\Users\bhemmat\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\bhemmat.Bdoc24a.j2459562\deeplearning_shared-ex90506783" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include\mkldnn" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\layer" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\onednn" /I ".\interface" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include" /I "." -DMODEL=HelperSpeechCommandRecognition_mex -DMW_NEEDS_VERSION_H /wd4251  sortIdx.cpp /Fobuild\win64\sortIdx.obj
sortIdx.cpp
[18/22] cl /c /Zp8 /GR /W3 /EHs /nologo /MD /D_CRT_SECURE_NO_DEPRECATE /D_SCL_SECURE_NO_DEPRECATE /D_SECURE_SCL=0   /DMATLAB_MEX_FILE  /DMATLAB_MEX_FILE /O2 /Oy- /DNDEBUG /fp:strict /utf-8 /I "." /I "C:\Users\bhemmat\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\bhemmat.Bdoc24a.j2459562\deeplearning_shared-ex90506783" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include\mkldnn" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\layer" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\onednn" /I ".\interface" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include" /I "." -DMODEL=HelperSpeechCommandRecognition_mex -DMW_NEEDS_VERSION_H /wd4251  DeepLearningNetwork.cpp /Fobuild\win64\DeepLearningNetwork.obj
DeepLearningNetwork.cpp
[19/22] cl /c /Zp8 /GR /W3 /EHs /nologo /MD /D_CRT_SECURE_NO_DEPRECATE /D_SCL_SECURE_NO_DEPRECATE /D_SECURE_SCL=0   /DMATLAB_MEX_FILE  /DMATLAB_MEX_FILE /O2 /Oy- /DNDEBUG /fp:strict /utf-8 /I "." /I "C:\Users\bhemmat\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\bhemmat.Bdoc24a.j2459562\deeplearning_shared-ex90506783" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include\mkldnn" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\layer" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\onednn" /I ".\interface" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include" /I "." -DMODEL=HelperSpeechCommandRecognition_mex -DMW_NEEDS_VERSION_H /wd4251  predict.cpp /Fobuild\win64\predict.obj
predict.cpp
[20/22] cl /c /Zp8 /GR /W3 /EHs /nologo /MD /D_CRT_SECURE_NO_DEPRECATE /D_SCL_SECURE_NO_DEPRECATE /D_SECURE_SCL=0   /DMATLAB_MEX_FILE  /DMATLAB_MEX_FILE /O2 /Oy- /DNDEBUG /fp:strict /utf-8 /I "." /I "C:\Users\bhemmat\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\bhemmat.Bdoc24a.j2459562\deeplearning_shared-ex90506783" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include\mkldnn" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\layer" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\onednn" /I ".\interface" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include" /I "." -DMODEL=HelperSpeechCommandRecognition_mex -DMW_NEEDS_VERSION_H /wd4251  HelperSpeechCommandRecognition.cpp /Fobuild\win64\HelperSpeechCommandRecognition.obj
HelperSpeechCommandRecognition.cpp
[21/22] cl /c /Zp8 /GR /W3 /EHs /nologo /MD /D_CRT_SECURE_NO_DEPRECATE /D_SCL_SECURE_NO_DEPRECATE /D_SECURE_SCL=0   /DMATLAB_MEX_FILE  /DMATLAB_MEX_FILE /O2 /Oy- /DNDEBUG /fp:strict /utf-8 /I "." /I "C:\Users\bhemmat\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\bhemmat.Bdoc24a.j2459562\deeplearning_shared-ex90506783" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include\mkldnn" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\layer" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\derived\win64\src\include\onednn" /I ".\interface" /I "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\include" /I "." -DMODEL=HelperSpeechCommandRecognition_mex -DMW_NEEDS_VERSION_H /wd4251  AsyncBuffer.cpp /Fobuild\win64\AsyncBuffer.obj
AsyncBuffer.cpp
[22/22] link build\win64\HelperSpeechCommandRecognition_data.obj build\win64\rt_nonfinite.obj build\win64\HelperSpeechCommandRecognition_initialize.obj build\win64\HelperSpeechCommandRecognition_terminate.obj build\win64\HelperSpeechCommandRecognition.obj build\win64\DeepLearningNetwork.obj build\win64\colon.obj build\win64\stft.obj build\win64\permute.obj build\win64\predict.obj build\win64\extractSpeechFeatures.obj build\win64\_coder_HelperSpeechCommandRecognition_api.obj build\win64\_coder_HelperSpeechCommandRecognition_mex.obj build\win64\eml_int_forloop_overflow_check.obj build\win64\log10.obj build\win64\sort.obj build\win64\sortIdx.obj build\win64\AsyncBuffer.obj build\win64\HelperSpeechCommandRecognition_mexutil.obj build\win64\_coder_HelperSpeechCommandRecognition_info.obj build\win64\cpp_mexapi_version.obj "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\lib\win64\mwdnnl.lib" "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\lib\win64\shared_layers.lib" "Z:\85\bhemmat.Bdoc24a.j2459562\matlab\lib\win64\onednn_layers.lib" /nologo /manifest   /DLL /LIBPATH:"Z:\85\bhemmat.Bdoc24a.j2459562\matlab\extern\lib\win64\microsoft" libmx.lib libmex.lib libmat.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib libMatlabDataArray.lib libMatlabEngine.lib  /out:"HelperSpeechCommandRecognition_mex.mexw64" libemlrt.lib libcovrt.lib libut.lib libmwmathutil.lib 
   Creating library HelperSpeechCommandRecognition_mex.lib and object HelperSpeechCommandRecognition_mex.exp

------------------------------------------------------------------------
### Generating compilation report ...
Code generation successful: View report

Perform Speech Command Recognition Using Deployed Code

Show the time scope and matrix viewer. Detect commands using the generated MEX for as long as both the time scope and matrix viewer are open or until the time limit is reached. To stop the live detection before the time limit is reached, close the time scope window or matrix viewer window.

show(timeScope)
show(matrixViewer)

timeLimit = 20;

tic
while isVisible(timeScope) && isVisible(matrixViewer) && toc < timeLimit
    x = adr();    
        
    [speechCommandIdx, auditorySpectrum] = HelperSpeechCommandRecognition_mex(x);
		
    matrixViewer(auditorySpectrum);
    timeScope(x);
   
    if (speechCommandIdx == backgroundIdx)
        timeScope.Title = ' ';
    else
        timeScope.Title = char(labels(speechCommandIdx));
    end
    drawnow
end

hide(matrixViewer)
hide(timeScope)

LivespeechCommandRecognsnapshot-right.png

Evaluate MEX Execution Time

Use tic and toc to compare the execution time to run the simulation completely in MATLAB with the execution time of the MEX function.

Measure the performance of the simulation code.

testDur = 50e-3;
x = pinknoise(fs*testDur,'single');
numLoops = 100;
tic
for k = 1:numLoops
    [speechCommandIdx, auditory_features] = HelperSpeechCommandRecognition(x);
end
exeTime = toc;
fprintf('SIM execution time per 50 ms of audio = %0.4f ms\n',(exeTime/numLoops)*1000);
SIM execution time per 50 ms of audio = 11.4019 ms

Measure the performance of the MEX code.

tic
for k = 1:numLoops
    [speechCommandIdx, auditory_features] = HelperSpeechCommandRecognition_mex(x);
end
exeTimeMex = toc;
fprintf('MEX execution time per 50 ms of audio = %0.4f ms\n',(exeTimeMex/numLoops)*1000);
MEX execution time per 50 ms of audio = 1.6856 ms

Evaluate the performance gained from using the MEX function. This performance test is performed on a machine using NVIDIA Titan V GPU and Intel(R) Xeon(R) W-2133 CPU running at 3.60 GHz.

PerformanceGain = exeTime/exeTimeMex
PerformanceGain = 6.7643