This example shows how to create an application to add echo effect to an audio signal captured from the microphone of an NVIDIA Jetson board. The application then sends this output to the playback device connected to the Jetson board.
Prerequisites
Target Board Requirements
NVIDIA Jetson embedded platform.
USB audio device for recording and playback of audio signals.
Sound eXchange (SoX) utility and its development and format libraries.
Environment variables on the target for the compilers and libraries. For more information, see Install and Setup Prerequisites for NVIDIA Boards.
Connect to NVIDIA Board
Create a live hardware connection from the MATLAB® software to the NVIDIA Jetson hardware by using the jetson function. This example reuses the settings from the most recent successful connection to a NVIDIA Jetson board.
Checking for CUDA availability on the Target...
Checking for 'nvcc' in the target system path...
Checking for cuDNN library availability on the Target...
Checking for TensorRT library availability on the Target...
Checking for prerequisite libraries is complete.
Gathering hardware details...
Checking for third-party library availability on the Target...
Gathering hardware details is complete.
Board name : NVIDIA Jetson TX2 Developer Kit
CUDA Version : 10.2
cuDNN Version : 8.2
TensorRT Version : 8.2
GStreamer Version : 1.14.5
V4L2 Version : 1.14.2-1
SDL Version : 1.2
OpenCV Version : 4.1.1
Available Webcams :
Available GPUs : NVIDIA Tegra X2
Available Digital Pins : 7 11 12 13 15 16 18 19 21 22 23 24 29 31 32 33 35 36 37 38 40
If this is your first time connecting to the target board,you must provide the host name or IP address, user name, and password of the target board. For example:
hwobj = jetson('jetson-board-name','ubuntu','ubuntu');
Identify Audio Devices Connected to NVIDIA Board
To find the list of audio capture devices connected to the target, use the listAudioDevices function. The output contains the Device IDs according to ALSA standards, the bit-depths, and the sample rates supported by the devices.
captureDevices = struct with fields:
Name: 'USB-Audio-PolyBlackwire3325Series↵PlantronicsPolyBlackwire3325Seriesatusb-3530000.xhci-2.3,fullspeed↵'
Device: '3,0'
Channels: {'1'}
BitDepth: {'16-bit integer'}
SamplingRate: {'8000' '16000'}
Similarly, to find the list of audio playback devices connected to the target, run the command.
playbackDevices = struct with fields:
Name: 'USB-Audio-PolyBlackwire3325Series↵PlantronicsPolyBlackwire3325Seriesatusb-3530000.xhci-2.3,fullspeed↵'
Device: '3,0'
Channels: {'2'}
BitDepth: {'16-bit integer'}
SamplingRate: {'8000' '48000'}
The echoGenerator Entry-Point Function
Open the echoGenerator() function. This function takes audio input from a device connected to an Nvidia board, adds an echo effect to the input and then sends this output to the playback device connected to the Nvidia board. The algorithm models the echo effect by delaying the audio signal and adding it back. Feedback is added to the delay line to give a fading effect. The echo effect is implemented in the audioexample.Echo class.
The echo algorithm has four parameters:
Gain - Linear gain of the delayed audio
Delay - Delay applied to audio signal, in seconds
FeedbackLevel - Feedback gain applied to delay line
WetDryMix - Ratio of wet signal added to dry signal
function echoGenerator(gain,feedbackLevel,delay,wetdrymix)
% echoGenerator Entry-point function to deploy an echo generator to NVIDIA
% hardware
% Copyright 2023 The MathWorks, Inc.
%#codegen
hwobj = jetson(); % Create connection to Nvidia board.
deviceName = "plughw:3,0"; % Change to device connected to your board.
% Create capture and playback objects for audio processing on the Nvidia
% board.
captureObj = audiocapture(hwobj,deviceName,"SampleRate",8000, ...
"SamplesPerFrame",800);
playbackObj = audioplayer(hwobj,deviceName,"SampleRate",8000);
Fs = 8192; % Sampling Frequency
echoObj = audiopluginexample.Echo;
setSampleRate(echoObj,Fs);
for k = 1:3000
% Capture audio input from the input device.
input = capture(captureObj);
echoSignal = zeros(size(double(input)),"like",double(input)); %#ok<PREALL>
echoObj.Gain = gain;
echoObj.FeedbackLevel = feedbackLevel;
echoObj.Delay = delay;
echoObj.WetDryMix = wetdrymix;
[echoSignal] = process(echoObj,double(input));
% Playback audio output using the output device output data needs to be
% of type int16.
play(playbackObj,int16(echoSignal));
end
end
Generate C++ Code for the Jetson Board
To generate a C++ executable that you can deploy on to an NVIDIA target, create a code configuration object for generating an executable.
To create a configuration object for the Jetson platform and assign it to the Hardware property of the code configuration object cfg, use the coder.hardware function.
To specify the folder for performing remote build process on the target board, use the BuildDir property. If the specified build folder does not exist on the target board, then the software creates a folder with the given name. If no value is assigned to cfg.Hardware.BuildDir, the remote build process occurs in the last specified build folder. If there is no stored build folder value, the build process takes place in the home folder.
Set the GenerateExampleMain property to generate an example C++ main file and compile it. This example does not require modifications to the generated main files.
To generate C++ code, use the codegen function and pass the code configuration and the size of the inputs for echoGenerator entry-point function. After the code generation takes place on the host, the generated files are copied over and built on the target.
Code generation successful: View report
Run echoGenerator on Target Board
To run the generated executable on the target board, use the MATLAB runApplication function.
### Launching the executable on the target...
Executable launched successfully with process ID 13545.
Displaying the simple runtime log for the executable...
Note: For the complete log, run the following command in the MATLAB command window:
system(hwobj,'cat /home/ubuntu/remoteBuildDir/MATLAB_ws/R2023b/home/lnarasim/Documents/MATLAB/ExampleManager/lnarasim.Bdoc23b.j2313133.1/nvidia-ex27597808/echoGenerator.log')
On successful deployment, hold the audio capture device close to your mouth and start speaking. You can hear the echo of your voice through the audio playback device.
Terminate Application on Target Board
To kill the executable launched:
killApplication(hwobj,'echoGenerator')
