Deploy Algorithms to Arduino using MATLAB Function blocks in Simulink

What is going on, everybody? My name is Meeshawn and welcome to this video on how to deploy your MATLAB Scripts to Arduino using Simulink. Wow, that’s new! Deploying MATLAB Scripts using Simulink?

You must be thinking: Is that even possible ?

Well, Simulink enables you to insert your MATLAB Scripts inside a MATLAB Function block and then generate code right away from Simulink. Don’t worry, we’ll talk about MATLAB Function blocks later in the video, but for now, let me just give you a quick demonstration of an audio processing application which I have developed in Simulink. I have implemented audio effects on a sound clip, generated code using Simulink, and then deployed the code to Arduino MKR1000 board from Simulink.

This is a Simulink model with a MATLAB Function block, which contains the implementation of the audio effects. The code generated from Simulink is running in real time on this board and I’m using Simulink to tune parameters and monitor signals via External mode.

So, you just now heard the original sound clip, followed by three different audio effects, implemented on the clip; namely, the echo effect, reverberation, and pitch-shift effect. Interesting, isn’t it?

Now, let’s take a closer look at the audio application developed in Simulink, starting with MATLAB Function blocks. Let’s open a blank Simulink model, and then navigate to Simulink Library. Under User-Defined Functions, you can locate the MATLAB Function block. Let’s add this to the blank model. You could also start typing MATLAB Function anywhere in the Simulink editor and obtain this block. Let’s add a constant block and then a display block. Now, let’s double-click this block. So, here is where you will implement your MATLAB Scripts. Let’s see a simple scalar multiplication example. Multiplying the input by 2. Now let’s just run this Simulink model. Perfect! The answer is 2, as expected.

So, we just saw a very basic MATLAB Function block implementation in Simulink. The algorithms behind the audio effects implemented in our Simulink model were developed as MATLAB Scripts and then used inside these MATLAB Function blocks in Simulink. Code is then generated and deployed directly to the Arduino MKR1000 board from Simulink. The audio output is available at the DAC pin of the board. In order to hear this, I just used a 3.5 mm audio breakout board and then connected it to the DAC pin and GND pin on the board. A headset/earphone can now be connected to this breakout board to hear the sound. Don’t worry if you don’t have this audio breakout board. You could simply wrap wires around the audio jack of your headset/earphones and connect it to the DAC and GND pins on the board appropriately.

That’s it! So for this audio application you just need a headset, Arduino, and Simulink!

All right. So now, with this information, let’s get back to our Simulink model. Apart from the MATLAB Function blocks, there are a couple of other i/o blocks used in this application. This is a constant block which is used to access the sound clip for the implementation of audio effects. An interesting thing to note here is that Simulink enables one to store data, such as the sound clip here, in the flash memory, which is useful when your target hardware doesn’t have sufficient RAM memory. In order to do this, you will need to set the sample time of the constant block to infinity, which is already done for this block. Secondly, you will need to set the default parameter behavior to inline, within the code generation optimization settings. Let’s quickly take a look at this. Let’s go to Configuration Parameters; under Code Generation tab, Optimization settings, you can find the default parameter behavior. It is already set to inlined.

All right, so our next block is the Analog output block for Arduino. It generates voltage on the specified DAC pin on the board. It is a part of the Simulink Support Package for Arduino.

Our next block is the To Workspace block. It helps us save the simulation output to a variable in the workspace in order to perform any post-processing or analysis of data in MATLAB.

Next, let’s check out this rotating knob. It is an interactive UI block which helps select an audio effect by assigning the selected value to a variable or a constant. As you can see here, there are three audio effects implemented in this application. Echo effect, reverberation effect, and pitch-shift effect. The option “original” refers to the original sound clip with no audio effects. The algorithm selected by the rotating knob is stored as a constant in this constant block. The MATLAB Function block then makes use of this constant value to implement the selected audio effect.

Now, if you’re interested in going through the original sound clip properties, click on this push button. It launches a MATLAB Script which reads the sound clip and defines many constants from its properties which are then used in this Simulink model. This script is run initially before loading the Simulink model.

All right, so we saw a couple of i/o and UI blocks. Now let’s focus on the brain of the model, which is the MATLAB Function block. In the center, you can see a subsystem block. This block groups all the MATLAB Function blocks corresponding to the audio processing application implemented in this Simulink model.

The first MATLAB Function block plays the original sound clip as it is. The script inside the block writes each audio sample to the DAC pin on the board. No other computation takes place in this script. Now since the Analog Output block only accepts a uint16 data type, all the variables and constants used inside the MATLAB Fcn Block are of uint16 data type. 

Our second MATLAB Fcn Block implements the echo audio effect. Before going through the script, lets first understand what an echo effect is. The echo effect is simply a superposition of current and delayed audio samples. As a result, one can perceive the present as well as past audio samples at the same time. This effect is known as the echo effect. As seen in this equation, the output is a weighted sum of the present and past audio samples. The parameter alpha controls the strength of the echo. TauD is the number of delayed samples corresponding to the actual delay in seconds. The final output equation is normalized, using the expression (1+alpha), to take care of sound saturation during playback.

All right, so now that we understand how echo works, lets come back to the MATLAB Fcn block implementation for echo effect. As discussed earlier, uint16 data type is used in all the MATLAB Fcn blocks since that is what the Analog Output block supports. But then how do we make use of floating-point values for different variables and constants in our MATLAB Script? For example, a delay of 0.5 seconds and a gain 0.8?

In order to solve this issue, the floating-point values are expressed as rational numbers, i.e., in the numerator by denominator form. As you can see here, the delay in seconds and the gain have been expressed as rational numbers.

However, this does not solve the problem completely. These rational numbers are now rearranged in the final output equation by taking the LCM, so as to have a common divisor. This reduces the number of divisions to just one, which ultimately reduces the loss of precision due to division of unsigned integer values.

As you can see here, this is the final output equation of the echo effect and it has been rearranged to have a common divisor and hence a single division operation takes place, which helps achieve a better precision.

Right, so this was the echo effect. Next, let’s take a look at the reverberation effect. Also known as reverb, this effect is a persistence of sound after the sound has been produced, created by multiple reflections which eventually decay as the sound is absorbed by different objects. This means the output equation depends on the past audio input samples as well as the past audio output samples to model the reflection as well as the decay phenomenon respectively.

As in the echo output equation, the parameter alpha controls the strength of the effect and tauD is the number of delayed samples corresponding the actual delay in seconds.

In order to model multiple reflections and decays, the output equation has been implemented with three different tauD values in a cascade fashion so as to model subsequent reflections and decay.

All right, so now we can jump to the MATLAB Fcn block implementation for the reverberation effect. Similar to the echo effect, floating point variables and constants have been expressed as rational numbers and the final output equation has been rearranged to have a common divisor with a single division operation to reduce the loss of precision due to division of unsigned integer values.

This is the first difference equation corresponding to the first delay value. The output of this equation is fed as an input to the second difference equation corresponding to the second delay value and so forth.

Finally, the output is then superimposed with the audio input and then normalized to take care of the saturation.

All right, so ’til now we have discussed echo and reverb algorithms. Let’s talk a bit about the performance of these algorithms. The sampling rate of the sound clip is 8KHz. This means that in order to hear a meaningful audio effect implemented on the sound clip, the algorithm needs to work within 125uS, which is the inverse of 8KHz. Failure to do so will lead to overrunning of the algorithms on the board and audio effect will be undesirable. Both echo as well as reverb algorithms have been optimized such that the execution time at every time step in Simulink is lesser than 125uS.

All right, so now let’s move on to our final algorithm, which is the pitch-shifting effect. This effect is achieved by stretching or compressing the sound clip and then playing back the resulting signal at a rate equal to the stretch or compression factor, resulting in an increase or decrease in the pitch respectively. For example, the stretching is achieved by fixing upon small window lengths at equal intervals of the sound clip and then extending the sound clip by appending the window at those intervals. This is essentially duplicating the sound clip at equal time intervals. The resultant sound clip has twice the number of samples now. Playing the resultant sound clip at twice the sample rate or playback rate the resultant sound clip plays for the same time duration, but with an increase in the pitch. Instead of appending audio samples, one can remove audio samples at equal intervals and then play back the signal at a slower rate to obtain a reduction in the pitch.

So now with this understanding, lets go back to our Simulink model, to the MATLAB Fcn block implementing the pitch-shift effect. A window length of 20ms has been implemented here. The algorithm accumulates audio samples equal to the window length and then appends it to the original sound clip at regular intervals.

Now as discussed, with the pitch-shift algorithm, we need to double the playback rate or, in other words, reduce the sample time of the MATLAB Fcn block by half, compared to the other algorithms. This scheduling can be easily done in Simulink. In fact, one of the core strengths of Simulink has been its scheduling feature. It’s very convenient to perform scheduling for different blocks within a Simulink model. It is as simple as setting the sample time from the block parameters. Let’s right click on a particular block. Navigate to block parameters. And here is the sample time. So for the pitch-shift MATLAB Fcn block, the sample time should be half as others which means twice the sampling rate, as required by the algorithm.

Additionally, we would require rate transition blocks, which take care of multi-rate modeling in Simulink. Since the pitch-shift MATLAB Fcn block runs at twice the sample rate, these blocks are inserted before and after this MATLAB Fcn block. If you don’t have multi-rates in your model, unlike me, you won’t need them.

The different line colors indicate the different sample times present in the model. Red is the fastest and green the slowest. Pink referrers to a constant, meaning having infinite sample time. Pink is like your friend! You’d ideally want a pink line to ensure that the parameter is always available for simulation and behaves like a constant and hence goes in to the flash memory during code generation.

Indeed, the scheduling feature in Simulink is something incredible since it can sometimes get too cumbersome to implement the same in other IDEs.

All right, so those were the three MATLAB Fcn block implementations for the three audio effects designed for this application. Now let’s simulate the model in Simulink before deploying the code to the hardware. Selecting “normal mode” simulates the model in the host computer, which is this laptop. Setting the duration of the simulation to 5s, since the length of the audio clip is 5s.

Okay, so now let’s play the simulation data. Normally I could add an “audio write” block from the Audio System Toolbox library to this model and then play the sound. However, if you don’t have one, you could play the sound from MATLAB, using the variable created at the end of the simulation by the To-Workspace block. This push button under the hood triggers a callback function which picks up the variable from the workspace and plays it from MATLAB.

That was the original sound clip. Now let’s implement some audio effects on the clip. So this is how an echo sounds. Next, let’s click on reverb and simulate. So, that was reverberation effect. Selecting our final algorithm, the pitch-shift effect. Let’s simulate for this effect. Perfect! So the simulation went well with all the algorithms.

Now we’ll move on to Target hardware deployment. Once deployed to the hardware, at a time only one algorithm will be running on the hardware. There is no way to switch between algorithms dynamically unless we connect some external push button switches which will only add more and more hardware to this application.

Fortunately, we do have a mechanism by which we can switch between algorithms dynamically from Simulink while they are running on the target. This is done by running the simulation in External Mode. The External Mode simulation is yet another interesting feature of Simulink which helps you run your model in real time on the hardware and apply live parameter changes in the Simulink model to the board. Hence, code generation is required only once to tune parameters and monitor signals. In our application, we will be switching between different algorithms while the code is running on the hardware with the help of External Mode simulation.

Before we begin, let’s quickly take a look at the hardware setup. It’s a simple setup with an Arduino MKR1000 board connected to the laptop with a micro USB cable and a 3.5 mm audio jack connected to the laptop at one end and the breakout board at the other end, as discussed previously, to record the final audio output. In your case, you would be simply connecting a suitable 3.5 mm headset to the breakout board, instead of this green audio cable, to listen to the audio output directly from Arduino.

All right, so now we will begin the external mode simulation, setting the time period to infinity since we will be running the simulation continuously. Let’s hit the run button. By default, the model is set to play the original sound clip. After playing the original sound clip, we will be selecting echo effect, followed by reverberation and pitch-shift effect, with the help of the rotating knob. These changes are directly communicated to the code running on the board with the help of External Mode simulation.

Well, well, so that was the external mode simulation for you.

With this simulation we come to the end of this video. The audio application implemented in this video was just a small example demonstrating the capabilities of MATLAB Fcn Blocks to implement your MATLAB Scripts in Simulink. You can go ahead and build your own applications using your MATLAB scripts along with MATLAB Fcn blocks in Simulink.

I hope you enjoyed watching this video. For more videos on other MATLAB and Simulink products, please visit our website www.mathworks.com/videos