dsp.LMSFilter

The mean squared error (MSE) measures the average of the squares of the errors between the desired signal and the primary signal input to the adaptive filter. Reducing this error converges the primary input to the desired signal. Determine the predicted value of MSE and the simulated value of MSE at each time instant using the msepred and msesim functions. Compare these MSE values with each other and with respect to the minimum MSE and steady-state MSE values. In addition, compute the sum of the squares of the coefficient errors given by the trace of the coefficient covariance matrix.

num = fir1(31,0.5);
fir = dsp.FIRFilter('Numerator',num);  
iir = dsp.IIRFilter('Numerator',sqrt(0.75),...
        'Denominator',[1 -0.5]);
x = iir(sign(randn(2000,25))); 
n = 0.1*randn(size(x));           
d = fir(x) + n; 

l = 32;
mu = 0.008;
m  = 5;

lms = dsp.LMSFilter('Length',l,'StepSize',mu);
[mmse,emse,meanW,mse,traceK] = msepred(lms,x,d,m);
[simmse,meanWsim,Wsim,traceKsim] = msesim(lms,x,d,m);

nn = m:m:size(x,1);
semilogy(nn,simmse,[0 size(x,1)],[(emse+mmse)...
    (emse+mmse)],nn,mse,[0 size(x,1)],[mmse mmse])
title('Mean Squared Error Performance')
axis([0 size(x,1) 0.001 10])
legend('MSE (Sim.)','Final MSE','MSE','Min. MSE')
xlabel('Time Index')
ylabel('Squared Error Value')

plot(nn,meanWsim(:,12),'b',nn,meanW(:,12),'r',nn,...
meanWsim(:,13:15),'b',nn,meanW(:,13:15),'r')
PlotTitle ={'Average Coefficient Trajectories for';...
            'W(12), W(13), W(14), and W(15)'}

PlotTitle = 2x1 cell
    {'Average Coefficient Trajectories for'}
    {'W(12), W(13), W(14), and W(15)'      }

title(PlotTitle)
legend('Simulation','Theory')
xlabel('Time Index')
ylabel('Coefficient Value')

semilogy(nn,traceKsim,nn,traceK,'r')
title('Sum-of-Squared Coefficient Errors')
axis([0 size(x,1) 0.0001 1])
legend('Simulation','Theory')
xlabel('Time Index')
ylabel('Squared Error Value')

The maxstep function computes the maximum step size of the adaptive filter. This step size keeps the filter stable at the maximum possible speed of convergence. Create the primary input signal, x, by passing a signed random signal to an IIR filter. Signal x contains 50 frames of 2000 samples each frame. Create an LMS filter with 32 taps and a step size of 0.1.

x = zeros(2000,50);
IIRFilter = dsp.IIRFilter('Numerator',sqrt(0.75),...
    'Denominator',[1 -0.5]);
for k = 1:size(x,2)   
  x(:,k) = IIRFilter(sign(randn(size(x,1),1))); 
end
mu = 0.1;     
LMSFilter = dsp.LMSFilter('Length',32,...
    'StepSize',mu);

Compute the maximum adaptation step size and the maximum step size in mean-squared sense using the maxstep function.

[mumax,mumaxmse] = maxstep(LMSFilter,x)

mumax = 
0.0625

mumaxmse = 
0.0536

System identification is the process of identifying the coefficients of an unknown system using an adaptive filter. The general overview of the process is shown in System Identification –– Using an Adaptive Filter to Identify an Unknown System. The main components involved are:

The objective of the adaptive filter is to minimize the error signal between the output of the adaptive filter $\mathit{y}\left(\mathit{k}\right)$ and the output of the unknown system (or the system to be identified) $\mathit{d}\left(\mathit{k}\right)$. Once the error signal is minimized, the adapted filter resembles the unknown system. The coefficients of both the filters match closely.

filt = dsp.FIRFilter;
filt.Numerator = fircband(12,[0 0.4 0.5 1],[1 1 0 0],[1 0.2],... 
{'w' 'c'});

x = 0.1*randn(250,1);
n = 0.01*randn(250,1);
d = filt(x) + n;

mu = 0.8;
lms = dsp.LMSFilter(13,'StepSize',mu)

lms = 
  dsp.LMSFilter with properties:

                   Method: 'LMS'
                   Length: 13
           StepSizeSource: 'Property'
                 StepSize: 0.8000
            LeakageFactor: 1
        InitialConditions: 0
           AdaptInputPort: false
    WeightsResetInputPort: false
            WeightsOutput: 'Last'

  Use get to show all properties

[y,e,w] = lms(x,d);
plot(1:250, [d,y,e])
title('System Identification of an FIR filter')
legend('Desired','Output','Error')
xlabel('Time index')
ylabel('Signal value')

stem([(filt.Numerator).' w])
title('System Identification by Adaptive LMS Algorithm')
legend('Actual filter weights','Estimated filter weights',...
       'Location','NorthEast')

mu = 0.2;
lms = dsp.LMSFilter(13,'StepSize',mu);
[~,~,w] = lms(x,d);
stem([(filt.Numerator).' w])
title('System Identification by Adaptive LMS Algorithm')
legend('Actual filter weights','Estimated filter weights',...
       'Location','NorthEast')

release(filt);
x = 0.1*randn(1000,1);
n = 0.01*randn(1000,1);
d = filt(x) + n;
[y,e,w] = lms(x,d);
stem([(filt.Numerator).' w])
title('System Identification by Adaptive LMS Algorithm')
legend('Actual filter weights','Estimated filter weights',...
       'Location','NorthEast')

release(filt);
n = 0.01*randn(1000,1);
for index = 1:4
  x = 0.1*randn(1000,1);
  d = filt(x) + n;
  [y,e,w] = lms(x,d);
end
stem([(filt.Numerator).' w])
title('System Identification by Adaptive LMS Algorithm')
legend('Actual filter weights','Estimated filter weights',...
       'Location','NorthEast')

plot(1:1000, [d,y,e])
title('System Identification of an FIR filter')
legend('Desired','Output','Error')
xlabel('Time index')
ylabel('Signal value')

To improve the convergence performance of the LMS algorithm, the normalized variant (NLMS) uses an adaptive step size based on the signal power. As the input signal power changes, the algorithm calculates the input power and adjusts the step size to maintain an appropriate value. The step size changes with time, and as a result, the normalized algorithm converges faster with fewer samples in many cases. For input signals that change slowly over time, the normalized LMS algorithm can be a more efficient LMS approach.

For an example using the LMS approach, see System Identification of FIR Filter Using LMS Algorithm.

filt = dsp.FIRFilter;
filt.Numerator = fircband(12,[0 0.4 0.5 1],[1 1 0 0],[1 0.2],... 
{'w' 'c'});

x = 0.1*randn(1000,1);
n = 0.001*randn(1000,1);
d = filt(x) + n;

mu = 0.2;
lms = dsp.LMSFilter(13,'StepSize',mu,'Method',...
   'Normalized LMS');

[y,e,w] = lms(x,d);

plot(1:1000, [d,y,e])
title('System Identification by Normalized LMS Algorithm')
legend('Desired','Output','Error')
xlabel('Time index')
ylabel('Signal value')

stem([(filt.Numerator).' w])
title('System Identification by Normalized LMS Algorithm')
legend('Actual filter weights','Estimated filter weights',...
       'Location','NorthEast')

An adaptive filter adapts its filter coefficients to match the coefficients of an unknown system. The objective is to minimize the error signal between the output of the unknown system and the output of the adaptive filter. When these two outputs converge and match closely for the same input, the coefficients are said to match closely. The adaptive filter at this state resembles the unknown system. This example compares the rate at which this convergence happens for the normalized LMS (NLMS) algorithm and the LMS algorithm with no normalization.

filt = dsp.FIRFilter;
filt.Numerator = fircband(12,[0 0.4 0.5 1],[1 1 0 0],[1 0.2],... 
{'w' 'c'});
x = 0.1*randn(1000,1);
n = 0.001*randn(1000,1);
d = filt(x) + n;

mu = 0.2;
lms_nonnormalized = dsp.LMSFilter(13,'StepSize',mu,...
    'Method','LMS');
lms_normalized = dsp.LMSFilter(13,'StepSize',mu,...
    'Method','Normalized LMS');

[~,e1,~] = lms_normalized(x,d);
[~,e2,~] = lms_nonnormalized(x,d);

plot([e1,e2]);
title('Comparing the LMS and NLMS Conversion Performance');
legend('NLMS derived filter weights', ...
       'LMS derived filter weights','Location', 'NorthEast');
xlabel('Time index')
ylabel('Signal value')

Cancel additive noise, n, added to an unknown system using an LMS adaptive filter. The LMS filter adapts its coefficients until its transfer function matches the transfer function of the unknown system as closely as possible. The difference between the output of the adaptive filter and the output of the unknown system represents the error signal, e. Minimizing this error signal is the objective of the adaptive filter.

The unknown system and the LMS filter process the same input signal, x, and produce outputs d and y, respectively. If the coefficients of the adaptive filter match the coefficients of the unknown system, the error, e, in effect represents the additive noise.

FrameSize = 100;
NIter = 10;
lmsfilt2 = dsp.LMSFilter('Length',11,'Method','Normalized LMS', ...
    'StepSize',0.05);
firfilt2 = dsp.FIRFilter('Numerator', fir1(10,[.5, .75]));
sinewave = dsp.SineWave('Frequency',0.01, ...
    'SampleRate',1,'SamplesPerFrame',FrameSize);
scope = timescope('TimeUnits','Seconds',...
    'YLimits',[-3 3],'BufferLength',2*FrameSize*NIter, ...
    'ShowLegend',true,'ChannelNames', ...
    {'Noisy signal', 'Error signal'});

for k = 1:NIter
    x = randn(FrameSize,1);
    d = firfilt2(x) + sinewave();
    [y,e,w] = lmsfilt2(x,d);
    scope([d,e])
end
release(scope)

When the amount of computation required to derive an adaptive filter drives your development process, the sign-data variant of the LMS (SDLMS) algorithm might be a very good choice, as demonstrated in this example.

In the standard and normalized variations of the LMS adaptive filter, coefficients for the adapting filter arise from the mean square error between the desired signal and the output signal from the unknown system. The sign-data algorithm changes the mean square error calculation by using the sign of the input data to change the filter coefficients.

When the error is positive, the new coefficients are the previous coefficients plus the error multiplied by the step size µ. If the error is negative, the new coefficients are again the previous coefficients minus the error multiplied by µ — note the sign change.

When the input is zero, the new coefficients are the same as the previous set.

In vector form, the sign-data LMS algorithm is:

where

with vector $\mathit{w}$ containing the weights applied to the filter coefficients and vector $\mathit{x}$ containing the input data. The vector $\mathit{e}$ is the error between the desired signal and the filtered signal. The objective of the SDLMS algorithm is to minimize this error. Step size is represented by $\mu$.

With a smaller $\mu$, the correction to the filter weights gets smaller for each sample, and the SDLMS error falls more slowly. A larger $\mu$ changes the weights more for each step, so the error falls more rapidly, but the resulting error does not approach the ideal solution as closely. To ensure a good convergence rate and stability, select $\mu$ within the following practical bounds.

where $\mathit{N}$ is the number of samples in the signal. Also, define $\mu$ as a power of two for efficient computing.

Note: How you set the initial conditions of the sign-data algorithm profoundly influences the effectiveness of the adaptation process. Because the algorithm essentially quantizes the input signal, the algorithm can become unstable easily.

A series of large input values, coupled with the quantization process might result in the error growing beyond all bounds. Restrain the tendency of the sign-data algorithm to get out of control by choosing a small step size $\left(\mu \ll 1\right)$ and setting the initial conditions for the algorithm to nonzero positive and negative values.

In this noise cancellation example, set the Method property of dsp.LMSFilter to 'Sign-Data LMS'. This example requires two input data sets:

For the signal, use a sine wave. Note that signal is a column vector of 1000 elements.

signal = sin(2*pi*0.055*(0:1000-1)');

Now, add correlated white noise to signal. To ensure that the noise is correlated, pass the noise through a lowpass FIR filter and then add the filtered noise to the signal.

noise = randn(1000,1);
filt = dsp.FIRFilter;
filt.Numerator = fir1(11,0.4);
fnoise = filt(noise);
d = signal + fnoise;

fnoise is the correlated noise and d is now the desired input to the sign-data algorithm.

To prepare the dsp.LMSFilter object for processing, set the initial conditions of the filter weights and mu (StepSize). As noted earlier in this section, the values you set for coeffs and mu determine whether the adaptive filter can remove the noise from the signal path.

In System Identification of FIR Filter Using LMS Algorithm, you constructed a default filter that sets the filter coefficients to zeros. In most cases that approach does not work for the sign-data algorithm. The closer you set your initial filter coefficients to the expected values, the more likely it is that the algorithm remains well behaved and converges to a filter solution that removes the noise effectively.

For this example, start with the coefficients used in the noise filter (filt.Numerator), and modify them slightly so the algorithm has to adapt.

coeffs = (filt.Numerator).'-0.01; % Set the filter initial conditions.
mu = 0.05; % Set the step size for algorithm updating.

With the required input arguments for dsp.LMSFilter prepared, construct the LMS filter object, run the adaptation, and view the results.

lms = dsp.LMSFilter(12,'Method','Sign-Data LMS',...
   'StepSize',mu,'InitialConditions',coeffs);
[~,e] = lms(noise,d);
L = 200;
plot(0:L-1,signal(1:L),0:L-1,e(1:L));
title('Noise Cancellation by the Sign-Data Algorithm');
legend('Actual signal','Result of noise cancellation',...
       'Location','NorthEast');
xlabel('Time index')
ylabel('Signal values')

When dsp.LMSFilter runs, it uses far fewer multiplication operations than either of the standard LMS algorithms. Also, performing the sign-data adaptation requires only multiplication by bit shifting when the step size is a power of two.

Although the performance of the sign-data algorithm as shown in this plot is quite good, the sign-data algorithm is much less stable than the standard LMS variations. In this noise cancellation example, the processed signal is a very good match to the input signal, but the algorithm could very easily grow without bound rather than achieve good performance.

Changing the weight initial conditions (InitialConditions) and mu (StepSize), or even the lowpass filter you used to create the correlated noise, can cause noise cancellation to fail.

In the standard and normalized variations of the LMS adaptive filter, the coefficients for the adapting filter arise from calculating the mean square error between the desired signal and the output signal from the unknown system, and applying the result to the current filter coefficients. The sign-error LMS (SELMS) algorithm replaces the mean square error calculation by using the sign of the error to modify the filter coefficients.

When the error is positive, the new coefficients are the previous coefficients plus the error multiplied by the step size $\mu$. If the error is negative, the new coefficients are the previous coefficients minus the error multiplied by $\mu$ — note the sign change. When the input is zero, the new coefficients are the same as the previous set.

In vector form, the sign-error LMS algorithm is:

$\mathit{w}\left(\mathit{k}+1\right)=\mathit{w}\left(\mathit{k}\right)+\mu \mathrm{sgn}\left(\mathit{e}\left(\mathit{k}\right)\right)\left(\mathit{x}\left(\mathit{k}\right)\right)$,

where

with vector $\mathit{w}$ containing the weights applied to the filter coefficients and vector $\mathit{x}$ containing the input data. The vector $\mathit{e}$ is the error between the desired signal and the filtered signal. The objective of the SELMS algorithm is to minimize this error.

With a smaller $\mu$, the correction to the filter weights gets smaller for each sample and the SELMS error falls more slowly. A larger $\mu$ changes the weights more for each step so the error falls more rapidly, but the resulting error does not approach the ideal solution as closely. To ensure a good convergence rate and stability, select $\mu$ within the following practical bounds.

where $\mathit{N}$ is the number of samples in the signal. Also, define $\mu$ as a power of two for efficient computation.

Note: How you set the initial conditions of the sign-error algorithm profoundly influences the effectiveness of the adaptation process. Because the algorithm essentially quantizes the error signal, the algorithm can become unstable easily.

A series of large error values, coupled with the quantization process might result in the error growing beyond all bounds. Restrain the tendency of the sign-error algorithm to become unstable by choosing a small step size $\left(\mu \ll 1\right)$ and setting the initial conditions for the algorithm to nonzero positive and negative values.

In this noise cancellation example, set the Method property of dsp.LMSFilter to 'Sign-Error LMS'. This example requires two input data sets:

For the signal, use a sine wave. Note that signal is a column vector of 1000 elements.

signal = sin(2*pi*0.055*(0:1000-1)');

Now, add correlated white noise to signal. To ensure that the noise is correlated, pass the noise through a lowpass FIR filter and then add the filtered noise to the signal.

noise = randn(1000,1);
filt = dsp.FIRFilter;
filt.Numerator = fir1(11,0.4);
fnoise = filt(noise);
d = signal + fnoise;

fnoise is the correlated noise and d is now the desired input to the sign-error algorithm.

To prepare the dsp.LMSFilter object for processing, set the initial conditions of the filter weights (InitialConditions) and mu (StepSize). As noted earlier in this section, the values you set for coeffs and mu determine whether the adaptive filter can remove the noise from the signal path.

In System Identification of FIR Filter Using LMS Algorithm, you constructed a default filter that sets the filter coefficients to zeros. In most cases that approach does not work for the sign-error algorithm. The closer you set your initial filter coefficients to the expected values, the more likely it is that the algorithm remains well behaved and converges to a filter solution that removes the noise effectively.

For this example, start with the coefficients used in the noise filter (filt.Numerator) and modify them slightly so the algorithm has to adapt.

coeffs = (filt.Numerator).'-0.01; % Set the filter initial conditions.
mu = 0.05; % Set the step size for algorithm updating.

With the required input arguments for dsp.LMSFilter prepared, run the adaptation and view the results.

lms = dsp.LMSFilter(12,'Method','Sign-Error LMS',...
   'StepSize',mu,'InitialConditions',coeffs);
[~,e] = lms(noise,d);
L = 200;
plot(0:199,signal(1:200),0:199,e(1:200));
title('Noise cancellation performance by the sign-error LMS algorithm');
legend('Actual signal','Error after noise reduction',...
       'Location','NorthEast')
xlabel('Time index')
ylabel('Signal value')

When the sign-error LMS algorithm runs, it uses far fewer multiplication operations than either of the standard LMS algorithms. Also, performing the sign-error adaptation requires only bit shifting multiples when the step size is a power of two.

Although the performance of the sign-error algorithm as shown in this plot is quite good, the sign-error algorithm is much less stable than the standard LMS variations. In this noise cancellation example, the adapted signal is a very good match to the input signal, but the algorithm could very easily become unstable rather than achieve good performance.

Changing the weight initial conditions (InitialConditions) and mu (StepSize), or even the lowpass filter you used to create the correlated noise, can cause noise cancellation to fail and the algorithm to become useless.

The sign-sign LMS algorithm (SSLMS) replaces the mean square error calculation by using the sign of the input data to change the filter coefficients. When the error is positive, the new coefficients are the previous coefficients plus the error multiplied by the step size $\mu$. If the error is negative, the new coefficients are the previous coefficients minus the error multiplied by $\mu$ — note the sign change. When the input is zero, the new coefficients are the same as the previous set.

In essence, the algorithm quantizes both the error and the input by applying the sign operator to them.

In vector form, the sign-sign LMS algorithm is:

where

Vector $\mathit{w}$ contains the weights applied to the filter coefficients and vector $\mathit{x}$ contains the input data. The vector $\mathit{e}$ is the error between the desired signal and the filtered signal. The objective of the SSLMS algorithm is to minimize this error.

With a smaller $\mu$, the correction to the filter weights gets smaller for each sample and the SSLMS error falls more slowly. A larger $\mu$ changes the weights more for each step, so the error falls more rapidly, but the resulting error does not approach the ideal solution as closely. To ensure a good convergence rate and stability, select $\mu$ within the following practical bounds.

where $\mathit{N}$ is the number of samples in the signal. Also, define $\mu$ as a power of two for efficient computation

Note:

How you set the initial conditions of the sign-sign algorithm profoundly influences the effectiveness of the adaptation process. Because the algorithm essentially quantizes the input signal and the error signal, the algorithm can become unstable easily.

A series of large error values, coupled with the quantization process might result in the error growing beyond all bounds. Restrain the tendency of the sign-sign algorithm to become unstable by choosing a small step size $\left(\mu \ll 1\right)$ and setting the initial conditions for the algorithm to nonzero positive and negative values.

In this noise cancellation example, set the Method property of dsp.LMSFilter to 'Sign-Sign LMS'. This example requires two input data sets:

For the signal, use a sine wave. Note that signal is a column vector of 1000 elements.

signal = sin(2*pi*0.055*(0:1000-1)');

Now, add correlated white noise to signal. To ensure that the noise is correlated, pass the noise through a lowpass FIR filter, then add the filtered noise to the signal.

noise = randn(1000,1);
filt = dsp.FIRFilter;
filt.Numerator = fir1(11,0.4);
fnoise = filt(noise);
d = signal + fnoise;

fnoise is the correlated noise and d is now the desired input to the sign-sign algorithm.

To prepare the dsp.LMSFilter object for processing, set the initial conditions of the filter weights (InitialConditions) and mu (StepSize). As noted earlier in this section, the values you set for coeffs and mu determine whether the adaptive filter can remove the noise from the signal path. In System Identification of FIR Filter Using LMS Algorithm, you constructed a default filter that sets the filter coefficients to zeros. Usually that approach does not work for the sign-sign algorithm.

The closer you set your initial filter coefficients to the expected values, the more likely it is that the algorithm remains well behaved and converges to a filter solution that removes the noise effectively. For this example, you start with the coefficients used in the noise filter (filt.Numerator), and modify them slightly so the algorithm has to adapt.

coeffs = (filt.Numerator).' -0.01; % Set the filter initial conditions.
mu = 0.05;

With the required input arguments for dsp.LMSFilter prepared, run the adaptation and view the results.

lms = dsp.LMSFilter(12,'Method','Sign-Sign LMS',...
   'StepSize',mu,'InitialConditions',coeffs);
[~,e] = lms(noise,d);
L = 200;
plot(0:199,signal(1:200),0:199,e(1:200));
title('Noise cancellation performance by the sign-sign LMS algorithm');
legend('Actual signal','Error after noise reduction',...
       'Location','NorthEast')
xlabel('Time index')
ylabel('Signal value')

When dsp.LMSFilter runs, it uses far fewer multiplication operations than either of the standard LMS algorithms. Also, performing the sign-sign adaptation requires only bit shifting multiples when the step size is a power of two.

Although the performance of the sign-sign algorithm as shown in this plot is quite good, the sign-sign algorithm is much less stable than the standard LMS variations. In this noise cancellation example, the adapted signal is a very good match to the input signal, but the algorithm could very easily become unstable rather than achieve good performance.

Changing the weight initial conditions (InitialConditions) and mu (StepSize), or even the lowpass filter you used to create the correlated noise, can cause noise cancellation to fail and the algorithm to become useless.