How to filter noise from a time series without losing important informations?

Question

Fabian Lürßen 2021-6-21

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/861900-how-to-filter-noise-from-a-time-series-without-losing-important-informations

评论： Mathieu NOE 2021-7-8

采纳的回答： Mathieu NOE

data.mat

Hello everyone,

as described in the topic I have a time series like this:

Zoomed in you can see there is a lot of noise.

I need to filter the noise in the green rectangle without losing informations like the spike in the green circle.

As can be seen in the frequency domain there is a lot going on.

I have tried smoothing and filtering endlessly with Data Analyser Toobox and the Signal Processing Toolbox. I have tried the Filter Designer just like the filter funktion with various settings like Savitzky-Golay Filter or Exponential Moving Average Filter or Bandpass. But everytime I get rid of the noise I also lose to much important informations.

Can anybody tell if and how I can achive this?

I have attached the data.mat with the time series in case there is someone who likes to try.

Best regards

Fabian

2 个评论
显示无隐藏无

Sharmin Kibria 2021-6-25

编辑：Sharmin Kibria 2021-6-25

Did you try denoising it using the Signal analyzer app? I tried to denoise the signal with symlet 4 wavelet (default setting) and got the attached denoised version. I was able to preserve 98% of the original signal energy in the denoising process. Do you think it is good enough?

Thanks

Sharmin

Fabian Lürßen 2021-6-28

Yes, indeed I tried. But a rest of the nois remains and although 98% of the original signal energy is preserved the form of the curves is changed.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Mathieu NOE 2021-6-28

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/861900-how-to-filter-noise-from-a-time-series-without-losing-important-informations#answer_735045

在 MATLAB Online 中打开

hello

this is my suggestion and the result : as you can see , the large amplitude spikes are not changed by the filtering (perfect overlay with raw data)

clc
clearvars
load('data.mat');
x = data(:,1);
y = data(:,2);
t = data(:,3);
dt = mean(diff(t));
Fs = 1/dt;
ind = find(t>3.9 & t < 4);
t = t(ind);
x = x(ind); 
y = y(ind); 
samples = length(y);
figure(1)
plot(t,x);
%% "smart" data smoothing 
N = 9;
xs = medfilt1(x, N,'truncate');
xs = medfilt1(xs, N,'truncate');
% replace noisy data by smoothed (only low amplitude signal is affected)
ind = find(abs(xs)<1);
xx = x;
xx(ind) = xs(ind);
figure(1)
% plot(t,x,'b',t,xs,'r',t,xx,'g');legend('Raw','Smoothed');
plot(t,x,'b',t,xx,'r');legend('Raw','"smart" Smoothed');
title(['Data samples at Fs = ' num2str(round(Fs)) ' Hz / Smoothed with medfilt1' ]);
grid on

2 个评论
显示无隐藏无

Mathieu NOE 2021-6-28

在 MATLAB Online 中打开

hello

even simpler code , only one stage of filtering , and noisy low amplitude signal simply replaced by zeros

clc
clearvars
load('data.mat');
x = data(:,1);
y = data(:,2);
t = data(:,3);
dt = mean(diff(t));
Fs = 1/dt;
ind = find(t>3.9 & t < 4);
t = t(ind);
x = x(ind); 
y = y(ind); 
samples = length(y);
figure(1)
plot(t,x);
%% "smart" data smoothing 
N = 9;
xs = medfilt1(x, N,'truncate');
% replace noisy data by smoothed (only low amplitude signal is affected)
ind = find(abs(xs)>3);
xx = zeros(size(x));
xx(ind) = x(ind);
figure(1)
% plot(t,x,'b',t,xs,'r',t,xx,'g');legend('Raw','Smoothed');
plot(t,x,'b',t,xx,'r');legend('Raw','"smart" Smoothed');
title(['Data samples at Fs = ' num2str(round(Fs)) ' Hz / Smoothed with medfilt1' ]);
grid on