您正在关注此问题

您将在关注的内容源中看到更新。
您可能会收到电子邮件，具体取决于您的通信预设项。

I tried Hampel filtering to discard the bad data, but it was not fruitful. How to avoid/filter

1 次查看（过去 30 天）

显示更早的评论

Ismita 2022-4-19

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1699615-i-tried-hampel-filtering-to-discard-the-bad-data-but-it-was-not-fruitful-how-to-avoid-filter

⋮

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1699615-i-tried-hampel-filtering-to-discard-the-bad-data-but-it-was-not-fruitful-how-to-avoid-filter

评论： Mathieu NOE 2022-4-21

Dear all,

I have attached the data plot. I tried Hampel filtering to discard the bad data, but it was not fruitful. How to avoid/filter

this bad data ?

10 个评论
显示 8更早的评论隐藏 8更早的评论

Rik 2022-4-19

此评论的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1699615-i-tried-hampel-filtering-to-discard-the-bad-data-but-it-was-not-fruitful-how-to-avoid-filter#comment_2110160

⋮

链接

此评论的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1699615-i-tried-hampel-filtering-to-discard-the-bad-data-but-it-was-not-fruitful-how-to-avoid-filter#comment_2110160

What exactly defines bad data in your case?

KSSV 2022-4-19

此评论的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1699615-i-tried-hampel-filtering-to-discard-the-bad-data-but-it-was-not-fruitful-how-to-avoid-filter#comment_2110170

⋮

链接

此评论的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1699615-i-tried-hampel-filtering-to-discard-the-bad-data-but-it-was-not-fruitful-how-to-avoid-filter#comment_2110170

Read about isoutlier.

Ismita 2022-4-19

此评论的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1699615-i-tried-hampel-filtering-to-discard-the-bad-data-but-it-was-not-fruitful-how-to-avoid-filter#comment_2111655

⋮

链接

此评论的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1699615-i-tried-hampel-filtering-to-discard-the-bad-data-but-it-was-not-fruitful-how-to-avoid-filter#comment_2111655

编辑：Ismita 2022-4-19

@Rik if you see the plot, there are some peaks for corresponding value. How can I avoid those points (e.g., in this plot E_D is >12000 for one case, E_D>4000 for another case, etc......)? Thank you

@KSSV thank you

Rik 2022-4-20

此评论的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1699615-i-tried-hampel-filtering-to-discard-the-bad-data-but-it-was-not-fruitful-how-to-avoid-filter#comment_2112305

⋮

链接

此评论的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1699615-i-tried-hampel-filtering-to-discard-the-bad-data-but-it-was-not-fruitful-how-to-avoid-filter#comment_2112305

The function KSSV suggested should do the trick in this case. Feel free to post a follow up comment if you have questions.

Ismita 2022-4-20

此评论的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1699615-i-tried-hampel-filtering-to-discard-the-bad-data-but-it-was-not-fruitful-how-to-avoid-filter#comment_2112680

⋮

链接

此评论的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1699615-i-tried-hampel-filtering-to-discard-the-bad-data-but-it-was-not-fruitful-how-to-avoid-filter#comment_2112680

在 MATLAB Online 中打开

when I am running the following code in Matlab online it shows the plot, but it is not working while running in installed Matlab. Could you please suggest me the solution? Thanks again.

"Code:

x = 1:10;
A = [60 59 49 49 58 200 61 57 48 58];
[TF,L,U,C] = isoutlier(A);
plot(x,A,x(TF),A(TF),'x',x,L*ones(1,10),x,U*ones(1,10),x,C*ones(1,10))
legend('Original Data','Outlier','Lower Threshold','Upper Threshold','Center Value')

Mathieu NOE 2022-4-20

此评论的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1699615-i-tried-hampel-filtering-to-discard-the-bad-data-but-it-was-not-fruitful-how-to-avoid-filter#comment_2112830

⋮

链接

此评论的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1699615-i-tried-hampel-filtering-to-discard-the-bad-data-but-it-was-not-fruitful-how-to-avoid-filter#comment_2112830

this code works on my R2020b

Rik 2022-4-20

此评论的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1699615-i-tried-hampel-filtering-to-discard-the-bad-data-but-it-was-not-fruitful-how-to-avoid-filter#comment_2112845

⋮

链接

此评论的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1699615-i-tried-hampel-filtering-to-discard-the-bad-data-but-it-was-not-fruitful-how-to-avoid-filter#comment_2112845

What release are you using?

Mathieu NOE 2022-4-21

此评论的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1699615-i-tried-hampel-filtering-to-discard-the-bad-data-but-it-was-not-fruitful-how-to-avoid-filter#comment_2114565

⋮

链接

此评论的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1699615-i-tried-hampel-filtering-to-discard-the-bad-data-but-it-was-not-fruitful-how-to-avoid-filter#comment_2114565

在 MATLAB Online 中打开

@Mst Ismita Tasnim

if you have an older release the second code example is for you

A = repmat([60 59 49 49 58 200 61 57 48 58],[1 10]);
nA = numel(A);
x = 1:nA;
% %% original code 
% [TF,L,U,C] = isoutlier(A);
% figure(1),plot(x,A,x(TF),A(TF),'dr',x,L*ones(1,nA),x,U*ones(1,nA),x,C*ones(1,nA))
% legend('Original Data','Outlier','Lower Threshold','Upper Threshold','Center Value')
%% alternative method
upper_threshold = 5; % above mean smoothed curve
lower_threshold = 25; % below mean smoothed curve
% apply smoothing (here twice)
N = 15;
B = myslidingavg(A, N); % or use smooth or smoothdata depending of what you have
B = myslidingavg(B, N); % or use smooth or smoothdata depending of what you have
TF = (A>B+upper_threshold) | (A<B-lower_threshold);
figure(2),plot(x,A,x,B,x,B-lower_threshold,x,B+upper_threshold,x(TF),A(TF),'dr')
legend('Original Data','Smoothed Data','Lower Threshold','Upper Threshold','Outlier')
function out = myslidingavg(in, N)
%   OUTPUT_ARRAY = MYSLIDINGAVG(INPUT_ARRAY, N)
%
%  The function 'slidingavg' implements a one-dimensional filtering, applying a sliding window to a sequence. Such filtering replaces the center value in
%  the window with the average value of all the points within the window. When the sliding window is exceeding the lower or upper boundaries of the input
%  vector INPUT_ARRAY, the average is computed among the available points. Indicating with nx the length of the the input sequence, we note that for values
%  of N larger or equal to 2*(nx - 1), each value of the output data array are identical and equal to mean(in).
%
%  *  The input argument INPUT_ARRAY is the numerical data array to be processed.
%  *  The input argument N  is the number of neighboring data points to average over for each point of IN.
%
%  *  The output argument OUTPUT_ARRAY is the output data array.
if (isempty(in)) | (N<=0)                                              % If the input array is empty or N is non-positive,
 disp(sprintf('SlidingAvg: (Error) empty input data or N null.'));     % an error is reported to the standard output and the
 return;                                                               % execution of the routine is stopped.
end % if
if (N==1)                                                              % If the number of neighbouring points over which the sliding 
 out = in;                                                             % average will be performed is '1', then no average actually occur and
 return;                                                               % OUTPUT_ARRAY will be the copy of INPUT_ARRAY and the execution of the routine
end % if                                                               % is stopped.
nx   = length(in);             % The length of the input data structure is acquired to later evaluate the 'mean' over the appropriate boundaries.
if (N>=(2*(nx-1)))                                                     % If the number of neighbouring points over which the sliding 
 out = mean(in)*ones(size(in));                                        % average will be performed is large enough, then the average actually covers all the points
 return;                                                               % of INPUT_ARRAY, for each index of OUTPUT_ARRAY and some CPU time can be gained by such an approach.
end % if                                                               % The execution of the routine is stopped.
out = zeros(size(in));         % In all the other situations, the initialization of the output data structure is performed.
if rem(N,2)~=1                 % When N is even, then we proceed in taking the half of it:
 m = N/2;                      % m = N     /  2.
else                           % Otherwise (N >= 3, N odd), N-1 is even ( N-1 >= 2) and we proceed taking the half of it:
 m = (N-1)/2;                  % m = (N-1) /  2.
end % if
for i=1:nx,                                                 % For each element (i-th) contained in the input numerical array, a check must be performed:
  dist2start = i-1;         %  index distance from current index to start index (1)
  dist2end = nx-i;          %  index distance from current index to end index (nx)
  if dist2start<m | dist2end<m        % if we are close to start / end of data, reduce the mean calculation on centered data vector reduced to available samples
      dd = min(dist2start,dist2end);    % min of the two distance (start or end)
  else
      dd = m;
  end % if
  out(i) = mean(in(i-dd:i+dd));     % mean of centered data , reduced to available samples at both ends of the data vector
end % for i
end

Ismita 2022-4-21

此评论的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1699615-i-tried-hampel-filtering-to-discard-the-bad-data-but-it-was-not-fruitful-how-to-avoid-filter#comment_2114670

⋮

链接

此评论的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1699615-i-tried-hampel-filtering-to-discard-the-bad-data-but-it-was-not-fruitful-how-to-avoid-filter#comment_2114670

Thank you so much

Mathieu NOE 2022-4-21

此评论的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1699615-i-tried-hampel-filtering-to-discard-the-bad-data-but-it-was-not-fruitful-how-to-avoid-filter#comment_2115095

⋮

链接

此评论的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1699615-i-tried-hampel-filtering-to-discard-the-bad-data-but-it-was-not-fruitful-how-to-avoid-filter#comment_2115095