How to fill zeros and NaNs with the average of the previous nonzero consecutive values (part 2)

Question

Margarida 2023-4-3

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1940249-how-to-fill-zeros-and-nans-with-the-average-of-the-previous-nonzero-consecutive-values-part-2

评论： Margarida 2023-4-7

在 MATLAB Online 中打开

Hi, here's a challenge that is too hard for me and I need enlightment, a few weeks ago I posted this question:

"So, i have a column like this:

T = [0;0;1;2;3;4;NaN;4;3;0;0;0;NaN;4;2;3;0;2;0];

And everytime I have a NaN or a zero, I would like to transform them in the average of the previous nonzero consecutive values (averages in bold):

T = [0;0;1;2;3;4;2.5;4;3;3.5;3.5;3.5;3.5;4;2;3;3;2;2];

I cannot figure out how I can do this in an efficient way (without for loops, because the column has thousands of rows)."

This was the only answer I got which helped me a lot:

"Using this FEX download,

https://www.mathworks.com/matlabcentral/fileexchange/78008-tools-for-processing-consecutive-repetitions-in-vectors

T = [0;0;1;2;3;4;NaN;4;3;0;0;0;NaN;4;2;3;0;2;0];
idx=find(T,1);
[stem,T]=deal(T(1:idx-1),T(idx:end));
G=groupTrue(~isnan(T) & T~=0);
[~,~,lengths]=groupLims(groupTrue(~G),1);
T(~G)=repelem( groupFcn(@mean,T,G) ,lengths);
T=[stem;T]
T'

"

However, I think sometime my data has too many NaNs or zeros and I get erros like this:

Error using splitapply
Group numbers must be a vector of positive integers, and cannot be a sparse vector.
Error in groupFcn (line 30)
    [varargout{1:nargout}]=splitapply(func,varargin{:},G);
Error in PowerDependency (line 130)
        T(~G)=repelem( groupFcn(@mean,T,G) ,lengths);
        

Making me think that it's risky to use these types of functions that I can't understand when I have errors (lol).

Anyways my help request is can anyone figure out another way? Doesn't have to be duper efficient and instant, I can handle a few seconds of run time, but not too much like verifying row by row :(

I would be extremely happy if someone could help, cheers!

14 个评论
显示 12更早的评论隐藏 12更早的评论

dpb 2023-4-3

编辑：dpb 2023-4-3

"I think sometime my data has too many NaNs or zeros and I get erros like this..."

I'd venture it's not enough, rather than too many. If there were an empty result that would produce the message.

Don't guess, set a breakpoint and see what is actually going on when the error occurs; we can't replicate the problem as you've not supplied a test case nor are all the elements in the above code snippet defined...

What's the answer for the first two elements in your input T array that are zero and so by the definition need to be replaced but there's no finite, nonzero value preceding them?

"...I can handle a few seconds of run time, but not too much like verifying row by row ..."

I'd venture a loop would be just fine; the code above is just hiding all the looping it's doing internally to find the groups and then use splitapply which is a looping construct internally, as well.

Just locate the start/stop locations and walk through them will probably be at least as fast if not faster...

dpb 2023-4-6

My first foray into the fray in response to "I think sometime my data has too many NaNs or zeros and I get erros like this..." was to observe that

:I'd venture it's not enough, rather than too many. If there were an empty result that would produce the message."

"Just sayin..." <vbg>

Margarida 2023-4-7

yes haha i guess you were right

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Peter Perkins 2023-4-6

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1940249-how-to-fill-zeros-and-nans-with-the-average-of-the-previous-nonzero-consecutive-values-part-2#answer_1210849

在 MATLAB Online 中打开

Just for fun: a varfun soln. Would look similar using grouptransform.

Step one is to define groups of elements by finding runs of non-NaN/non-zero followed by runs of NaN/zero.

x = [0;0;1;2;3;4;NaN;4;3;0;0;0;NaN;4;2;3;0;2;0];
x(x == 0) = NaN;
i = isnan(x);
starts = [false; diff(i) < 0];
group = cumsum(starts);
T = table(x,i,starts,group)
T = 19×4 table
xistartsgroup___________________

    NaN    true     false       0  
    NaN    true     false       0  
      1    false    true        1  
      2    false    false       1  
      3    false    false       1  
      4    false    false       1  
    NaN    true     false       1  
      4    false    true        2  
      3    false    false       2  
    NaN    true     false       2  
    NaN    true     false       2  
    NaN    true     false       2  
    NaN    true     false       2  
      4    false    true        3  
      2    false    false       3  
      3    false    false       3  

Step 2 is to replace NaNs with the group means.

T2 = varfun(@myFun,T,InputVariables="x",GroupIngVariable="group")
T2 = 19×3 table
groupGroupCountmyFun_x______________________

      0          2           NaN  
      0          2           NaN  
      1          5             1  
      1          5             2  
      1          5             3  
      1          5             4  
      1          5           2.5  
      2          6             4  
      2          6             3  
      2          6           3.5  
      2          6           3.5  
      2          6           3.5  
      2          6           3.5  
      3          4             4  
      3          4             2  
      3          4             3  
T.xFilled = T2.myFun_x
T = 19×5 table
xistartsgroupxFilled__________________________

    NaN    true     false       0        NaN  
    NaN    true     false       0        NaN  
      1    false    true        1          1  
      2    false    false       1          2  
      3    false    false       1          3  
      4    false    false       1          4  
    NaN    true     false       1        2.5  
      4    false    true        2          4  
      3    false    false       2          3  
    NaN    true     false       2        3.5  
    NaN    true     false       2        3.5  
    NaN    true     false       2        3.5  
    NaN    true     false       2        3.5  
      4    false    true        3          4  
      2    false    false       3          2  
      3    false    false       3          3  

function x = myFun(x)
m = mean(x,"omitmissing");
x(isnan(x)) = m;
end

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

How to fill zeros and NaNs with the average of the previous nonzero consecutive values (part 2)

14 个评论
显示 12更早的评论隐藏 12更早的评论

回答（1 个）

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

另请参阅

类别

标签

Community Treasure Hunt

How to fill zeros and NaNs with the average of the previous nonzero consecutive values (part 2)

14 个评论 显示 12更早的评论隐藏 12更早的评论

回答（1 个）

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

另请参阅

类别

标签

Community Treasure Hunt

14 个评论
显示 12更早的评论隐藏 12更早的评论

0 个评论
显示 -2更早的评论隐藏 -2更早的评论