Speed up my "for" loop

Question

Nurlan Mukanov 2013-9-10

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/86833-speed-up-my-for-loop

编辑： Mazin Mustafa 2016-7-17

Hi Everyone,

Could you please help to speed up this piece of code? What it does, it basically converts 50 Hz data to a 5 Hz data, averaging each 10 values of each field. Second and last field names are not under this conversion. And the last # of values of each field is less than ten, therefore , I use last two lines inside the loop to take care of it.

so, in total there are 11 fieldnames, each containing 12666529 values

tic

for n = [1, 3:length(names)-1]

    for k = 0:count
        data.(names{n})(k+1) = mean(data.(names{n})((1+10*k):(10*(k+1))));
    end
    data.(names{n})(count + 2) = mean(data.(names{n})(total - (total - (count+1)*10 + 1):total));   
    data.(names{n})(count+3:total) = [];
    toc
end

The time I spent right now is 304 secs:

each loop takes around 25 secs

Elapsed time is 25.251421 seconds.

Elapsed time is 50.271446 seconds.

Elapsed time is 75.096751 seconds.

Elapsed time is 100.987307 seconds.

Elapsed time is 126.373710 seconds.

Elapsed time is 151.917404 seconds.

Elapsed time is 177.191153 seconds.

Elapsed time is 202.475424 seconds.

Elapsed time is 227.890811 seconds.

Elapsed time is 253.500255 seconds.

Elapsed time is 278.895494 seconds.

Elapsed time is 304.218428 seconds.

Elapsed time is 304.268410 seconds.

Thanks a lot! Nurlan

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Mazin Mustafa 2016-7-17

编辑：Mazin Mustafa 2016-7-17

Hi,

I think that if you can use vectors instead of the for loop, this might speed up the calculations. Matrix operations are the fastest in MATLAB.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Jan 2013-9-11

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/86833-speed-up-my-for-loop#answer_96404

编辑：Jan 2013-9-12

在 MATLAB Online 中打开

You forgot to pre-allocate the results. Letting the arrays grow iteratively requires a huge amount of memory allocations and copies. For 12666529 values the resulting vector grows 1266652 times and this requires sum(1:1266652) * 8 bytes to be allocated and copied: 802 GigaByte!

So in a first step a pre-allocation is recommended:

result = zeros(1, count + 1);  % Pre-allocate !!!
value  = data.(names{n});      % Faster shortcut
for k = 0:count
   result(k+1) = mean(value((1+10*k):(10*(k+1))));
end

For testing measure the timings with this method at first. Then in a next step the blockwise sum can be calculated in a vectorized form:

result = sum(reshape(data.(names{n})(1:10*(count+1), 10, []), 1) / 10;

4 个评论
显示 2更早的评论隐藏 2更早的评论

Nurlan Mukanov 2013-9-12

在 MATLAB Online 中打开

hey Jan,

In your first comment the last line didn't quite work...(or maybe I was doing something wrong) So, using your code from the first comment I got this(below) But it doesn't save me much...instead of 300sec I get 250secs, which is good, but not so much...

    tic
    names = fieldnames(originalStruct);
    % initialize a empty struct 
    size = [1:count+2]';
    data = struct(names{1},size, names{2},originalStruct.twoField,...
                  names{3},size, names{4},size,...
                  names{5},size, names{6},size,...
                  names{7},size, names{8},size,...
                  names{9},size, names{10},size,...
                  names{11},size, names{12},size,...
                  names{13},size, names{14}, size(end));
count = floor(length(originalStruct.oneField)/10)-1;  % size of each field divided by 10 and minus 1
total = length(originalStruct.oneField); % size of each field
    col = zeros(count+2,1);  % Pre-allocate !!!
    for n = ([1, 3:length(names)-1])
        for k = 0:count
           col(k+1) = mean(originalStruct.(names{n})((1+10*k):(10*(k+1))));
        end
        col(count+2)= mean(originalStruct.(names{n})(total - (total - (count+1)*10+1):total));
        data.(names{n}) = col;
        toc
    end
    toc

I didn't have time yet to work with your last comment... But i think it's close to what I did...

Thanks, Nurlan

Jan 2013-9-12

I do not get it. Instead of calling mean() inside a loop, it is much faster to use reshape to convert the vector to a matrix with 10 rows, calculate the sum() and finally divide by 10.

请先登录，再进行评论。

Answer 2

Ken Atwell 2013-9-11

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/86833-speed-up-my-for-loop#answer_96402

Do you have access to Signal Processing Toolbox? Does resample do any better?

3 个评论
显示 1更早的评论隐藏 1更早的评论

Ken Atwell 2013-9-11

I have not used is much, but from the doc:

First argument is the signal
Second argument is the desired sample rate
Third argument is the current sample rate

It could actually be slower because it may be doing something more mathematically sophisticated than taking the mean, but it is worth trying. More on resampling here.

Nurlan Mukanov 2013-9-12