How can I reduce the execution time when calculating mean values within a 3 times loop?

Question

Melina Maria 2014-7-1

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/139635-how-can-i-reduce-the-execution-time-when-calculating-mean-values-within-a-3-times-loop

评论： Melina Maria 2014-7-1

Deal all, good morning!

I would like to ask your help regarding to a computational problem that I have. I'm trying to calculate 10 minutes values on a daily basis. So my original file is something like this: [year DoY time(minutes) value]

2010 132 150 1.52

2010 132 151 2.5

2010 132 153 3.4

...

2013 365 1440 0.2

So I have to calculate the ten minute mean values for each year and each day of the year. I created a code with with 3 loops in it. The first one for the year, the second one for the day and the third one for the minutes. It looks more or less like this: for y=2010:2013 idy=find(year==y) if ~isempty(idy) for d=1:366 idd=find(doy(idy)==d) if ~isempty(idd) for m=1:10:1440 idm=find(m(idd(idy))==m if ~isempty(idm) mean=[mean; y d m mean(value(idy(idd(idm))))]; end end end end end end

Now, where is the problem?!

It takes days to run and at the end matlab stucks so I have to exit and I can never get to the results...

Any sugestions would be really really APRECIATED!!!

Thank you in advance.

Cheers,

Melina

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Melina Maria 2014-7-1

在 MATLAB Online 中打开

Sorry, the code looks like ...

for year=2004:2014

    idy=[];
    idy=find(NILU.y==year);
    if ~isempty(idy)
for day=1:366
    iddoy=[];
    iddoy=find(NILU.doy(idy)==day);
    if ~isempty(iddoy)
for lo=1:10:1440
    idmi=[];
    idmi=find(NILU.dt(idy(iddoy))>=lo & NILU.dt(idy(iddoy))<lo+10);
    if~isempty(idmi)
    TENNILU=[TENNILU;year day mean(NILU.dt(idy(iddoy(idmi)))) mean(NILU.sza(idy(iddoy(idmi)))) mean(NILU.tot(idy(iddoy(idmi)))) mean(NILU.par(idy(iddoy(idmi)))) mean(NILU.alg(idy(iddoy(idmi)))) mean(NILU.flmine(idy(iddoy(idmi)))) length(idy(iddoy(idmi)))];
    end
end
    end
end
    end
end

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Titus Edelhofer 2014-7-1

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/139635-how-can-i-reduce-the-execution-time-when-calculating-mean-values-within-a-3-times-loop#answer_143120

在 MATLAB Online 中打开

Hi,

I would suggest to preallocate TENNILU and take some of the operations out of the loops:

TENNILU = zeros((2014-2004+1)*366*144, 9);
counter = 0;
for year = ...
  for day=1:366
    iddoy = find(NILU.doy(idy)==day);
    if ~isempty(iddoy)
      idydoy = idy(iddoy);
      for lo=1:10:1440
        idmi=find(NILU.dt(idydoy))>=lo & NILU.dt(idydoy))<lo+10);
        if ~isempty(idmi)
          counter = counter + 1;
          id = idy(iddoy(idmi);
          TENNILU(counter,:) = [year day mean(NILU.dt(id)) mean(NILU.sza(id) ...];
        end
     ...
end
% delete the empty rows at the end
TENNILU(counter+1:end,:) = [];

Does this help to reduce the time?

Titus

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Melina Maria 2014-7-1

Thank you Titus!

I've got your point and I'm gonna try it!

Melina.

P.S. Would it be convinient to set TENnilu=[ ] at the beginning, before the first loop instead of setting TENNILU = zeros((2014-2004+1)*366*144, 9)?

请先登录，再进行评论。

Answer 2

amanita 2014-7-1

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/139635-how-can-i-reduce-the-execution-time-when-calculating-mean-values-within-a-3-times-loop#answer_143122

You can also use a parfor somewhere (if it is possible) but you will have to change your code a little bit.

3 个评论
显示 1更早的评论隐藏 1更早的评论

amanita 2014-7-1

You need independent jobs, so if you use parfor in your outer loop you will need to create N TENNILU's, where N is the total number of years (e.g you can create a cell for TENNILU, preallocate TENNILU{year} after the parfor year and set your counter to zero). If you use parfor you will also need to create a parallel pool (parpool)

Melina Maria 2014-7-1

Thank you!

请先登录，再进行评论。

Answer 3

Andrei Bobrov 2014-7-1

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/139635-how-can-i-reduce-the-execution-time-when-calculating-mean-values-within-a-3-times-loop#answer_143142

在 MATLAB Online 中打开

one way

d = [2010 132 150 1.52
2010 132 151 2.5
2010 132 153 3.4
2012 20 289 4.69
2013 365 1440 0.2];
sdate = addtodate(datenum(d(:,1),1,1),d(:,2) + d(:,3)/1440,'day');
b = datevec(sdate(1));
c = ceil(b(end)*.1)*10;
dte = datenum([[b(1:4),floor(b(5)*.1)*10,0];[c(1:4),ceil(c(5)*.1)*10,0]]);
z = addtodate(sdate(1),(0:10:ceil(diff(dte)*144)*10)','minute');
out = [datevec(z),accumarray(ii,d(:,end),size(z),@mean)];