Faster for loops/or how to cut some time

Question

Happy_Shovel 2021-10-21

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1568733-faster-for-loops-or-how-to-cut-some-time

回答： Jan 2021-10-22

Hello! I have the following problem: I have two arrays, call them a and b. I want to compute the maximum value of a scalar that comes from the following function F, applied to different combinations of a and b.

function Fstat=F(a,b)
T1=length(a); 
T2=length(b);
T=T1+T2; 
eta=[a;b]; 
eta=eta-mean(eta); 
Z=[repmat(-T/T1,T1,1);repmat(T/T2,T2,1)].*eta;
[be,~,r]=regress(eta,[ones(T,1),Z]);
varb=T*mean((Z.*r).^2)/sum(Z.^2)^2;
Fstat=be(2)^2/varb;
end

The combinations, however, are not at random. I need to take windows of their values that go from 20% of the length of the arrays, to 50% of their length, starting from each possible position, and combining them (that is, I don't need random combinations of them, but windows). To put it short, the preallocated possible arrays can be put in cells through the following code:

function Y=using_cells1(a,W)
T=length(a);
for i=1:T-W
    Y{i}=a(i:W+i);
end
end

and:

function [Yc,Yp]=using_cells2(a,b)
Tc=length(a);
Tp=length(b);
lbc=0.2*Tc;
ubc=0.5*Tc;
lbp=0.2*Tp;
ubp=0.5*Tp;
Wc=round(lbc:ubc);
Wp=round(lbp:ubp);
for i=1:length(Wc)
    Yc{i}=using_cells1(a,Wc(i));
end
for i=1:length(Wp)
    Yp{i}=using_cells1(b,Wp(i));
end
end

Which means that I will have two "variables" for both of the arrays, the starting position of the array, and their length. This makes it such that running a loop for the two arrays becomes extremely slow for high dimensions of either of them.

a=randn(50,1);
b=randn(50,1);
tic
[aa,bb]=using_cells2(a,b);
eloopC=length(aa); 
eloopP=length(bb);
for i=1:(eloopC) 
    for j=1:(eloopP)
        for k=1:(length(aa{i}))
            for l=1:(length(bb{j}))
                fstats(i,k,j,l)=F(aa{i}{1,k},bb{j}{1,l});
             end
        end
    end
end
[~, position] = max(fstats(:)); 
[p1,p2,p3,p4] = ind2sub(size(fstats),position);
toc

Is there any way I can cut time?

I posted a similar question before, using sums as an example, and the answers I recieved turned out to be sum specific. So I posted the full function I need to effectively use, I hope there is no problem with that.

2 个评论
显示无隐藏无

Jan 2021-10-22

编辑：Jan 2021-10-22

In you other question the pre-allocation of the output was mentioned already. The editor mentions the problem already by underlining the concerned variables.

Happy_Shovel 2021-10-22

You're certainly right. Preallocating doesn't substantially cut the time though. I sincerely appreciate your help btw! Your previous reply was super helpful!

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Jan 2021-10-22

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1568733-faster-for-loops-or-how-to-cut-some-time#answer_814183

在 MATLAB Online 中打开

a=randn(50,1);
b=randn(50,1);
tic
[aa,bb]=using_cells2(a,b);
eloopC=length(aa); 
eloopP=length(bb);
for i=1:(eloopC) 
    for j=1:(eloopP)
        for k=1:(length(aa{i}))
            for l=1:(length(bb{j}))
                fstats(i,k,j,l)=F(aa{i}{1,k},bb{j}{1,l});
             end
        end
    end
end
[~, position] = max(fstats(:)); 
[p1,p2,p3,p4] = ind2sub(size(fstats),position);
toc
% Tiny modifications to let the code run 10% faster:
% - pre-allocation
% - sum(X)/numel(X) instead of mean(X) in F2()
tic
[aa,bb] = using_cells2(a,b);
eloopC  = length(aa); 
eloopP  = length(bb);
fstats  = zeros(eloopC, eloopP, eloopC, eloopP);  % Pre-allocation
for i = 1:eloopC
    for j = 1:eloopP
        bbj = bb{j};  % Indexing a cell needs time
        for k = 1:length(aa{i})
            aaik = aa{i}{1,k};  % Indexing a cell needs time
            for l = 1:length(bb{j})
                fstats(i,k,j,l) = F2(aaik, bbj{l});
            end
        end
    end
end
[~, position] = max(fstats(:)); 
[p1,p2,p3,p4] = ind2sub(size(fstats),position);
toc
function Y=using_cells1(a,W)
T = length(a);
Y = cell(1, T - W);
for i = 1:T-W
    Y{i} = a(i:W+i);
end
end
function [Yc,Yp]=using_cells2(a,b)
Tc=length(a);
Tp=length(b);
lbc=0.2*Tc;
ubc=0.5*Tc;
lbp=0.2*Tp;
ubp=0.5*Tp;
Wc=round(lbc:ubc);
Wp=round(lbp:ubp);
Yc = cell(1, length(Wc));
for i = 1:length(Wc)
    Yc{i} = using_cells1(a, Wc(i));
end
Yp = cell(1, length(Wp));
for i = 1:length(Wp)
    Yp{i} = using_cells1(b, Wp(i));
end
end
function Fstat=F(a,b)
T1=length(a); 
T2=length(b);
T=T1+T2; 
eta=[a;b]; 
eta=eta-mean(eta); 
Z=[repmat(-T/T1,T1,1);repmat(T/T2,T2,1)].*eta;
[be,~,r]=regress(eta,[ones(T,1),Z]);
varb=T*mean((Z.*r).^2)/sum(Z.^2)^2;
Fstat=be(2)^2/varb;
end
function Fstat = F2(a,b)
T1  = length(a); 
T2  = length(b);
T   = T1+T2; 
eta = [a; b]; 
eta = eta - sum(eta) / numel(eta); 
Z   = [repmat(-T/T1,T1,1); repmat(T/T2,T2,1)] .* eta;
[be,~,r] = regress(eta, [ones(T,1),Z]);
varb  = T * sum((Z.*r) .^ 2) / numel(Z) / sum(Z.^2)^2;
Fstat = be(2)^2 / varb;
end

I do not have the Statistic and Machienlerning Toolbox, so I cannot run regress() locally. Use the profiler to find the bottleneck of the code. If this is regress(), try to write a leaner function to solve the calculations.