Slow down when accessing arrays
13 次查看(过去 30 天)
显示 更早的评论
I have some code that really slows down when storing results. Code 2 runs orders of magnitude faster than Code1, especially as k increase in size (e.g. k = 500). I would need to store the results from each iteration i, but it seems very costly computationally. Why is this? Is there a better way to code it.
Code 1:
Q=zeros(k,k,N)
for i=1:N
Q(:,:,i) = Q(:,:,i) + e(i)^2*XtXt;
end
Code 2:
Q=zeros(k,k)
for i=1:N
Q = Q + e(i)^2*XtXt;
end
Here is the entire function if it is helpful. There are a lot of loops, but I found a clever way to eliminate any of them.
function varBhat = NeweyWest(e, X, L, invXX )
% e = (T x N)
% X = (T x k)
% L = scalar
% invXX = (k x k)
% Determine the size of the matrix of regressors
[T, k] =size(X);
% Determine the number of units
N = size(e,2);
% Calculate the Newey-West autocorrelation consistent covariance
Q = zeros(k,k,N);
for l = 0:L
w_l = 1-l/(L+1);
for t = l+1:T
if (l==0) % This calculates the S_0 portion
XtXt = compute_XtXt(X,t); %(k x k)
% reuse the computed matrix for all of N units
for i=1:N
Q(:,:,i) = Q(:,:,i) + e(t,i)^2 *XtXt;
end
else % This calculates the off-diagonal terms
XtXl = compute_XtXl(X,t,l,w_l); % (k x k)
% reuse the computed matrix for all of the generators
for i=1:N
Q(:,:,i) = Q(:,:,i) + e(t,i)*e(t-l,i)*XtXl;
end
end
end
end
Q = 1/(T-k) * Q;
% Calculate Newey-White standard errors (loops over each unit)
varBhat = finalNW(T, X, Q, invXX, N);
end
2 个评论
回答(2 个)
Nathaniel
2012-5-25
How much memory in your computer, and how big is N? If you run out of available physical memory, then you will spend a lot of time waiting while your O/S swaps to and from your hard disk.
Image Analyst
2012-5-26
Well yeah! If the second case you're just overwriting a scalar N times so you're just doing N operations. In the first case you're assigning a kxk matrix N times. And with k = 500 that means 250,000 times N memory locations are being assigned. So I don't doubt that case 1 would run hundreds of times slower simply because you're assigning and storing hundreds of thousands more elements. What is the value of N? If it's less than about 5, then this should still happen in just a few seconds. But if N is also like 500, then it could take a very long time, and you might even run out of memory.
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Logical 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!