reduce memory for matrix multiplication

2 次查看（过去 30 天）
behzad
behzad 2018-10-22
评论： Bruno Luong 2018-10-24
Hi dudes, I have 2 vectors multiplied by a 1e4*1e4 matrix as below example:
A = 1:1:10000;
B = rand(10000);
C = (1:1:10000)';
an I have a result matrix like D, each array of which are obtained by the multiplication of these matrices as below:
D(each array) = A*(B*C);
the size of D matrix is 11*11 or larger, meaning that I should do this multiplication 121 times or more consuming large memory and time. Any Idea how to optimize the multiplication process?
  19 个评论
显示 17更早的评论
behzad
behzad 2018-10-22
"... This is a most pertinent..."
Unfortunately I am not familiar with c programming enough. But I try to find out what is going on in the mex file.
James Tursa
James Tursa 2018-10-22
编辑：James Tursa 2018-10-22
So, would you way that your pseudo-code is really something like this:
A = zeros(11,11); D = A;
F = some large matrix (fixed value for the loop)
k1 = some integer (fixed value for the loop)
k2 = some integer (fixed value for the loop)
for q = 1:200
for p = 1:1:121
A = rand(1,10000); % some calculation that changes each time
B = F(k1+1:k1+10000,k2+1:k2+10000);
C = rand(10000,1); % some calculation that changes each time
D(p) = A*(B*C);
end
A = D + A;
end
I.e., B can obviously be pulled out of the loop in the above code and the data copy only done once. It is critical to know which things change during the loop and which things don't.
We can help you with the C mex stuff, but only if it really makes sense, and we won't know that until we know exactly how the pseudo code looks.

请先登录，再进行评论。

请先登录，再回答此问题。

回答（2 个）

Matt J
Matt J 2018-10-22
编辑：Matt J 2018-10-22
One possibility might be to zero-pad A and C, embedding them in larger sparse vectors to be multiplied with F. This way you don't have to pull B out of F and allocate separate memory for it.
Za=spalloc(1,size(F,1), 1e4);
Zc=spalloc(size(F,2),1,1e4);
for p = 1:121
A = ...
Apad=Za;
Apad(k1+1:k1+1e4)=A;
C=...
Cpad=Zc;
Cpad(k2+1:k2+1e4)=C;
D(p) = Apad*(F*Cpad);
end
  9 个评论
显示 7更早的评论
Matt J
Matt J 2018-10-23
OK, how about this:
N=4e4+1;
F=ones(N);
x=ones(1e4,1); x(N)=0;
xs=sparse(x);
tic; %full
ans=x.'*(F*x)+7;
toc;
%Elapsed time is 0.428139 seconds.
tic; %sparse
ans=xs.'*(F*xs)+7;
toc;
%Elapsed time is 0.374981 seconds.
Bruno Luong
Bruno Luong 2018-10-23
The result with my PC on your code
FULL: Elapsed time is 0.278291 seconds.
SPARSE: Elapsed time is 0.360029 seconds.
Of course you can try to increase N where at the limit become favorable more and more to sparse.

请先登录，再进行评论。

Bruno Luong
Bruno Luong 2018-10-22
编辑：Bruno Luong 2018-10-24
If you are using R2018a/b you can use the MEX file attached here to avoid forming B. The matrix-vector product is replaced by a loop on vector x vector, it can be slower than MATLAB (twice according to my test).
[m,n] = size(F);
AFk = zeros(1,size(C,1));
for k=1:length(AFk)
Fk = mxCreateSharedMatrix2018(F,k1+(k2+k-1)*m,size(A,2),1);
AFk(k) = A*Fk;
end
mxUnshareMatrix2018(Fk,[],1);
clear Fk
D = AFk*C; % or set D(k1,k2)?

类别

在 Help Center 和 File Exchange 中查找有关 Sparse Matrices 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by