How can I multiply two big matrices, avoiding out of memory?

20 次查看（过去 30 天）

Xin Liu 2019-8-25

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/477500-how-can-i-multiply-two-big-matrices-avoiding-out-of-memory

评论： Bruno Luong 2019-8-26

for example, I = A*B, in which size(A) = [1024^2, 3], size(B) = [3, 1001^2]. So, size(I) = [1024^2, 1001^2] which could cause out of memory.

I have tried using tall arrays like the following code:

AA = tall(A);

II = AA*B;

I = gather(II);

but the command line still shows error: out of memory.

Sincerely thanks!

%% update %%

Thanks for your answer very much! Originally, I want to calculate this equation:

where

so I write the following matrix equation:

m and n are variable, for example,m = 1024, n=1001^2 whatever. It is worth noting that the two matrices in exp(...) are not too big, but the result of their multiplication is too large to be storaged in memory. I've tried to split the two matrices into several small patches, but it need to input the number of the patches manually rather than automatically, moreover, the for loop could not be avoided in this case.

I have no good ideal currently.

Best regards!

2 个评论
显示无隐藏无

Bruno Luong 2019-8-25

编辑：Bruno Luong 2019-8-25

Storage of your matrix requires 8 Terabytes. Do you have a HD that big?

Xin Liu 2019-8-26

I don't have so big HD - -、, so I'm searching for a better method to avoid big matrix :)

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Bruno Luong 2019-8-26

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/477500-how-can-i-multiply-two-big-matrices-avoiding-out-of-memory#answer_389018

编辑：Bruno Luong 2019-8-26

在 MATLAB Online 中打开

You can process by chunk of smaller (100 here);

% Generate small test data
m = 100;
n = 100^2;
C = rand(m^2,3); % your [alpha,beta,gamma]
XYZ = rand(3,n); % your [x; y; z];
A = rand(1,m^2);
psize = 100; % chunk size
E = zeros(1,n);
count = 0;
while count < n
    p = min(psize,n-count);
    j = count+(1:p);
    E(j) = A*exp(C*XYZ(:,j));
    count = count + p;
end