How can I do a memory efficient sparse matrix multiplication?

16 次查看（过去 30 天）
Steffen
Steffen 2013-11-5
编辑： Matt J 2013-11-6
I have a sparse matrix A (dimension 4000000 x 1000000) and I want to calculate the matrix product:
B = A * A'
This results in: "Out of memory. Type HELP MEMORY for your options."
  2 个评论
Cedric
Cedric 2013-11-5
编辑：Cedric 2013-11-5
What is the density or nnz? What is its size in memory (whos), and are you able to evaluate
B = A.' ;
without generating this out of memory error?
Steffen
Steffen 2013-11-5
16891004 non-zero elements in a 4172241 x 1032306 matrix.
density: 3.921729424555354E-6
Yes, I can calculate B = A.' without generating this error.

请先登录，再进行评论。

请先登录，再回答此问题。

回答（1 个）

Matt J
Matt J 2013-11-5
编辑：Matt J 2013-11-5
If you only plan to use A*A' in matrix multiplication, you might be able to use my ProdCascade class
>> A=sprand(4e6, 1e6,1e-5); c=rand(4e6,1);
>> tic; B=A*A'; B*c; toc;
Out of memory. Type HELP MEMORY for your options.
>> tic; B=ProdCascade({A,A.'}); B*c; toc;
Elapsed time is 7.531522 seconds.
  8 个评论
显示 6更早的评论
Steffen
Steffen 2013-11-6
D=C\Y;
E=C\X;
doesn't work because of memory restrictions. I use the pseudoinverse of C to calculate D and E. I wanted to do the gradient descend to improve the C matrix.
Matt J
Matt J 2013-11-6
编辑：Matt J 2013-11-6
If C is full rank, then pinv(C)*Y is equivalent to D=C\Y so iterative minimization shouldn't improve anything.
Since C has only 5 columns, I can't imagine memory restrictions being a problem.

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Creating and Concatenating Matrices 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by