Out of memory for Blkdiag

1 次查看(过去 30 天)
I have two big matrices ( 5000X329176 and 5000X328658 ) and I want to combine them into one matrix which is in a form of these two in diagonal and all the other elements 0. basically I want blkdiag(A,B).
The problem is even though I made both these guys in int8 variable type and sparse and they take a small size but for blkdiag function, it needs to make a bigger matrix first ,zeros (5000+5000 X 329176+328658) and this guy is by default made in double which takes a huge amount of Ram.
I know e.g. zeros(100000,650000) needs 484 GB memory and sparse(zeros(100000,650000)) because of execution order needs the same amount ! however zeros(100000,650000, 'int8') needs 60GB memory which is available.
Unfortunately, blkdiag does not have this option. So, any idea about solving the problem?

采纳的回答

Matt J
Matt J 2017-3-28
编辑:Matt J 2017-3-28
If the matrices are sparse, why store them as int8? Why not store them as sparse doubles? If you pre-convert A and B to type sparse, then blkdiag will assemble them without creating non-sparse type matrices, e.g.,
>> A=sprand( 5000,329176,.0001 );
>> B=sprand( 5000,328658,.0001 );
>> C=blkdiag(A,B);
>> Whos C
Name Size Kilobytes Class Attributes
C 10000x657834 10279 double sparse
and sparse(zeros(100000,650000)) because of execution order needs the same amount !
You would never create a sparse matrix this way. You always use one of the following syntaxes, which avoid allocating a full sized matrix,
S=sparse(m,n);
S=sparse(I,J,S);
  1 个评论
Matt J
Matt J 2017-3-28
Vahid Hematian Dehkordi commented:
Thank you so much. I thought sparse logical or sparse int8 should have the minimum size but it seems sparse double is the right way.

请先登录,再进行评论。

更多回答(0 个)

类别

Help CenterFile Exchange 中查找有关 Matrix Indexing 的更多信息

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by