Out of memory for Blkdiag
2 次查看(过去 30 天)
显示 更早的评论
I have two big matrices ( 5000X329176 and 5000X328658 ) and I want to combine them into one matrix which is in a form of these two in diagonal and all the other elements 0. basically I want blkdiag(A,B).
The problem is even though I made both these guys in int8 variable type and sparse and they take a small size but for blkdiag function, it needs to make a bigger matrix first ,zeros (5000+5000 X 329176+328658) and this guy is by default made in double which takes a huge amount of Ram.
I know e.g. zeros(100000,650000) needs 484 GB memory and sparse(zeros(100000,650000)) because of execution order needs the same amount ! however zeros(100000,650000, 'int8') needs 60GB memory which is available.
Unfortunately, blkdiag does not have this option. So, any idea about solving the problem?
0 个评论
采纳的回答
Matt J
2017-3-28
编辑:Matt J
2017-3-28
If the matrices are sparse, why store them as int8? Why not store them as sparse doubles? If you pre-convert A and B to type sparse, then blkdiag will assemble them without creating non-sparse type matrices, e.g.,
>> A=sprand( 5000,329176,.0001 );
>> B=sprand( 5000,328658,.0001 );
>> C=blkdiag(A,B);
>> Whos C
Name Size Kilobytes Class Attributes
C 10000x657834 10279 double sparse
and sparse(zeros(100000,650000)) because of execution order needs the same amount !
You would never create a sparse matrix this way. You always use one of the following syntaxes, which avoid allocating a full sized matrix,
S=sparse(m,n);
S=sparse(I,J,S);
1 个评论
Matt J
2017-3-28
Vahid Hematian Dehkordi commented:
Thank you so much. I thought sparse logical or sparse int8 should have the minimum size but it seems sparse double is the right way.
更多回答(0 个)
另请参阅
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!