Out of memory for Blkdiag
1 次查看(过去 30 天)
显示 更早的评论
I have two big matrices ( 5000X329176 and 5000X328658 ) and I want to combine them into one matrix which is in a form of these two in diagonal and all the other elements 0. basically I want blkdiag(A,B).
The problem is even though I made both these guys in int8 variable type and sparse and they take a small size but for blkdiag function, it needs to make a bigger matrix first ,zeros (5000+5000 X 329176+328658) and this guy is by default made in double which takes a huge amount of Ram.
I know e.g. zeros(100000,650000) needs 484 GB memory and sparse(zeros(100000,650000)) because of execution order needs the same amount ! however zeros(100000,650000, 'int8') needs 60GB memory which is available.
Unfortunately, blkdiag does not have this option. So, any idea about solving the problem?
0 个评论
采纳的回答
Matt J
2017-3-28
编辑:Matt J
2017-3-28
If the matrices are sparse, why store them as int8? Why not store them as sparse doubles? If you pre-convert A and B to type sparse, then blkdiag will assemble them without creating non-sparse type matrices, e.g.,
>> A=sprand( 5000,329176,.0001 );
>> B=sprand( 5000,328658,.0001 );
>> C=blkdiag(A,B);
>> Whos C
Name Size Kilobytes Class Attributes
C 10000x657834 10279 double sparse
and sparse(zeros(100000,650000)) because of execution order needs the same amount !
You would never create a sparse matrix this way. You always use one of the following syntaxes, which avoid allocating a full sized matrix,
S=sparse(m,n);
S=sparse(I,J,S);
1 个评论
Matt J
2017-3-28
Vahid Hematian Dehkordi commented:
Thank you so much. I thought sparse logical or sparse int8 should have the minimum size but it seems sparse double is the right way.
更多回答(0 个)
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Matrix Indexing 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!