Multiplication of gigantic matrices
3 次查看(过去 30 天)
显示 更早的评论
Hi,
I am trying to multiply two gigantic matrices: [1585152 90]*[90 1585152]. MATLAB runs out of memory. Is there any workaround for this?
5 个评论
John D'Errico
2023-3-6
Do you have the memory? The size of your result will be:
1585152*1585152
elements. Multiply that by 8, assuming doubles. Converting to rough gigabytes:
1585152*1585152*8/2^30
So only 18700 gigabytes, or 18.7 terabytes of RAM.
Again, do you have that much RAM? I'm not talking about disk space, where even that would be seriously big. Even in these days of large chunks of cheap RAM, 18 terabytes is flat out significantly big.
Unless your arrays are seriously sparse matrices, even converting them to sparse will not help much.
Far better is to do as has been suggested, to NOT do this. A little linear algebra will show you how to do much with the original arrays. And, worse, when you do that multiplication, you actually make things numerically pooly behaved. There are many good reasons to not do what you want to do. And one good reason why you CANNOT do it, because of the memory requirenemt.
Just because you have a formula that shows that product of matrices, does not mean it is a good idea to write code following that formula.
Steven Lord
2023-3-7
What were you hoping to do with this extremely large matrix? There may be alternate approaches that don't require nearly so much memory. For example, if you were hoping to solve a system using the normal equations don't do that. Use mldivide, \ or one of the iterative methods for solving a system.
回答(1 个)
Allen
2023-3-7
You can use datastores and tall arrays to work with large amounts of data. See references to both.
1 个评论
Bruno Luong
2023-3-7
See John's comment, it need 18.7 Tb to store it, without looking to time to work with huge matrix.
Explicit multiplication is a bad idea that will lead pratically to nowhere.
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Matrix Indexing 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!