How to solve out of memory error when constructing sparse matrix

3 次查看（过去 30 天）
lala
lala 2023-9-24
评论： lala 2023-9-27
I am construcing a stiffness matrix in finite element computation.
I have got i, j, and v vectors as input arguments for sparse.
Both i and j are 222263280-by-1 int32 vectors while v is a 222263280-by-1 double vector. The stiffness matrix is 1202454-by-1202454.
However I encountered out of memory error when I run the command S = sparse(i,j,v,m,n).
Are there any solutions or suggestions to address this problem?
Why does not sparse support tall array? I think this problem can be solved if tall array used.
Thanks.

请先登录，再回答此问题。

采纳的回答

Bruno Luong
Bruno Luong 2023-9-24
编辑：Bruno Luong 2023-9-24
You might try the syntax
nz = ... ; < 222263280
S = sparse(i,j,v,m,n,nz);
with nz is the estimation of non-zero elements of your matrix. If you build P1 FEM, it's the number of edges of your setup. You might use Euler formula to estimte it : https://www.mathsisfun.com/geometry/eulers-formula.html
or using
nz = size(unique([i(:) j(:)], 'row'),1)
Note that the byte size required to store a sparse matrix S is
if isa(S,'double')
elbyte = 8;
else
elbyte = 1;
end
Sbytesize = 8*(size(S,2)+1) + ... % Jc array
8*nnzmax(S) + ... % Ir array
elbyte*nnzmax(S) % data value array
So if you deal well, your S will take less than 3.566 Gb (I assumz nz < 222263280),
>> (1202454+1)*8+222263280*8+222263280*8
ans =
3.5658e+09
plus the space of I, J, V. It will require 7 Gb to build it. If your have other suff around 16 Gb become a little tight.

更多回答（2 个）

Matt J
Matt J 2023-9-24
It's a 5 GB matrix. Are you sure you have enough RAM?
  4 个评论
显示 2更早的评论
lala
lala 2023-9-24
No, I need help. 😭
since all i, j, and v can be maintained in RAM, why sparse failed? Does sparse need more memory than i, j, and v?
Torsten
Torsten 2023-9-24
If I clear all variables except i, j, and v for sparse, it can construct the sparse matrix successfully.
This should show you that it's not possible with the RAM available.

请先登录，再进行评论。

John D'Errico
John D'Errico 2023-9-24
编辑：John D'Errico 2023-9-24
Ran in:
Remember that i and j and v also take memory.
(222263280*4*2 + 222263280*8)/1024^3
ans = 3.3120
So those three variables alone require 3.3 GB. And then the sparse array would require 5 GB. Next, you need to factor in that MATLAB will require CONTIGUOUS memory for any arrays. And MATLAB itself, and your OS take up memory. So I can understand that you may just have room to create that array, if you clear everything else. But you still won't be able to do virtually anything with it once built.
This is not really a problem of needing support for tall arrays to build a sparse matrix. That is just a crutch that will encourage you to build larger arrays yet. If you are going to work with these problems on that computer, then you desperately need to get more RAM. RAM is cheap these days. Honestly, I would suggest 16 GB of RAM is just barely a viable amount these days if you are doing any serious work in MATLAB.
John's law: Computer problems always expand to just a little larger than the memory available on the largest computer you can find.
(So, yes, according to the law I just gave you, even if you do get more RAM, you will quickly decide to solve larger problems yet. Since who wants to solve small problems? But it gives you a few years of grace.)

类别

在 Help Center 和 File Exchange 中查找有关 Sparse Matrices 的更多信息

标签

产品


版本

R2023a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by