Matlab crashes with large matrix
9 次查看(过去 30 天)
显示 更早的评论
I need to create a large matrix F=[F1, F2, ... , FN] where each Fn has the same number of rows (about 12,000) but different number of columns. What I am doing now is to create F as a cell array using a loop with element F{n} = Fn. The problem is that when the number of columns of F is very large, Matlab crashes: at some point it shuts down (I run this on a server, and I find matlab closed). I suspect that it shuts down when I convert the cell array into a single matrix, F=cell2mat(F);
Notice that everything works ok when F has a number of columns less than say, say 1,000,000,
Any help is much appreciated. Thanks!
0 个评论
采纳的回答
Walter Roberson
2023-10-25
编辑:Walter Roberson
2023-10-25
Well, that is certainly possible... if you make the arrays large enough then there might just not be enough memory available.
Is it possible to predict the number of columns that will be needed for each Fn before doing the actual callculations? If so then you could consider doing that calculation and totaling them, and pre-allocating the output array, then writing directly into the output array instead of building the cell array. If you pre-build the sizes then you can use cumsum() to create an array that will tell you which indices to write at
As a side note: it is more efficient to use
F = horzcat(F{:});
than F = cell2mat(F); -- but this would not solve your memory problems.
2 个评论
Bruno Luong
2023-10-25
No it's not normal. But when all memory are consumed, many tasks are slowing down and requesred memory get swap to hard drive. The PC are in a "non standard" functioning mode and things can get slippy from there. Even pop up a warning message or windows become a hercules task.
更多回答(1 个)
Bruno Luong
2023-10-25
编辑:Bruno Luong
2023-10-25
For 1e6 column the memory required in Gb for one array is
memgGb = 1e6*12e3*8 / 1e9
And you have the cell array + the concatenate array (single contiguous emory blockà, so you need 192 Gb of free memory.
You might consider using single-class array, write directly to the allocated matrix as Walter suggests or using tall array assuming you store the transposed of the original intended array.
2 个评论
Bruno Luong
2023-10-25
licols is Matt's FEX, but yeah I quite participate to popular the idea. Thanks
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Logical 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!