Changing large matrices by not completely loading them into memory

2 次查看（过去 30 天）
Moritz
Moritz 2015-6-18
评论： Walter Roberson 2015-6-18
Hi,
I'm attempting to modify very large matrices (single, 50e3 x 50e3), which don't make sense to load into the memory. I was wondering what you could recommend me as a data handling strategy? I thought ideally I could always load a let us say 100x100 square modify it and write it back. My working machine uses a SSD connected via M2 so it should be relatively speedy (however of course not nearly as fast as RAM). What suggestions do you have?
Thanks,
Moritz

请先登录，再回答此问题。

回答（2 个）

Stephen23
Stephen23 2015-6-18
编辑：Stephen23 2015-6-18
You should read TMW's own advice on working with big data:
http://www.mathworks.com/discovery/big-data-matlab.html
And in particular you might find memmapfile to be of significant interest to you:
http://www.mathworks.com/help/matlab/ref/memmapfile.html
  1 个评论
Walter Roberson
Walter Roberson 2015-6-18
Or instead of memmapfile, save the .mat with -v7.3 and then use matFile objects to read in portions of the array.

请先登录，再进行评论。

Alessandro
Alessandro 2015-6-18
Did you check the sparse command out?
  1 个评论
Moritz
Moritz 2015-6-18
Yes I did. However, I believe this only works if you have a considerable amount of zero elements. In my case however, the amount of zero elements are < 5%.

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Large Files and Big Data 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by