Best way to save big matrices

Question

Ingo 2011-12-29

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/24899-best-way-to-save-big-matrices

I'm looking for the best way to save big (both full and sparse) matrices in files. What I need to be able to do is:

- Read only certain entries from the file (certain matrix entries, that is)

- Create a sparse matrix in MATLAB from this file without creating a full matrix first (assuming the file contains a sparse matrix, of course)

The reason is quite simple: The nonsparse matrices will be too big to be stored into memory as a full matrix.

I read about the CDF file format which I'm not yet familiar with, so I don't know if it's possible to create a sparse matrix in Matlab from a CDF File?

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Ingo 2011-12-29

A little elaboration:

Given are n data points and now I want to calculate the (n x n) matrix containing the pairwise distance between all points. This is a full matrix, too big to store it in memory.

The next step would be to make this matrix sparse by, for example, ignoring all values less than a certain treshold. This sparse matrix can be stored in memory.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

the cyclist 2011-12-29

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/24899-best-way-to-save-big-matrices#answer_32719

The most recent versions of MATLAB have the matfile() command, which I believe does at least some of what you want.

3 个评论
显示 1更早的评论隐藏 1更早的评论

the cyclist 2011-12-29

If you do find it works out for you, please "accept" the solution, which may help future users with similar problems.

Ingo 2011-12-30

Hi. Sorry for being late on that!

Yes, matfile works for what I intended to do. However, for my purposes it is just way too slow. I'll have to stick with smaller data sets and try to create the resulting sparse matrices without ever calculating full matrices.

However, matfile() still does solve the problem.

Thanks again and Greetings

Ingo

请先登录，再进行评论。